Web Crawler Developer

2 days ago

Davangere, Karnataka, India beBeeData Full time ₹ 6,00,000 - ₹ 10,00,000
Job Overview:

We are seeking a skilled Web Crawler Developer to join our team. As a junior professional, you will be responsible for designing and implementing web crawlers, extracting valuable insights from the web, and ensuring data quality.

The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.

Our company specializes in AI-powered data extraction and automation, delivering highly accurate firmographic and enterprise insights across numerous domains. We enable businesses to automate workflows, reduce manual rework, and access high-quality data at scale.

About the Role:
  • Maintain and enhance existing web scraping and data crawling projects.
  • Develop and refine crawlers using Python-based tools and frameworks.
  • Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content.
  • Clean, validate, and integrate extracted data into downstream storage systems.
  • Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.).
  • Optimize crawling efficiency and ensure compliance with web crawling best practices.
Required Skills & Qualifications:
  • Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy).
  • Strong knowledge of browser automation tools such as Playwright or Selenium.
  • Solid understanding of HTML, CSS, and selector languages (XPath/CSS).
  • Experience in handling anti-scraping challenges and ensuring robust data extraction.
  • Familiarity with distributed scraping techniques and data pipelines.
  • Ability to troubleshoot and optimize web crawlers for performance and reliability.
  • Strong analytical and problem-solving skills with attention to detail.
  • Excellent communication and inter-personal skills.
Your Benefits:

You will have the opportunity to work with a cutting-edge technology platform, collaborate with experienced professionals, and contribute to the development of innovative data extraction solutions. Our company offers a supportive and dynamic work environment, promoting continuous learning and growth.

In this role, you will have the chance to develop your skills in web crawling, data extraction, and AI-powered automation. You will be part of a talented team dedicated to delivering high-quality results and making a real impact in the industry.

This is an excellent opportunity for a motivated and detail-oriented individual to join our team and take their career to the next level. If you are passionate about data extraction, web crawling, and AI-powered automation, we encourage you to apply for this exciting position.

  • Web Development Specialist

    6 days ago

    Davangere, Karnataka, India beBeeDeveloper Full time ₹ 30,00,000 - ₹ 60,00,000

    At Prolync, we empower businesses and individuals through intelligent digital solutions.Job OverviewThis role is designed to provide hands-on experience with web technologies.Key ResponsibilitiesAssist in front-end and back-end web development tasks.Design responsive web designs and implement web development projects.Support the team in achieving project...

  • Senior Web Developer

    1 week ago

    Davangere, Karnataka, India beBeeSoftware Full time ₹ 7,50,000 - ₹ 15,00,000

    About our company, we are a growing startup delivering modern software solutions for clients.Key Responsibilities:We are seeking a skilled developer to design and maintain responsive websites using web development tools.The successful candidate will perform regular updates, backups, and performance optimization.Troubleshooting website issues and implementing...

  • Web Application Developer

    2 days ago

    Davangere, Karnataka, India beBeeFrontend Full time ₹ 15,00,000 - ₹ 25,00,000

    Our team is seeking a highly skilled Frontend Engineer to join our ranks.This key role plays a pivotal part in the development and maintenance of our web applicationsuser-facing aspects. The successful candidate will have a deep understanding of core web technologiesand be proficient in modern JavaScript frameworks.This position requires strong proficiency...

  • Chief Web Developer

    2 weeks ago

    Davangere, Karnataka, India beBeeDeveloper Full time ₹ 12,00,000 - ₹ 18,00,000

    We are seeking an experienced and skilled web professional to lead our front-end development efforts.Key Responsibilities:Lead the design, development, and maintenance of scalable and responsive web interfaces using HTML5, CSS3, and Bootstrap.Guide junior developers, conduct code reviews, and foster best practices in coding standards.Translate high-fidelity...

  • Senior Web Application Developer

    7 hours ago

    Davangere, Karnataka, India beBeeProfessional Full time ₹ 6,00,000 - ₹ 12,00,000

    Web Development ProfessionalThis role is about collaborating with a team to develop web applications and contribute to web design and functionality enhancements. The ideal candidate will have experience in front-end and back-end development, proficiency in programming languages, and strong problem-solving skills.Collaborate with the design and development...

  • Front-End Web Developer

    5 days ago

    Davangere, Karnataka, India beBeeWebDevelopment Full time ₹ 25,00,000 - ₹ 50,00,000

    Job Opportunity:Grownex is a cutting-edge branding and marketing agency dedicated to empowering businesses to stand out, scale rapidly, and remain unforgettable.Our Services:Brand StrategyVisual IdentityDigital MarketingSocial MediaContent MarketingWebsite DesignUI/UXVideo ProductionGraphic DesignGrowth CampaignsKey Responsibilities:This role involves...

  • Expert Web Developer

    2 days ago

    Davangere, Karnataka, India beBeeDeveloper Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Web Development ExpertWe are looking for a skilled web development expert to join our team. As a key member of our product team, you will play a crucial role in designing and developing high-performance web applications.The ideal candidate will have a strong grasp of modern front-end and back-end technologies, including HTML5, CSS3, SASS/SCSS,...

  • Web Page Designer/Developer

    2 days ago

    Davangere, Karnataka, India beBeeDesigner Full time ₹ 7,20,000 - ₹ 10,80,000

    Job Title:Web Page Designer/Developer">As a skilled Web Page Designer/Developer, you will be responsible for designing and developing engaging web pages that meet marketing and sales requirements. You will play a crucial role in leveraging HubSpot's features and capabilities to support our overall business objectives and enhance our customer experience.Main...

  • Web Development Intern

    6 days ago

    Davangere, Karnataka, India beBeeInternship Full time ₹ 40,00,000 - ₹ 80,00,000

    Job Opportunity:We are seeking an experienced and skilled intern to join our web development team. As a key member of the team, you will gain hands-on experience in developing and customizing websites using WordPress and Shopify.Key Responsibilities:Assist in designing and implementing front-end solutions using HTML, CSS, JavaScript, and React.Learn and...

  • E-commerce Web Developer

    1 week ago

    Davangere, Karnataka, India beBeeEcommerce Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: E-commerce Web DeveloperBaheri Media is seeking a skilled e-commerce web developer to manage website changes and automation requirements. The ideal candidate will have 5+ years of experience in web development (Shopify, WordPress, and custom websites) and 2+ years of process automation using tools like Make.com.Key Responsibilities:Maintain and...