Web Crawler Development Specialist

1 day ago

Dombivli, Maharashtra, India beBeeCrawling Full time ₹ 90,00,000 - ₹ 1,20,00,000
Job Opportunity: Web Crawler Development

We are seeking a skilled Web Crawler Development Specialist who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality.

About the Role:

  • Maintain and enhance existing web scraping and data crawling projects.
  • Develop and refine crawlers using Python-based tools and frameworks.
  • Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content.
  • Clean, validate, and integrate extracted data into downstream storage systems.
  • Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.).
  • Optimize crawling efficiency and ensure compliance with web crawling best practices.
  • Collaborate with cross-functional teams to improve data acquisition strategies.

Required Skills & Qualifications:

  • Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy).
  • Strong knowledge of browser automation tools such as Playwright or Selenium.
  • Solid understanding of HTML, CSS, and selector languages (XPath/CSS).
  • Experience in handling anti-scraping challenges and ensuring robust data extraction.
  • Familiarity with distributed scraping techniques and data pipelines.
  • Ability to troubleshoot and optimize web crawlers for performance and reliability.
  • Strong analytical and problem-solving skills with attention to detail.
  • Excellent communication and inter-personal skills.

Benefits of Working with Us:

We offer a dynamic work environment with opportunities for growth and development. Our team is passionate about delivering innovative solutions that make a real impact.

Other Requirements:

A fast internet connection, a modern computer with at least 8 GB of RAM, and a reliable power supply are essential for this role.

  • Senior Web Solutions Specialist

    6 days ago

    Dombivli, Maharashtra, India beBeeWebDeveloper Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Title:Senior Web Solutions SpecialistAbout the Role:We're seeking a highly skilled Senior Web Solutions Specialist to join our team. This is an exciting opportunity to showcase your expertise in designing and developing high-quality web solutions that meet our clients' needs.Main Responsibilities:Create custom themes, plugins, and extensions for various...

  • Full Stack Web Development Specialist

    1 week ago

    Dombivli, Maharashtra, India beBeeWebDevelopment Full time ₹ 4,00,000 - ₹ 5,00,000

    Job Description:As a skilled Web Development Specialist, you will play a pivotal role in designing, developing, and maintaining functional, user-friendly, and visually appealing websites on multiple platforms. Your primary responsibility will be to collaborate with clients to understand website design, functionality, and technical specifications.You will...

  • Web Development Specialist

    4 days ago

    Dombivli, Maharashtra, India beBeeExpert Full time ₹ 12,00,000 - ₹ 18,00,000

    Job Title:Web Development ExpertDescription:This role is open for a Web Development Expert with expertise in Shopify and WordPress to design, develop, and maintain responsive websites. The ideal candidate should have experience in customizing themes, optimizing performance, integrating third-party apps, and ensuring security.The successful candidate will...

  • Web Development Specialist

    1 day ago

    Dombivli, Maharashtra, India beBeeFrontend Full time ₹ 50,00,000 - ₹ 70,00,000

    Job Title: Frontend Web DeveloperWe're seeking an experienced frontend web developer to create fast, responsive, and pixel-perfect websites using WordPress.About the Role:In this position, you will be responsible for converting high-fidelity mockups into fully functional websites that meet the highest standards of performance, accessibility, and SEO...

  • Web Development Specialist

    2 days ago

    Dombivli, Maharashtra, India beBeeFrontend Full time ₹ 9,00,000 - ₹ 12,00,000

    About Web DevelopmentWe are seeking a skilled Frontend Developer to play a pivotal role in shaping our company's online presence. This is an exceptional opportunity for an ambitious professional looking to drive digital transformation and innovation.

  • Chief Web Development Specialist

    5 days ago

    Dombivli, Maharashtra, India beBeeDeveloper Full time ₹ 6,00,000 - ₹ 12,00,000

    Web Developer RoleWe are seeking a skilled Web Developer to join our team.The ideal candidate will be responsible for creating user interface components, collaborating with back-end developers, and designing responsive web applications.Experience in Front-End Development and Responsive Web Design is essential.Knowledge of Back-End Web Development and Web...

  • Web Development Specialist

    6 days ago

    Dombivli, Maharashtra, India beBeeSoftware Full time ₹ 32,00,000 - ₹ 40,00,000

    Key Responsibilities of Senior Software Engineer:Develop high-performance web applications using ReactJS, OpenLayers, DeckGL and other libraries.Convert visual designs to UI elements with an acute eye for detail and design fidelity.Make web applications responsive and cross-browser compatible.Optimize application performance and reduce page loading speed.As...

  • Web Development Specialist

    2 days ago

    Dombivli, Maharashtra, India beBeeDesign Full time ₹ 35,00,000 - ₹ 60,00,000

    Do you aspire to build captivating web pages and seek hands-on experience working on real-world projects? We invite you to join a dynamic internship where you'll have the opportunity to design and develop live websites for innovative startups as part of our portfolio in the UK. What You Will Be Doing - Collaborate with a creative team to bring ideas to...

  • Web Development Specialist

    3 days ago

    Dombivli, Maharashtra, India beBeeSoftware Full time ₹ 1,20,00,000 - ₹ 2,40,00,000

    As a seasoned Front-End Engineer, you will play a pivotal role in shaping our digital landscape. Our organization thrives on innovation and collaboration, where autonomy and craftsmanship are highly valued.A glimpse into your responsibilities:Design and develop scalable web applications using cloud-native architecture patterns.Collaborate with product...

  • Web Development Expert

    5 days ago

    Dombivli, Maharashtra, India beBeeDevelopment Full time ₹ 10,00,000 - ₹ 15,00,000

    Grow your career as a web development expert in a dynamic and innovative environment. Assist in creating cutting-edge front-end and back-end web solutions using WordPress.Our team is looking for a talented individual with strong knowledge of WordPress, hands-on experience with HTML, CSS, JavaScript, and PHP, familiarity with MySQL database management, and...