Web Crawler Specialist

4 days ago

Anand, Gujarat, India beBeeProfessional Full time ₹ 20,00,000 - ₹ 25,00,000

We are seeking a talented web crawling professional to build and maintain advanced web crawlers.

The ideal candidate will have strong Python programming skills, expertise in web scraping frameworks and browser automation tools, and the ability to handle anti-scraping mechanisms effectively.

Key Responsibilities:

  • Maintain and enhance existing web scraping and data crawling projects to ensure optimal efficiency.
  • Develop and refine crawlers using Python-based tools and frameworks, incorporating machine learning algorithms for enhanced data extraction.
  • Utilize browser automation tools (e.g., Playwright, Selenium) to efficiently handle dynamic content.
  • Clean, validate, and integrate extracted data into downstream storage systems, ensuring seamless data flow.
  • Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.) to prevent system downtime.

Required Skills & Qualifications:

  • Proficiency in Python and 2 years of work experience with web scraping frameworks (especially Scrapy).
  • Strong knowledge of browser automation tools such as Playwright or Selenium.
  • Solid understanding of HTML, CSS, and selector languages (XPath/CSS).
  • Experience in handling anti-scraping challenges and ensuring robust data extraction.
  • Familiarity with distributed scraping techniques and data pipelines.

Benefits:

Since this is a completely remote position, you will require -

High-speed internet connectivity for video calls and efficient work.

A capable business-grade computer with modern processor, 8 GB+ of RAM, and no other obstacles to uninterrupted, efficient work.

Headphones with clear audio quality.

Stable power connection and backups in case of internet/power failure.

Additional Requirements:

Ability to collaborate closely with cross-functional teams to improve data acquisition strategies and optimize crawling efficiency.

  • Expert Data Integrator

    1 week ago

    Anand, Gujarat, India beBeeIntegration Full time ₹ 15,00,000 - ₹ 25,00,000

    Integration SpecialistWe are seeking an experienced Integration Specialist to join our team.Develop expertise in integrating systems, designing and building data crawlers, and creating effective packages for data validation and transformation.The ideal candidate will possess strong technical skills in ETL processes, databases (SQL Server, MySQL), and...

  • High Performance Data Engineer

    4 hours ago

    Anand, Gujarat, India beBeeSoftware Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Web Scraping Engineer Job Description">About this Role:Design and implement a high-throughput data ingestion pipeline across hundreds of domains.Owning the crawling/extraction layer end-to-end, including HTTP-first crawling with Playwright fallback, per-domain learned selectors, and reliable PDF handling.This role spans crawling (discovering &...

  • Web Interface Specialist

    4 days ago

    Anand, Gujarat, India beBeeWeb Full time ₹ 30,00,000 - ₹ 60,00,000

    Job OpportunityWe are seeking a talented Web Interface Specialist to join our team. As a key member of our development unit, you will be responsible for designing and building engaging user interfaces, ensuring seamless cross-browser compatibility, and optimizing web pages for speed and scalability.About the RoleIn this role, you will work closely with our...

  • Web Development Specialist

    1 week ago

    Anand, Gujarat, India beBeeDevelopment Full time ₹ 8,00,000 - ₹ 12,00,000

    Job Title:Web Development SpecialistJob Description:We are seeking a skilled professional to join our team as a Web Development Specialist. The ideal candidate will possess strong knowledge of web development principles, experience with content management systems, and a solid understanding of search engine optimization best practices.The successful candidate...

  • Web Developer Specialist

    1 week ago

    Anand, Gujarat, India beBeeDevelopment Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Web Developer SpecialistWe are seeking a skilled Web Developer Specialist to join our team. This role involves designing and implementing high-quality web applications using various technologies.This position is ideal for an individual who enjoys staying up-to-date with emerging technologies, collaborating with cross-functional teams, and ensuring...

  • Data Scientist for Web Scraping

    1 week ago

    Anand, Gujarat, India beBeeData Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Data Extraction SpecialistWe are seeking a skilled Data Extraction Specialist to automate data extraction processes from web platforms using Python, Selenium, Pandas, SQL, and APIs.

  • Web Applications Specialist

    4 days ago

    Anand, Gujarat, India beBeefrontend Full time ₹ 12,00,000 - ₹ 25,00,000

    As a Frontend Developer, you will play a pivotal role in shaping the digital landscape by implementing web design and development principles to build stable and efficient web applications.About the RoleThis is a full-time, on-site position for a skilled developer located in Gurugram.The successful candidate will be responsible for translating UI/UX design...

  • Expert Web Developer

    3 days ago

    Anand, Gujarat, India beBeeDevelopment Full time ₹ 90,00,000 - ₹ 1,20,00,000

    Web Performance ExpertAt Textellent, we are seeking a skilled web performance expert to join our marketing team. As the go-to specialist for maintaining and improving our custom WordPress website, you will be responsible for implementing updates, optimizing conversion rate & site performance, managing content changes, and providing web analytics insights...

  • Web Design Specialist

    6 days ago

    Anand, Gujarat, India beBeeWordpressDeveloper Full time ₹ 8,00,000 - ₹ 12,00,000

    Job Title: WordPress DeveloperAs a key member of our digital team, you will be responsible for designing and developing visually appealing websites using the WordPress platform. Your expertise in web design and development will play a crucial role in creating engaging user experiences for our clients.The ideal candidate will have a solid understanding of...

  • Ecommerce Web Specialist

    2 weeks ago

    Anand, Gujarat, India beBeeWebDeveloper Full time ₹ 1,92,000 - ₹ 2,16,000

    Shopify Developer RoleWe specialize in e-commerce, marketing, and web development for global clients.This role involves working on exciting U.S.-based projects, developing and customizing Shopify stores for client needs.Main Responsibilities:Develop responsive and user-friendly web designs.Work on theme customization, Liquid coding, and Shopify API...