Data Driven Web Crawler Specialist

2 days ago

Meerut, Uttar Pradesh, India beBeeWebCrawler Full time ₹ 8,00,000 - ₹ 12,00,000

As a Junior Web Crawling Engineer, you will be responsible for designing and implementing web crawlers to extract valuable insights from the web. You will work on building and maintaining complex web scraping projects, utilizing Python-based tools and frameworks.

Key Responsibilities:
  • Maintain and enhance existing web scraping and data crawling projects.
  • Develop and refine crawlers using Python-based tools and frameworks.
  • Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content.
  • Clean, validate, and integrate extracted data into downstream storage systems.
  • Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.).
  • Optimize crawling efficiency and ensure compliance with web crawling best practices.
  • Collaborate with cross-functional teams to improve data acquisition strategies.
Requirements:
  • Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy).
  • Strong knowledge of browser automation tools such as Playwright or Selenium.
  • Solid understanding of HTML, CSS, and selector languages (XPath/CSS).
  • Experience in handling anti-scraping challenges and ensuring robust data extraction.
  • Familiarity with distributed scraping techniques and data pipelines.
  • Ability to troubleshoot and optimize web crawlers for performance and reliability.
  • Strong analytical and problem-solving skills with attention to detail.
  • Excellent communication and inter-personal skills.
Infrastructure Requirements:

You will require the following infrastructure for this position:

  • High-speed internet connectivity for video calls and efficient work.
  • Capable business-grade computer (e.g., modern processor, 8 GB+ of RAM).
  • Headphones with clear audio quality.
  • Stable power connection and backups in case of internet/power failure.

This is a remote position that requires self-motivation and discipline to deliver high-quality results. If you are passionate about web development and data analysis, we encourage you to apply for this exciting opportunity.

  • Data-Driven Web Performance Specialist

    4 days ago

    Meerut, Uttar Pradesh, India beBeeAnalytics Full time ₹ 17,50,000 - ₹ 19,90,000

    Web Analyst Job OpportunityWe are seeking a highly skilled and data-driven individual to lead the end-to-end tracking and analytics setup for our web platform. This role is responsible for designing and implementing a comprehensive measurement framework, ensuring accurate data collection, and delivering actionable insights that drive business performance and...

  • Senior Web Data Specialist

    6 days ago

    Meerut, Uttar Pradesh, India beBeeDataMiningAnalyst Full time ₹ 8,00,000 - ₹ 12,00,000

    We are seeking a skilled Data Mining Analyst to join our team in Delhi. The ideal candidate will be experienced in automating data extraction processes from web platforms using Python, Selenium, and APIs.Key Responsibilities:Design, develop, and maintain robust web scraping solutions to extract structured and unstructured data from various websites and...

  • Data Driven Marketing Expertise

    4 days ago

    Meerut, Uttar Pradesh, India beBeeData Analyst Full time ₹ 10,00,000 - ₹ 15,00,000

    Business Intelligence SpecialistThe role requires strong analytical skills to interpret marketing data, derive insights, and support strategic decision-making for business growth.Key Responsibilities:Collect and analyze data from multiple marketing channels (digital campaigns, CRM, web analytics, etc.).Build dashboards and reports to track KPIs, campaign...

  • AI-Driven Web Development Expert

    6 days ago

    Meerut, Uttar Pradesh, India beBeeFrontend Full time ₹ 15,00,000 - ₹ 25,00,000

    OverviewWe are seeking a seasoned Frontend Developer to spearhead the development of cutting-edge web applications utilizing modern JavaScript frameworks and libraries.About You6-8 years of experience in building and deploying responsive web applications using modern JavaScript frameworks/librariesStrong understanding of micro-frontend architecture and...

  • Python Developer

    7 days ago

    Meerut, Uttar Pradesh, India beBeeScraping Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Python Developer - Web Scraping SpecialistWe are seeking a skilled Python developer to join our team. As a web scraping specialist, you will design and develop scalable frameworks using Scrapy, ensuring clean, structured, and scalable data delivery.Familiarity with Scrapy is mandatory. Additional libraries such as BeautifulSoup, Selenium,...

  • Web Automation Specialist

    6 days ago

    Meerut, Uttar Pradesh, India beBeeAutomation Full time ₹ 50,000 - ₹ 70,000

    Job Title: Web Automation SpecialistWe are seeking an experienced developer to design and develop an intelligent web automation system. This system should be capable of interacting with websites, performing smart actions, and extracting data automatically.The role involves combining AI techniques with web crawling/automation tools to create a scalable and...

  • Data Insights Specialist

    7 days ago

    Meerut, Uttar Pradesh, India beBeeDataInsights Full time ₹ 8,00,000 - ₹ 12,00,000

    Job Title: Data Insights SpecialistKey ResponsibilitiesCraft and maintain robust web scraping solutions to extract structured and unstructured data from diverse sources.Pursue optimization of scraping workflows to ensure efficient, accurate, and scalable data extraction.Engage with APIs to gather and integrate data, upholding rate limits and authentication...

  • Web Developer Specialist

    6 days ago

    Meerut, Uttar Pradesh, India beBeeDeveloper Full time ₹ 3,60,000 - ₹ 4,80,000

    Job Title: Web Developer SpecialistWe are seeking a skilled and experienced web developer to join our team as a specialist. The successful candidate will be responsible for managing and enhancing our websites during night-shift hours.Key ResponsibilitiesMaintain and update website content (blogs, events, podcasts, resources).Design and develop new pages,...

  • Data Driven Operations Specialist

    7 days ago

    Meerut, Uttar Pradesh, India beBeeBusiness Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Technical Business Analyst - EDP PlatformThis role is focused on supporting operations across process improvement, data platform engineering, and automation.To be successful, you will need to analyze EDP platform workflows and operational processes to identify optimization opportunities. This includes developing comprehensive KPIs and performance dashboards...

  • Web Application Security Specialist

    5 days ago

    Meerut, Uttar Pradesh, India beBeeSecurity Full time ₹ 15,00,000 - ₹ 20,00,000

    Web Application Security SpecialistWe are seeking a highly skilled Web Application Security Specialist to manage security rules and configurations for web applications.Tune web application firewalls (WAFs) to reduce false positives and enable protection against OWASP Top 10 threats.Monitor web traffic anomalies, bot activity, and take preventive actions to...