Senior Web Data Extraction Specialist

1 week ago

Surat, Gujarat, India beBeeWebScraping Full time ₹ 14,00,000 - ₹ 18,00,000

Key Roles and Responsibilities:

  • Develop a high-performance HTTP-first crawler using Scrapy or aiohttp with Playwright as a fallback for JavaScript-heavy pages.
  • Implement differential sitemap processing and conditional GET requests (ETag/Last-Modified) to optimize incremental runs.
  • Create an efficient 'needs JS?' classifier based on HTML length, JSON-LD presence, and data-product markers to dynamically route HTTP and Playwright requests.
  • Enforce per-domain rate limiting and backoff mechanisms (2-4 concurrent/domains; automatic lowering on 429/503 errors).
  • Integrate URL normalization, canonicalization, and de-duplication, respecting hash-based deduping for PDFs.
  • Handle PDF discovery and download by first sending HEAD requests to eliminate duplicates; implement size and concurrency caps, and utilize SHA-256 keys.
  • Apply resource budget constraints on Playwright browser automation (block images/fonts/analytics; kill outliers based on size/CPU/time metrics).
  • Integrate third-party APIs as primary sources: manage authentication (API keys/OAuth2), pagination, and rate limits; unify API and crawl outputs.
  • Own the automation and orchestration of scheduled runs (Airflow/Temporal/Celery or cron jobs), idempotent retries, and alerting.
  • Establish per-domain selectors (YAML) with verification checks on hold-outs; re-train only when performance drops.
  • Maintain observability metrics: per-site field coverage, error rates, retries, average page time, and PDF success rates.

  • Web Crawler Specialist

    2 weeks ago

    Surat, Gujarat, India beBeeWebCrawler Full time ₹ 9,00,000 - ₹ 15,00,000

    We are seeking a skilled Web Crawler Specialist to build and maintain web crawlers, extract valuable insights from the web, and ensure data quality.The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.Key Responsibilities:Maintain and enhance...

  • Data Information Specialist

    2 weeks ago

    Surat, Gujarat, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Information SpecialistAbout the Role:We are seeking a highly skilled data information specialist to join our team. The ideal candidate will have experience in designing and implementing efficient data extraction systems.Design, develop, and maintain robust data scraping solutions to extract structured and unstructured data from various...

  • Senior Web Services Architect

    1 week ago

    Surat, Gujarat, India beBeeIntegration Full time ₹ 80,00,000 - ₹ 1,20,00,000

    Web Integration Specialist"> The role of Web Integration Specialist is crucial in maintaining seamless interactions between internal systems and external partners. We seek an individual with expertise in integrating web services, leveraging JSON as the primary data format.

  • Web Designer

    2 weeks ago

    Surat, Gujarat, India Geek Web Solution Full time ₹ 1,50,000 - ₹ 4,00,000 per year

    Job Title: Web DesignerLocation: Surat (Onsite)Experience: 1–4 YearsJob Type: Full-timeAbout the Role:We are looking for a creative and detail-oriented Web Designer to join our team. The ideal candidate should have strong expertise in designing visually appealing, user-friendly, and responsive websites. You will be responsible for converting ideas into...

  • Senior Business Insights Specialist

    1 week ago

    Surat, Gujarat, India beBeeInsights Full time ₹ 1,00,00,000 - ₹ 1,40,00,000

    Job Overview:We are seeking a highly skilled Senior Business Insights Specialist to drive strategic decision-making through data-driven insights.Key Responsibilities:Analyze large datasets to uncover patterns and trends in loan and retail banking markets.Design and execute SQL queries to extract, transform, and validate complex data sets.Develop and maintain...

  • Senior Web Application Automation Specialist

    2 weeks ago

    Surat, Gujarat, India beBeeAutomation Full time ₹ 8,00,000 - ₹ 12,00,000

    We are on the hunt for a skilled Web Developer to join our long-term freelance project.About the MissionThis strategic data system will require extracting and structuring information from various online sources, including dynamic content, custom headers, request simulation, automation flows, and more.Key RequirementsExtensive experience with Playwright,...

  • Data Scientist Leader

    6 days ago

    Surat, Gujarat, India beBeeEngineer Full time ₹ 8,00,000 - ₹ 15,00,000

    Lead Data EngineerThe ideal candidate will play a key role in developing and maintaining our data extraction systems. Job Description:Web Scraping & Crawling: Develop scalable web scraping systems to extract structured and unstructured data from websites, APIs, and public datasets.Responsibilities:Data Parsing & Cleaning: Normalize scraped data, remove...

  • Order Management and Data Expert

    6 days ago

    Surat, Gujarat, India beBeeDataMigration Full time ₹ 15,55,555 - ₹ 21,11,111

    Job Title:Senior Business Analyst Order Management and Data SpecialistJob DescriptionWe are seeking an experienced Senior Business Analyst Order Management and Data Specialist to join our team.The successful candidate will have a strong background in business analysis, with a focus on order management and data migration from external tools to ServiceNow.The...

  • Automation Specialist

    1 week ago

    Surat, Gujarat, India beBeeDataMining Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Opportunity:This is a challenging position for a skilled Data Mining Analyst to work with us. We are seeking an experienced professional with expertise in automating data extraction processes from web platforms.Key Responsibilities:Design, develop, and maintain robust web scraping solutions to extract structured and unstructured data from various...

  • Senior Business Intelligence Specialist

    2 weeks ago

    Surat, Gujarat, India beBeeDataAnalyst Full time ₹ 6 - ₹ 9

    Job Title: Senior Data AnalystJob DescriptionWe are looking for an experienced and skilled Senior Data Analyst to join our team. In this role, you will be responsible for designing, developing, and maintaining robust data pipelines to support our growing data needs.The ideal candidate will have a minimum of 5 years of experience in data management,...