Web Crawling

2 weeks ago

Remote, India FullStackTechies Full time ₹ 36,00,000 - ₹ 1,08,00,000 per year

Web Crawling & Data Extraction Engineer (WFH)

Experience: 1–7 Years

Location: Remote (Work from Home)

Mode of Engagement: Full-time

No of Positions: 3 to 8

Educational Qualification: Bachelor's degree in Computer Science, IT, or related field

Industry: IT / Software Services / Data & AI

Notice Period: Immediate Joiners

What We Are Looking For

  • Strong hands-on experience in web crawling, scraping, and automated data extraction.
  • Experience working with Playwright, Puppeteer, Selenium, and Python Requests for scraping JS-heavy and dynamic websites.
  • Good understanding of HTML DOM, XPaths, JSON APIs, and handling pagination, infinite scroll, and dynamic page loads.
  • Ability to manage sessions, cookies, headers, token-based flows, and bypass basic anti-bot protections.
  • Experience building automated extraction scripts that run on schedules or pipelines.
  • Ability to write clean, structured, and optimized Python code for crawling and automation.

Responsibilities

  • Build and maintain web crawlers & automated extraction scripts using Playwright/Puppeteer/Selenium/Requests.
  • Extract data from complex, JavaScript-rendered, or protected websites.
  • Handle dynamic elements, redirects, headers, authentication, and anti-bot mitigations.
  • Clean, validate, and structure extracted data (JSON, CSV, DB).
  • Use SQL/NoSQL databases to store extracted content.
  • Maintain automation scripts, scheduling, retries, logging, and monitoring for stable crawling.
  • Work closely with internal teams to deliver accurate and timely data.

Qualifications

  • 1–4 years of hands-on experience in Python-based web crawling and data extraction.
  • Experience with Playwright, Puppeteer, Selenium, and Python Requests/BeautifulSoup.
  • Understanding of proxies, user-agents, sessions, and anti-bot workarounds.
  • Experience with SQL or MongoDB.
  • Strong debugging, analytical, and logical reasoning skills.
  • Good English communication skills.

Job Type: Full-time

Pay: ₹30, ₹90,000.00 per month

Benefits:

  • Health insurance
  • Provident Fund
  • Work from home

Application Question(s):

  • How many projects involved anti-bot handling (proxies, sessions, headers, fingerprinting, CAPTCHA solving, logins)?
  • How many projects have you independently built end-to-end (requirement → crawling → extraction → automation → deployment)?
  • How many automated data pipelines have you built (scheduling, retries, data cleaning, storing into DB)?
  • How many JavaScript-rendered websites have you handled using Playwright/Puppeteer/Selenium?
  • How many years of hands-on experience do you have in web crawling, scraping, and data extraction using Python?
  • Current CTC (monthly)
  • Notice period (days)
  • How many years of experience do you have using Python Requests for crawling normal/static websites?

Work Location: Remote

  • IT Support Executive

    1 week ago

    Remote, India Brandzzy SoftTech Private Limited Full time

    Key responsibilities: 1. Conduct keyword research and optimize web content. 2. Evaluate website performance and suggest changes. 3. Monitor and improve site performance (crawling, indexing, and page speed). 4. Create high-quality backlinks and manage link development efforts. 5. Monitor and report on SEO performance with tools like as Google Analytics and...

  • AEO and SEO Engineer

    5 days ago

    Remote, India Colours Of Fashion Full time

    We are looking for experiencedAEO and SEO Engineer Specialist analyzes, tests, and modifies websites to optimize them for search engines, leading to higher rankings in major search results like Google and Bing)A Search Engine Optimization (SEO) Specialist analyzes, tests, and modifies websites to optimize them for search engines, leading to higher rankings...

  • SEO Strategist

    1 week ago

    Remote, India NotHuman Full time ₹ 1,20,000 - ₹ 1,80,000 per year

    Job Description About NotHuman.Al NotHuman. Al is a new-age marketing technology agency that blends cutting-edge development, automation, SEO, and brand strategy under one roof. We partner with brands to scale their digital presence-from disruptive startups to industry leaders. As we grow our SEO capabilities, we're looking for a detail-oriented, driven, and...

  • Technical Architect – Data Engineering

    2 weeks ago

    Remote, India Rojgar group Full time ₹ 20,00,000 - ₹ 26,00,000 per year

    Technical Architect – Data Engineering | LLM | Elastic Search | RAG | React | PythonLocation: Remote - WFHExperience: 10+ Years (Architect / Senior Level)Employment Type: Full-TimeAbout the RoleWe are looking for a Technical Architect with a strong background in data engineering, large language models (LLMs), and search technologies. This role is ideal for...

  • Technical Architect – Data Engineering

    2 weeks ago

    Remote, India Rojgar group Full time ₹ 18,00,000 - ₹ 27,00,000 per year

    Technical Architect – Data Engineering | LLM | Elastic Search | RAG | React | PythonLocation: Remote - WFHExperience: 10+ Years (Architect / Senior Level)Employment Type: Full-TimeAbout the RoleWe are looking for a Technical Architect with a strong background in data engineering, large language models (LLMs), and search technologies. This role is ideal for...

  • SEO Executive

    2 weeks ago

    Remote, India Websar Technologies LLP Full time

    **Job Title**:SEO Executive **Experience**: Minimum 1 year **Location**:Chhatrapati Sambhaji Nagar / Pune / Remote **Job Type**: Full-time **Company Overview**: Websar IT Solutions LLP is a leading digital marketing agency committed to delivering high-quality SEO and digital marketing services. We work with a diverse range of clients, helping them boost...

  • Data Engineer

    2 weeks ago

    Remote, India Slooze Full time ₹ 12,00,000 per year

    About the RoleWe're looking for a Data Engineer who can turn messy, unstructured information into clean, usable insights. You'll be building crawlers, integrating APIs, and setting up data flows that power our analytics and AI layers. If you love data plumbing as much as data puzzles — this role is for you.ResponsibilitiesBuild and maintain Python-based...

  • Sr. SEO Executive

    1 week ago

    Remote, India Royal Signs & Awnings Full time ₹ 9,71,642 - ₹ 15,00,000 per year

    Job Posting: SEO Expert |U.S Shift | 8am - 5pm Houston Time | Work From Office/Home |Experience: Minimum 8 YearsWe are seeking a highly seasoned SEO professional to join our team. To ensure we find the right fit for this role, please note that we will only be able to consider applications from candidates with a minimum of 6 years of hands-on SEO experience....

  • SEO Specialist

    1 day ago

    Remote, India FORBIX SEMICON Full time

    Job Description: SEO Specialist / Engineer (Remote)Company: FORBIX SEMICON Technologies Pvt LtdLocation: RemotePosition: SEO Specialist / SEO EngineerAbout Us:FORBIX SEMICON is a dynamic and ambitious new agency dedicated to delivering top-tier, data-driven SEO results for our clients. We're building a team of passionate experts who thrive on solving complex...

  • Digital Marketer Executive

    7 days ago

    Remote, India Magas Services Pvt Ltd Full time

    We are looking for a Digital Marketer with strong SEO knowledge to support our digital visibility and online growth initiatives. The ideal candidate will work on improving search rankings, enhancing website performance, and supporting broader digital marketing activities.Key ResponsibilitiesDevelop and implement SEO initiatives across on-page, off-page, and...