Junior Web Crawling Engineer

3 weeks ago

Alwar, Rajasthan, India Forage AI Full time

We are seeking a Junior Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.

About Forage AI: Forage AI is a pioneering AI-powered data extraction and automation company that transforms complex, unstructured web and document data into clean, structured intelligence. Our platform combines web crawling, NLP, LLMs, and agentic AI to deliver highly accurate firmographic and enterprise insights across numerous domains. Trusted by global clients in finance, real estate, and healthcare, Forage AI enables businesses to automate workflows, reduce manual rework, and access high-quality data at scale.

Key Responsibilities:

  • Maintain and enhance existing web scraping and data crawling projects.
  • Develop and refine crawlers using Python-based tools and frameworks.
  • Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content.
  • Clean, validate, and integrate extracted data into downstream storage systems.
  • Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.).
  • Optimize crawling efficiency and ensure compliance with web crawling best practices.
  • Collaborate with cross-functional teams to improve data acquisition strategies.

Required Skills & Qualifications:

  • Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy).
  • Strong knowledge of browser automation tools such as Playwright or Selenium.
  • Solid understanding of HTML, CSS, and selector languages (XPath/CSS).
  • Experience in handling anti-scraping challenges and ensuring robust data extraction.
  • Familiarity with distributed scraping techniques and data pipelines.
  • Ability to troubleshoot and optimize web crawlers for performance and reliability.
  • Strong analytical and problem-solving skills with attention to detail.
  • Excellent communication and inter-personal skills.

Other Infrastructure Requirements

Since this is a completely work-from-home position, you will also require the following -

● High-speed internet connectivity for video calls and efficient work.

● Capable business-grade computer (e.g., modern processor, 8 GB+ of RAM, and

no other obstacles to interrupted, efficient work).

● Headphones with clear audio quality.

● Stable power connection and backups in case of internet/power failure.

  • Security Engineer

    3 weeks ago

    Alwar, Rajasthan, India TAC Security Full time

    Job descriptionAs a Security Engineer - VAPT, you will be responsible for conducting comprehensive security assessments, identifying vulnerabilities, and implementing effective remediation strategies. Leveraging your expertise in penetration testing and ethical hacking, you will play a key role in enhancing the security posture of our clients' systems and...

  • Senior Frontend Developer

    3 weeks ago

    Alwar, Rajasthan, India My3Tech Full time

    Overview:We are looking for a Senior Frontend Web Developer / Full Stack Tech Lead to take ownership of an existing web application and lead frontend development. This role requires deep expertise in modern web frameworks, strong UI/UX sensibility, and proven leadership. You will guide the team in implementing clean, modular designs and high-performance...

  • Quality assurance automation engineer

    4 weeks ago

    Alwar, Rajasthan, India AvenDATA Full time

    We're Hiring: QA Automation Engineer (Selenium)Location: 100% Remote (India)Experience: 3–6 YearsType: Full-time, Long-termStart Date: ASAPAbout UsAven DATA is a European IT company specializing in decommissioning and archiving legacy systems like SAP, Oracle, Navision, and others. We support clients worldwide with secure, scalable, and high-performance...

  • GCP Data Engineer

    3 weeks ago

    Alwar, Rajasthan, India _VOIS Full time

    Key Responsibilities: Design, develop, and maintain scalable data pipelines and ETL processes using GCP services such as BigQuery, Cloud Data Fusion, Dataflow, Pub/Sub, Cloud Storage, Composer ,Cloud Function, Cloud RUN Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions....

  • Cloud Engineer

    3 weeks ago

    Alwar, Rajasthan, India Coforge Full time

    Job Location:- Hyderabad & NoidaExperience Required: 5 to 8 YearsSkills (Hands on experience required)- Spark, Scala, Big Data (Hadoop), AWS/Azure (+Databricks mandatory/ GCP (+DBT Mandatory), SQLShift- UK ShiftStreaming data Technical skills requirements :-Experience- 5+ Years Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS...

  • Jr. Technical Support Engineering

    4 weeks ago

    Alwar, Rajasthan, India CloudLabs Inc Full time

    About CloudLabs:CloudLabs Inc was founded in 2014 with the mission to provide exceptional IT & Business consulting services at a competitive price, to help clients realize the best value from their investments.Within a short span, CloudLabs evolved from pure-play consulting into a transformative partner for Business Acceleration Advisory, Transformative...

  • SRE & DevOps Engineer (Java) | AI Platform Team

    4 weeks ago

    Alwar, Rajasthan, India Plumeria Tech Full time

    We're hiring: SRE & DevOps Engineer (Java) | AI Platform Team Location: Hyderabad / Bangalore, IndiaA leading multinational in global commerce & technology is looking for an experienced SRE & DevOps Engineer (Java) to join its AI Platform Team. This role is all about supporting large-scale AI/ML infrastructure, enabling researchers and data scientists to...

  • Senior Software Engineer – Java Developer

    4 weeks ago

    Alwar, Rajasthan, India NMS Consultant Full time

    Job Title - Senior Software Engineer – Java DeveloperExperience - 6+ YearsLocation - BangaloreJob DescriptionExperienced Senior Core Java Developer to design, develop and optimize high-performancebackend applications, services and APIs. The ideal candidate will have expertise in Java,database architecture, data analytics and AI/ML integration. This role...

  • Tech Lead

    3 weeks ago

    Alwar, Rajasthan, India Actowiz Solutions Full time

    Job Title: Tech Lead - Python & Web Data SolutionsLocation: Ahmedabad / WFOExperience Level: Senior (4+ years)Employment Type: Full-timeJob SummaryWe are seeking a highly skilled and experienced Lead Data Scraping Engineer to join our team. The ideal candidate will have a minimum of 4 years of hands-on experience in IT scraping, with at least 2 years leading...

  • Staff Engineer

    3 weeks ago

    Alwar, Rajasthan, India DISH Network Technologies Full time

    13+ years of experience in online charging systems, with at least 4-5 years in a solution architect role focused on Matrixx online charging system. Deep understanding of online and offline charging principles.Proven hands-on experience designing and implementing solutions using the Matrixx Digital Commerce Platform (DCP).Strong work experience in Matrixx...