Junior Web Crawling Engineer

3 weeks ago

Ajmer, Rajasthan, India Forage AI Full time

We are seeking a Junior Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.

About Forage AI: Forage AI is a pioneering AI-powered data extraction and automation company that transforms complex, unstructured web and document data into clean, structured intelligence. Our platform combines web crawling, NLP, LLMs, and agentic AI to deliver highly accurate firmographic and enterprise insights across numerous domains. Trusted by global clients in finance, real estate, and healthcare, Forage AI enables businesses to automate workflows, reduce manual rework, and access high-quality data at scale.

Key Responsibilities:

  • Maintain and enhance existing web scraping and data crawling projects.
  • Develop and refine crawlers using Python-based tools and frameworks.
  • Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content.
  • Clean, validate, and integrate extracted data into downstream storage systems.
  • Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.).
  • Optimize crawling efficiency and ensure compliance with web crawling best practices.
  • Collaborate with cross-functional teams to improve data acquisition strategies.

Required Skills & Qualifications:

  • Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy).
  • Strong knowledge of browser automation tools such as Playwright or Selenium.
  • Solid understanding of HTML, CSS, and selector languages (XPath/CSS).
  • Experience in handling anti-scraping challenges and ensuring robust data extraction.
  • Familiarity with distributed scraping techniques and data pipelines.
  • Ability to troubleshoot and optimize web crawlers for performance and reliability.
  • Strong analytical and problem-solving skills with attention to detail.
  • Excellent communication and inter-personal skills.

Other Infrastructure Requirements

Since this is a completely work-from-home position, you will also require the following -

● High-speed internet connectivity for video calls and efficient work.

● Capable business-grade computer (e.g., modern processor, 8 GB+ of RAM, and

no other obstacles to interrupted, efficient work).

● Headphones with clear audio quality.

● Stable power connection and backups in case of internet/power failure.

  • Fraud Prevention Data Engineering Specialist

    2 weeks ago

    Ajmer, Rajasthan, India beBeeDataEngineering Full time ₹ 90,00,000 - ₹ 1,20,00,000

    As a seasoned Data Engineer, you will play a pivotal role in shaping our technology vision and architecture. We're looking for an expert who can drive the development of cutting-edge products to prevent fraud.Key ResponsibilitiesData Acquisition & Processing: Design and implement efficient data scraping systems using multi-threaded crawlers that handle...

  • Lead Quality Assurance Engineer

    2 weeks ago

    Ajmer, Rajasthan, India Backgammon Galaxy Full time

    Lead QA Engineer (Remote) Company: Backgammon Galaxy (backgammongalaxy.com) Location: Remote (HQ in Cyprus) | Full-timeBackgammon Galaxy is the leading web and mobile platform for playing backgammon, with a community of 150,000+ players worldwide. We've been growing for four years and are now looking for a Lead QA Engineer to help us scale further. Our team...

  • Senior Quality Assurance Lead

    2 weeks ago

    Ajmer, Rajasthan, India beBeeQuality Full time ₹ 18,00,000 - ₹ 25,20,000

    Job DescriptionWe are seeking a seasoned quality assurance engineer to lead our testing efforts. The ideal candidate will have a strong background in software testing and experience leading QA teams.The successful candidate will be responsible for developing and implementing test automation strategies, establishing quality standards and best practices, and...

  • Business Development Executive – WordPress

    4 weeks ago

    Ajmer, Rajasthan, India Volcone Web Solutions, LLC Full time

    Business Development Officer (BDO)Job Type: Full-Time (Remote)Shift Timing: Night Shift (5:30 PM – 2:30 AM IST)Experience: 1–3 YearsSalary: 25K - 75KAbout Company:Volcone Web Solutions cares as much about your technology as you do. We provide white-label website design and development services, along with professional-level quality assurance, design, and...

  • Senior Software Engineer

    2 weeks ago

    Ajmer, Rajasthan, India beBeeSoftware Full time ₹ 9,00,000 - ₹ 12,00,000

    We are seeking an experienced senior software engineer to lead the development of scalable web applications.The ideal candidate will possess expertise in Angular 16+, .NET Core 6, and SQL Server. They should have a strong background in designing, developing, and maintaining high-quality web applications while ensuring efficient test coverage.Design and...

  • Databricks Engineer

    3 weeks ago

    Ajmer, Rajasthan, India DataE2E Technologies Full time

    Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes on Databricks.Lead the architecture and implementation of scalable, secure, and high-performance data solutions.Collaborate with business stakeholders, data scientists, and analysts to gather requirements and deliver actionable data solutions.Optimize data workflows for...

  • Expert Unreal Engine Developer

    2 weeks ago

    Ajmer, Rajasthan, India beBeeRealtime Full time ₹ 7,00,000 - ₹ 15,00,000

    As a seasoned Unreal Engine expert, this role requires an individual with 10+ years of industry experience and a strong passion for real-time project creation and optimization.This is an opportunity to take on leadership roles, collaborate with diverse teams, and drive the delivery of interactive experiences using Unreal Engine.Main Responsibilities:Design,...

  • Structural Graduate Engineer

    4 weeks ago

    Ajmer, Rajasthan, India Accaerre Full time

    We are hiring a Graduate Structural Engineer – Remote from India Remote from India | GSTIN required | Full-time collaborationWe are an Italian structural engineering firm based in Italy, looking for a Graduate Structural Engineer to join our growing remote team. This role is designed for recent Master's graduates who are eager to start their professional...

  • SIEM Platform Engineer

    2 weeks ago

    Ajmer, Rajasthan, India beBeeSiem Full time ₹ 40,000 - ₹ 1,35,000

    Job RoleThe Senior Security Operations Center (SOC) Engineer – SIEM Engineering is a key position within our organization responsible for the optimal performance and functionality of the SIEM platform.Main Responsibilities:We are seeking an experienced professional to configure, deploy, and maintain the SIEM platform to ensure seamless integration and...

  • Transformative UI Engineer

    2 weeks ago

    Ajmer, Rajasthan, India beBeefrontend Full time ₹ 1,80,00,000 - ₹ 2,20,00,000

    We are looking for a skilled Frontend Developer to drive frontend projects, shape technical architecture, and contribute to user experience evolution. This role offers an opportunity to leverage cutting-edge web technologies, including React.js, JavaScript, CSS, TypeScript, AWS, and GraphQL, while collaborating on innovative projects involving Machine...