Web Crawling Engineer

1 week ago

Dindigul, India Forage AI Full time

We are seeking a Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms. Salary budget upto - Rs. 9 LPA About Forage AI : Forage AI is a pioneering AI-powered data extraction and automation company that transforms complex, unstructured web and document data into clean, structured intelligence. Our platform combines web crawling, NLP, LLMs, and agentic AI to deliver highly accurate firmographic and enterprise insights across numerous domains. Trusted by global clients in finance, real estate, and healthcare, Forage AI enables businesses to automate workflows, reduce manual rework, and access high-quality data at scale. Key Responsibilities: Maintain and enhance existing web scraping and data crawling projects. Develop and refine crawlers using Python-based tools and frameworks. Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content. Clean, validate, and integrate extracted data into downstream storage systems. Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.). Optimize crawling efficiency and ensure compliance with web crawling best practices. Collaborate with cross-functional teams to improve data acquisition strategies. Required Skills & Qualifications : Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy). Strong knowledge of browser automation tools such as Playwright or Selenium. Solid understanding of HTML, CSS, and selector languages (XPath/CSS). Experience in handling anti-scraping challenges and ensuring robust data extraction. Familiarity with distributed scraping techniques and data pipelines. Ability to troubleshoot and optimize web crawlers for performance and reliability. Strong analytical and problem-solving skills with attention to detail. Excellent communication and inter-personal skills. Other Infrastructure Requirements Since this is a completely work-from-home position, you will also require the following - ● High-speed internet connectivity for video calls and efficient work. ● Capable business-grade computer (e.g., modern processor, 8 GB+ of RAM, and no other obstacles to interrupted, efficient work). ● Headphones with clear audio quality. ● Stable power connection and backups in case of internet/power failure.

  • Web Developer

    1 week ago

    Dindigul, India Cloudologix Full time

    Job Title: Web Developer (Angular / React / Next.js)Location: RemoteExperience: 4+ yearsAbout the RoleWe’re looking for an experienced Web Developer with strong expertise in Angular (mandatory) and hands-on experience with React.js and/or Next.js. You’ll be responsible for building and maintaining high-performance, scalable web applications with...

  • Web Content Specialist

    5 hours ago

    Dindigul, India Marsh McLennan Full time

    We are seeking a talented individual to join our operations team (MMA) at Marsh. This role will be based in Mumbai. This is a hybrid role that has a requirement of working at least three days a week in the office.We are seeking a detail-oriented and proactive colleague to support the development and testing of our website. This role is critical in ensuring...

  • Quality Assurance Automation Engineer

    2 weeks ago

    Dindigul, India QAIN-Finity Full time

    Job Description: QA Automation Engineer – Jaipur (On-Site)Location:Jaipur (On-Site Only) Role Type:Full-time About the RoleWe are looking for a QA Automation Engineer who is strong inmanual testing , has solid experience inweb application automation , and has a good understanding ofAPI testing . Experience withPlaywrightandE-commerce platformsis a strong...

  • Software Engineer

    3 weeks ago

    Dindigul, India capeiron Full time

    We’re Hiring: Software Engineer [Remote]2-Month Contract Role at Capeiron TechnologyFixed Compensation for contract: INR 5,00,000Roles and ResponsibilitiesAs a Software Engineer, you’ll be responsible for developing and maintaining the end-to-end application stack, ensuring seamless integrations and high-quality user experiences. You’ll collaborate...

  • Senior quality assurance automation engineer

    2 weeks ago

    Dindigul, India Sourcebae Full time

    Job Alert: Senior Automation Test Engineer (4-7 Yrs) in Jaipur ???? Are you a seasoned Automation Engineer with a passion for quality and robust backend testing? Join our dynamic team at PSI (Pratham Software) and drive automation excellence! We are looking for a Senior Automation Test Engineer with 4-9 years of experience to lead the design and...

  • Senior Dot Net Full Stack Engineer

    2 weeks ago

    Dindigul, India NEOGOV Full time

    Senior Dot Net Full Stack Engineer NEOGOV, an HRIS Product company based in Los Angeles is the market and technology leader in human resources software for the public sector. Our HR suite empowers public sector HR to streamline processes in a centralized platform.We are looking for seasoned Senior Dot Net Full Stack Engineers (dot net+Angular+sql) for...

  • Linux System Engineer

    5 days ago

    Dindigul, Tamil Nadu, India SysKnox Solutions Full time ₹ 1,32,000 - ₹ 3,00,000 per year

    SysKnox Solutions LLPSysKnox Solutions LLP is a growing IT startup specializing in professional Linux server management and outsourced technical support. We value accountability, proactive learning, and reliable communication - traits that form the foundation of our team culture.We are looking for committed and responsible individuals who can grow with us...

  • Cloud Security Engineer

    1 week ago

    Dindigul, India ValueMomentum Full time

    Job Description for Cloud Security EngineerLocation: Hyderabad, Pune, CoimbatoreExperience: 6- 8 years of experienceWorking Mode: 5 Days Work from OfficeJob Summary: We are looking for a Cloud Security Engineer with a minimum of 5 years of experience in Amazon Web Services (AWS) to join our dynamic team. The ideal candidate will have a deep understanding of...

  • Backend Engineer — Data

    3 weeks ago

    Dindigul, India Branch International Full time

    Branch OverviewImagine a world where every person has improved access to financial services. People could start new businesses, pay for their children’s education, cover their emergency medical bills – the possibilities to improve life are endless. Branch is a global technology company revolutionizing financial access for millions of underserved banking...

  • Full Stack Developer

    2 weeks ago

    Dindigul, India Turing Full time

    Role Overview:Turing is seeking experienced Full Stack Developers to help build end-to-end AI-driven applications for US customers — spanning backend services, web frontends, and evaluation tooling.In this role, you’ll work with Python and modern JS/TS to ship production-quality features, design and run model evaluations, and transform cutting-edge AI...