Web Crawling Engineer

1 week ago

bangalore, India Forage AI Full time

We are seeking a Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms. Salary budget upto - Rs. 9 LPA About Forage AI : Forage AI is a pioneering AI-powered data extraction and automation company that transforms complex, unstructured web and document data into clean, structured intelligence. Our platform combines web crawling, NLP, LLMs, and agentic AI to deliver highly accurate firmographic and enterprise insights across numerous domains. Trusted by global clients in finance, real estate, and healthcare, Forage AI enables businesses to automate workflows, reduce manual rework, and access high-quality data at scale. Key Responsibilities: Maintain and enhance existing web scraping and data crawling projects. Develop and refine crawlers using Python-based tools and frameworks. Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content. Clean, validate, and integrate extracted data into downstream storage systems. Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.). Optimize crawling efficiency and ensure compliance with web crawling best practices. Collaborate with cross-functional teams to improve data acquisition strategies. Required Skills & Qualifications : Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy). Strong knowledge of browser automation tools such as Playwright or Selenium. Solid understanding of HTML, CSS, and selector languages (XPath/CSS). Experience in handling anti-scraping challenges and ensuring robust data extraction. Familiarity with distributed scraping techniques and data pipelines. Ability to troubleshoot and optimize web crawlers for performance and reliability. Strong analytical and problem-solving skills with attention to detail. Excellent communication and inter-personal skills. Other Infrastructure Requirements Since this is a completely work-from-home position, you will also require the following -● High-speed internet connectivity for video calls and efficient work. ● Capable business-grade computer (e.g., modern processor, 8 GB+ of RAM, and no other obstacles to interrupted, efficient work). ● Headphones with clear audio quality. ● Stable power connection and backups in case of internet/power failure.

  • Web Crawling Engineer

    1 week ago

    bangalore, India Forage AI Full time

    We are seeking a Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.Salary budget...

  • Data Engineer

    3 weeks ago

    Bangalore Division, India Publicis Global Delivery (PGD) Full time

    Note: It's a Data Scrapping / Web Scrapping based role for Bangalore & Gurugram location. Mandatory Skills: Python and any scrapping framework (Scrapy, Selenium, Playwright, or BeautifulSoup) . Role: Data Acquisition Engineer Location: Bangalore / Gurgaon Experience: 3-5 years An ideal candidate has minimum 3 years of hands-on programming experience in...

  • Python Automation and Web Scraping Specialist

    11 hours ago

    Bangalore Division, India Foresiet Full time

    Company Description Foresiet is an AI-enabled SaaS-based Cybersecurity company that provides a comprehensive solution for Digital Risk Prevention. Leveraging the Cyber Digital Investigator platform, Foresiet proactively detects, monitors, and secures identity, data, and asset threats. Our unique combination of Human Intelligence (HUMINT) and Applied Research...

  • Engineer, Software

    1 week ago

    bangalore, India MeasureOne Full time

    What We're Looking ForWe are seeking an Engineer with over 3 years of experience in Web Automation to take ownership of our existing automation workflows and enhance them for speed, resilience, and scalability. You will collaborate closely with our product team to improve systems that extract data from diverse and dynamic endpoints. The ideal candidate is a...

  • Technical Architect

    2 weeks ago

    Bangalore, India Terra Technology Circle Consulting Private Limited Full time

    It is a Full-time Job, “No” remote work. Technical Architect - Data Engineering willing to work on a 12-month contract/full-time may apply. Experience Candidates should have experience between 4-6 years We are seeking a Technical Architect with strong hands-on coding experience to design and implement scalable, data-driven, and AI-enhanced systems. This...

  • Search Engine Optimization Specialist

    2 weeks ago

    bangalore, India Web in Prime Full time

    About the RoleWe’re seeking a dedicated and detail-oriented SEO Specialist to join our digital marketing team. You’ll be responsible for improving website rankings, driving organic traffic, and implementing effective SEO strategies across multiple projects. This is a remote position — perfect for someone who thrives in a flexible and performance-driven...

  • Search Engine Optimization Specialist

    1 day ago

    bangalore, India Web in Prime Full time

    About the RoleWe’re seeking a dedicated and detail-oriented SEO Specialist to join our digital marketing team. You’ll be responsible for improving website rankings, driving organic traffic, and implementing effective SEO strategies across multiple projects. This is a remote position — perfect for someone who thrives in a flexible and performance-driven...

  • Engineer, Software

    1 day ago

    bangalore, India MeasureOne Full time

    What We're Looking For We are seeking an Engineer with over 3 years of experience in Web Automation to take ownership of our existing automation workflows and enhance them for speed, resilience, and scalability. You will collaborate closely with our product team to improve systems that extract data from diverse and dynamic endpoints. The ideal candidate is a...

  • Search Engine Optimization Specialist

    7 days ago

    bangalore, India Web in Prime Full time

    About the Role We’re seeking a dedicated and detail-oriented SEO Specialist to join our digital marketing team. You’ll be responsible for improving website rankings, driving organic traffic, and implementing effective SEO strategies across multiple projects. This is a remote position — perfect for someone who thrives in a flexible and...

  • Technical Architect – Data Engineering

    2 weeks ago

    bangalore, India Terra Technology Circle Consulting Private Limited Full time

    LocationThe successful candidate shall be placed at Bangalore Location. It is a Full-time Job, “No” remote work. Technical Architect - Data Engineering willing to work on a 12-month contract/full-time may apply.ExperienceCandidates should have experience between 4-6 yearsRole DescriptionAbout the RoleWe are seeking a Technical Architect with strong...