Web Crawling Engineer

5 days ago

Solapur, India Forage AI Full time

We are seeking a Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms. Salary budget upto - Rs. 9 LPA About Forage AI: Forage AI is a pioneering AI-powered data extraction and automation company that transforms complex, unstructured web and document data into clean, structured intelligence. Our platform combines web crawling, NLP, LLMs, and agentic AI to deliver highly accurate firmographic and enterprise insights across numerous domains. Trusted by global clients in finance, real estate, and healthcare, Forage AI enables businesses to automate workflows, reduce manual rework, and access high-quality data at scale. Key Responsibilities: - Maintain and enhance existing web scraping and data crawling projects. - Develop and refine crawlers using Python-based tools and frameworks. - Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content. - Clean, validate, and integrate extracted data into downstream storage systems. - Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.). - Optimize crawling efficiency and ensure compliance with web crawling best practices. - Collaborate with cross-functional teams to improve data acquisition strategies. Required Skills & Qualifications: - Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy). - Strong knowledge of browser automation tools such as Playwright or Selenium. - Solid understanding of HTML, CSS, and selector languages (XPath/CSS). - Experience in handling anti-scraping challenges and ensuring robust data extraction. - Familiarity with distributed scraping techniques and data pipelines. - Ability to troubleshoot and optimize web crawlers for performance and reliability. - Strong analytical and problem-solving skills with attention to detail. - Excellent communication and inter-personal skills. Other Infrastructure Requirements Since this is a completely work-from-home position, you will also require the following - ● High-speed internet connectivity for video calls and efficient work. ● Capable business-grade computer (e.g., modern processor, 8 GB+ of RAM, and no other obstacles to interrupted, efficient work). ● Headphones with clear audio quality. ● Stable power connection and backups in case of internet/power failure.

  • Web App SDK Engineer

    3 weeks ago

    Solapur, India Huxley Full time

    Company: Japanese Tech GiantSalary: $50,000 USDLanguage requirements: English onlyMandatory Qualifications:- Extensive experience (5+ years) in JavaScript and TypeScript development, with a strong focus on SDK development for Web applications.- Proven track record in leading the development and maintenance of popular SDKs.- Deep understanding of JavaScript...

  • Full Stack Engineer

    6 days ago

    Solapur, India CES Full time

    About the RoleWe’re looking for a Senior Software Engineer who’s passionate about building high-performance, scalable web applications using .NET and ReactJS. You’ll play a key role in designing, developing, and delivering robust solutions while mentoring others and driving engineering excellence.Key ResponsibilitiesDesign, develop, and maintain modern...

  • Infrastructure Engineer

    3 days ago

    Solapur, India People Prime Worldwide Full time

    Important Note (Please Read Before Applying)🚫 Do NOT apply if:You have less than 8 years or more than 15 years of hands-on Infrastructure Engineering / Cloud Architecture experienceYou do not have strong AWS experience (this is an AWS-focused role)You have no real-world Terraform or AWS CDK implementation experienceYou have never worked with Kubernetes or...

  • Full Stack Engineer

    1 week ago

    Solapur, India Discoveries Quintessential Full time

    Software Development Engineer (SDE-II) Location:Gurgaon, India | Onsite Job Type:Full-time Experience:6–8 YearsAbout the Role We are seeking a passionateSoftware Development Engineer (SDE-II)to join our dynamic tech team in Gurgaon. In this role, you’ll design, develop, and scale enterprise-grade backend and frontend modules powering high-traffic retail...

  • Automation QA Engineer

    3 days ago

    Solapur, India Webknot Technologies Full time

    Job Title: QA Automation EngineerLocation: BangaloreExperience: 1–3 YearsEmployment Type: Full-TimeRole OverviewWe are looking for a skilled QA Automation Engineer with 1–3 years of experience to design, develop, and execute automated test scripts. The ideal candidate will ensure the quality, performance, and reliability of applications by working...

  • Full Stack Engineer

    3 days ago

    Solapur, India Tuna Software Solutions Pvt Ltd Full time

    We are looking for a talented and motivated Full-stack JavaScript Developer who is passionate about building high-quality, scalable, and performant web and mobile applications. The ideal candidate should be proficient in React.js, Next.js, and Node.js, with additional experience in React Native being a strong plus. You will collaborate with cross-functional...

  • Senior Software Engineer

    1 week ago

    Solapur, India Anblicks Full time

    Responsibilities: Develop and maintain robust web applications using Laravel and Angular. Design, develop, and implement new features and enhancements. Write clean, efficient, and well-documented code. Collaborate with cross-functional teams to define, design, and ship new features. Troubleshoot and debug issues to ensure optimal performance and reliability....

  • Artificial Intelligence Intern

    1 week ago

    Solapur, India Teqrox® Full time

    About the jobRole: Full Stack AI Engineer Intern Company Location: Bengaluru Work Mode: Remote Duration: 3 Months (Full-time) Internship Type : Unpaid Email to Apply: riyaz@teqrox.com | riddhi@teqroxgroup.comAbout TeqroxTeqrox is a forward-thinking management consultancy and technology solutions company. We combine consulting expertise with AI-driven...

  • Artificial Intelligence Intern

    4 days ago

    Solapur, India Teqrox® Full time

    About the job Role: Full Stack AI Engineer Intern Company Location: Bengaluru Work Mode: Remote Duration: 3 Months (Full-time) Internship Stipend: 8k Per Month Email to Apply: | About Teqrox Teqrox is a forward-thinking management consultancy and technology solutions company. We combine consulting expertise with AI-driven innovation to deliver scalable,...

  • Senior Java Software Engineer

    3 weeks ago

    Solapur, India Reveille Technologies,Inc Full time

    Java Developer – C2H (Contract to Hire)📍 Location: Chennai🕓 Experience: 4–5 Years🚀 Notice Period: Immediate Joiners PreferredJob Description:We are looking for an experienced Java Developer with 4–5 years of hands-on experience in designing and developing scalable web applications using Java 8/11 and modern frameworks.Key...