Web crawling engineer

2 days ago

Belgaum, India Forage AI Full time

We are seeking a Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms. Salary budget upto - Rs. 9 LPA About Forage AI : Forage AI is a pioneering AI-powered data extraction and automation company that transforms complex, unstructured web and document data into clean, structured intelligence. Our platform combines web crawling, NLP, LLMs, and agentic AI to deliver highly accurate firmographic and enterprise insights across numerous domains. Trusted by global clients in finance, real estate, and healthcare, Forage AI enables businesses to automate workflows, reduce manual rework, and access high-quality data at scale. Key Responsibilities: Maintain and enhance existing web scraping and data crawling projects. Develop and refine crawlers using Python-based tools and frameworks. Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content. Clean, validate, and integrate extracted data into downstream storage systems. Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.). Optimize crawling efficiency and ensure compliance with web crawling best practices. Collaborate with cross-functional teams to improve data acquisition strategies. Required Skills & Qualifications : Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy). Strong knowledge of browser automation tools such as Playwright or Selenium. Solid understanding of HTML, CSS, and selector languages (XPath/CSS). Experience in handling anti-scraping challenges and ensuring robust data extraction. Familiarity with distributed scraping techniques and data pipelines. Ability to troubleshoot and optimize web crawlers for performance and reliability. Strong analytical and problem-solving skills with attention to detail. Excellent communication and inter-personal skills. Other Infrastructure Requirements Since this is a completely work-from-home position, you will also require the following - ● High-speed internet connectivity for video calls and efficient work. ● Capable business-grade computer (e.g., modern processor, 8 GB+ of RAM, and no other obstacles to interrupted, efficient work). ● Headphones with clear audio quality. ● Stable power connection and backups in case of internet/power failure.

  • Web Designer

    3 weeks ago

    Belgaum, India Tychi Wallet Full time

    Company DescriptionTychi Labs is a forward-thinking blockchain company driving innovation in the decentralized digital economy. With a global presence across Auckland, Sydney, Delhi, and Tel Aviv, the company creates breakthrough blockchain solutions. At the core of its offerings is the Tychi Wallet—a versatile multi-chain crypto wallet engineered to...

  • Web Designer

    3 weeks ago

    Belgaum, India Tychi Wallet Full time

    Company DescriptionTychi Labs is a forward-thinking blockchain company driving innovation in the decentralized digital economy. With a global presence across Auckland, Sydney, Delhi, and Tel Aviv, the company creates breakthrough blockchain solutions. At the core of its offerings is the Tychi Wallet—a versatile multi-chain crypto wallet engineered to...

  • Développeur web

    3 weeks ago

    Belgaum, India 360 Space Full time

    LONG-TERM FREELANCE - Expert Web Scraping/Data Extraction We are looking for a high-level scraper / data automation expert ready to take on a long-term project with serious ambition. We’re talking about a smart, evolving system that requires advanced technical skill, reliability, and creativity to bypass complex data access challenges. 💡 About the...

  • Senior Software Engineer

    1 week ago

    Belgaum, India Whatjobs IN C2 Full time

    About us Endowus is Asia’s leading fee-only wealth platform. Headquartered in Singapore, we are the first digital advisor to span both private wealth and public pension savings (CPF & SRS), helping everyone grow all their money with expert advice, institutional access to financial solutions, low & fair fees, and a delightful personalised digital wealth...

  • Quality Assurance Engineer

    4 weeks ago

    Belgaum, India Whatjobs IN C2 Full time

    Job Title: QA Engineer About Blossom Social: Blossom Social is building the future of social investing. Our platform empowers retail investors by combining the best of social networks with investing tools—enabling users to connect, learn, and grow their financial confidence together. Join us to discover new investment ideas, manage your stock portfolio,...

  • Software Engineer

    3 days ago

    Belgaum, India capeiron Full time

    We’re Hiring: Software Engineer [Remote]2-Month Contract Role at Capeiron TechnologyFixed Compensation for contract: INR 5,00,000Roles and ResponsibilitiesAs a Software Engineer, you’ll be responsible for developing and maintaining the end-to-end application stack, ensuring seamless integrations and high-quality user experiences. You’ll collaborate...

  • Full Stack Engineer(AWS/Oracle)

    2 days ago

    Belgaum, India Starlightify Full time

    Company DescriptionStarlightify has over a decade of IT consulting expertise, serving industries such as finance, telecom, and healthcare. We specialize in helping businesses tackle complex IT challenges while staying aligned with their budgets and goals. Our solutions are tailored to meet client-specific needs and deliver significant value. Starlightify...

  • [Apply Now] Senior Automation Engineer

    3 weeks ago

    Belgaum, India ACL Digital Full time

    ACL digital is hiring test engineers for Bangalore location. Location Bangalore- Work from office Notice Period- Immediate to 15 Days max. Required Qualifications: - 5+ years of Industry experience. Hands on experience on End-to-End Solution testing in Security SDWAN area. - Strong understanding of network security concepts, protocols, and technologies. - QA...

  • Salesforce System Engineer

    1 week ago

    Belgaum, India Whatjobs IN C2 Full time

    Shift Timings : 6 pm to 3 am IST Position Responsibilities : We’re looking for a Salesforce System Engineer I–II who thrives in a fast-paced, collaborative environment and can work independently with minimal supervision. This role is ideal for someone who enjoys solving technical challenges, optimizing systems, and communicating clearly across global...

  • Software Engineer

    4 weeks ago

    Belgaum, India UrbanPiper Full time

    Software Engineer I/II - Quality Assurance Manual Testing About UrbanPiper: UrbanPiper is the leading SaaS platform in the restaurant industry. We're helping the smallest of restaurants to the largest chains across the world to grow their in-store and online commerce. Right from automating all their workflows across online delivery platforms such as Swiggy,...