Web Crawling Engineer

5 days ago

Tirupati, India Whatjobs IN C2 Full time

We are seeking a Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms. Salary budget upto - Rs. 9 LPA About Forage AI : Forage AI is a pioneering AI-powered data extraction and automation company that transforms complex, unstructured web and document data into clean, structured intelligence. Our platform combines web crawling, NLP, LLMs, and agentic AI to deliver highly accurate firmographic and enterprise insights across numerous domains. Trusted by global clients in finance, real estate, and healthcare, Forage AI enables businesses to automate workflows, reduce manual rework, and access high-quality data at scale. Key Responsibilities: Maintain and enhance existing web scraping and data crawling projects. Develop and refine crawlers using Python-based tools and frameworks. Utilize browser automation tools (e.G., Playwright, Selenium) to handle dynamic content. Clean, validate, and integrate extracted data into downstream storage systems. Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.). Optimize crawling efficiency and ensure compliance with web crawling best practices. Collaborate with cross-functional teams to improve data acquisition strategies. Required Skills & Qualifications: Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy). Strong knowledge of browser automation tools such as Playwright or Selenium. Solid understanding of HTML, CSS, and selector languages (XPath/CSS). Experience in handling anti-scraping challenges and ensuring robust data extraction. Familiarity with distributed scraping techniques and data pipelines. Ability to troubleshoot and optimize web crawlers for performance and reliability. Strong analytical and problem-solving skills with attention to detail. Excellent communication and inter-personal skills. Other Infrastructure Requirements Since this is a completely work-from-home position, you will also require the following - ● High-speed internet connectivity for video calls and efficient work. ● Capable business-grade computer (e.G., modern processor, 8 GB+ of RAM, and no other obstacles to interrupted, efficient work). ● Headphones with clear audio quality. ● Stable power connection and backups in case of internet/power failure.

  • Software Development Engineer II

    2 weeks ago

    Tirupati, India ClearDemand Full time

    Job Summary:Data is the foundation of our business, and your work will ensure that we continue to deliver high-quality competitive intelligence at scale. Web platforms are constantly evolving, deploying sophisticated anti-bot measures—your job is to stay ahead of them. If you thrive on solving complex technical challenges and enjoy working with real-world...

  • Technical seo specialist

    3 hours ago

    Tirupati, India QR TIGER - QR Code Generator Full time

    Technical SEO Specialist (Expert) Your Mission: As a Technical SEO Specialist at QR TIGER, you will take a role in shaping and maintaining the technical foundation of our digital ecosystem. Your mission is to optimize site infrastructure, enhance crawlability and indexation, and ensure our website consistently meets modern search engine standards. While you...

  • SEO Developer

    2 weeks ago

    Tirupati, India BLOGMERCE Full time

    Company: BlogmerceLocation: Remote (Preference: India)Schedule: Monday–Friday (8 h/day) + Saturday (4 h light session)Language: Excellent English (written + spoken)About BlogmerceBlogmerce is a fast-growing digital company that builds and scales outdoor and lifestyle brands through advanced SEO systems, automation, and data-driven growth.Our focus is on...

  • Web Designer Jobs Opening in A Client of Freshersworld at TirupatiLess

    3 weeks ago

    Tirupati, India A Client of Freshersworld Full time

    Web Designer Key Responsibilities:- PSD to HTML responsive conversions.- PSD to HTML responsive wordpress theme conversions.- Conceptualize and design digital advertising, marketing collateral, presentations, print, social and mobile content.- You must be a hands-on designer with a strong ability to work individually as well as a part of a team- Knowledge...

  • Technical support engineer for a web/mobile application for us company

    6 days ago

    Tirupati, India Textellent, Inc. Full time

    Full Time | 100% Work From Home | US Office HoursYou will work closely with Client Success and support them by providing Tier-2 client support for software technical issues and work with Engineering department to debug and resolve the issues. RESPONSIBILITIES:Provide technical support to internal and external users including customer service teams as well as...

  • SEO Specialist

    2 days ago

    Tirupati, Andhra Pradesh, India Young Minds Technology Solutions Pvt Ltd Full time ₹ 2,00,000 - ₹ 4,00,000 per year

    Job Title: SEO SpecialistKey Responsibilities: Perform comprehensive keyword research to support content strategies. Optimize website content, blogs, landing pages, and metadata for SEO. Execute on-page SEO strategies (URL optimization, internal linking, title tags, alt texts). Implement effective off-page SEO techniques (link-building, guest...

  • Software Engineer in Test

    2 weeks ago

    Tirupati, India Atlys Full time

    Job description Atlys' mission is to enable every person on earth to travel freely.At Atlys, we believe that the path to creating a more open world is by making it efficient to travel. Travelers cite visas as the most frustrating pain point, and we're starting by automating that completely. We're looking for talented people who are interested in building the...

  • Full Stack Engineer

    1 week ago

    Tirupati, India Goodfit Full time

    Company DescriptionGoodfit is a platform that revolutionizes the hiring process by using AI-driven first-round interviews and bias-free screening to connect companies with the right people efficiently. Hiring based on potential and not just keywords and formatting, Goodfit aims to make hiring about people, not paperwork. The platform is dedicated to creating...

  • Lead Engineer

    3 weeks ago

    Tirupati, India Houndstooth Strategic Communications Full time

    Role:Staff/Lead Developer Location:Remote (Anywhere in India only) Shift time:2 pm - 11 pm IST Company:HoundstoothSC (https://houndstoothsc.com/) Team size:8 Availability to join:Immediate to 15 daysJob Summary: We are seeking a highly skilled and motivated Staff Engineer / Lead Developer to lead our e commerce development initiatives. The ideal candidate...

  • Staff frontend engineer

    4 weeks ago

    Tirupati, India True Probe Full time

    Staff Frontend Engineer (Mobile Heavy)– RemoteAbout UsTrue Probe is an innovative HR technology startup revolutionizing the employment verification space. We leverage intelligent AI agents to deliver fast, comprehensive, and fraud-aware background verification reports, helping modern hiring teams make better decisions.Role OverviewWe’re looking for a...