Web Scraping Engineer

4 days ago


India MrScraper Full time
Job Overview:

We are seeking a skilled Web Scraping Engineer to help us extract, process, and analyze large-scale data from the web. This role involves developing efficient, scalable, and resilient scraping solutions to gather structured and unstructured data from diverse online sources.

Key Responsibilities:
  • Design, build, and maintain web scraping scripts, bots, and crawlers to collect data from websites, APIs, and other online sources.
  • Ensure scrapers are efficient, scalable, and resilient to website structure changes and anti-bot mechanisms.
  • Develop data parsing, cleaning, and structuring workflows for downstream applications.
  • Optimize scraping performance, handling concurrency, rate-limiting, and distributed architectures.
  • Collaborate with stakeholders to understand data needs and translate them into automated scraping solutions.
  • Monitor, troubleshoot, and maintain scrapers to ensure data accuracy and integrity.
  • Implement best practices for ethical web scraping and comply with legal requirements.
Required Skills & Qualifications:
  • Programming: Proficiency in Python and libraries like Scrapy, BeautifulSoup, Selenium, or Playwright.
  • Web Technologies: Strong understanding of HTML, CSS, JavaScript, and browser behavior.
  • Networking & HTTP: Familiarity with REST APIs, proxies, headers, and cookies.
  • Data Processing: Experience with pandas, regex, JSON, XML, and database integration.
  • Automation & Scaling: Knowledge of asynchronous programming, multiprocessing, and distributed computing frameworks.
  • Security & Compliance: Awareness of anti-scraping mechanisms and ethical scraping guidelines.
Fulfilling Work Environment:
  • Work on cutting-edge data extraction and automation projects.
  • Flexible work environment with remote opportunities.
  • Competitive salary, benefits, and career growth opportunities.
  • Join a passionate, innovative team that values data-driven decision-making.


  • India MrScraper Full time

    Job Overview:We're looking for a Web Scraping Engineer to join our team and help us extract, process, and analyze large-scale data from the web. You'll be responsible for developing efficient, scalable, and resilient scraping solutions to gather structured and unstructured data from diverse online sources. If you thrive on solving complex problems, working...


  • India Forage AI Full time

    We are seeking a talented Python Web Scraping Developer to join our team at Forage AI. As a Python Web Scraping Developer, you will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality.You will develop and refine crawlers using Python-based tools and frameworks such as Scrapy, and...

  • Web Scraping Expert

    5 days ago


    India YipitData Full time

    Crafting Resilient Web ScrapersYipitData is seeking a skilled Web Scraping Expert to join our Data Solutions team. In this transformative role, you will design, refactor, and maintain web scrapers that drive critical reports across our organization, ensuring data ingestion processes are resilient, efficient, and scalable.About the JobCollaborate with...

  • Web Scraping Expert

    7 days ago


    India brightedge Full time

    Data Engineer Job DescriptionThe successful candidate will be responsible for developing and maintaining data collection pipelines to efficiently ingest and store structured and unstructured data. They will also implement web scraping techniques and APIs for real-time and batch data retrieval. Additionally, they will optimize data storage and query...

  • Data Scientist

    4 days ago


    India Adglobal360 Full time

    We are seeking a highly skilled Data Scientist to join our team as a Web Scraping Specialist. In this role, you will be responsible for designing and developing robust web scraping solutions using Python and related libraries.">The ideal candidate will have strong expertise in web scraping and data extraction using Python, proficiency in web scraping...


  • India brightedge Full time

    We are seeking a Backend Engineer to design, implement, and optimize data collection, storage, and processing pipelines. The ideal candidate will have expertise in handling structured and unstructured data, optimizing query performance, and ensuring efficient data retrieval. This role involves working with APIs, cloud logging, data streaming, and anomaly...


  • India Forage AI Full time

    We are seeking a Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.Key...


  • India hotsourced Full time

    Job DescriptionSummaryWere a fully female-founded company on a mission to change the way people search and shop online for fashionforever Were going to spark a new era of fashion discovery, igniting confidence in everybody and every body, and to create a world where fashion confidence starts with us.We are at the beginning of an exciting journey and were...


  • Anywhere in India/Multiple Locations ALTERNATIVE PATH PRIVATE LIMITED Full time

    Company Overview">The Web Scraping Team at Alternative Path Private Limited is a dynamic and innovative group focused on delivering high-quality proprietary datasets, timely notifications, and exceptional service to support clients' daily investment decisions.Job DescriptionWe are seeking a creative and detail-oriented developer to contribute to client...


  • India Petras Solutions Private Limited Full time

    About the Job : In full growth, particularly internationally, we are looking for new collaborators to join our fabulous team A young but experienced, dynamic and complementary team: a resolutely start-up spirit Real job and career opportunities A friendly atmosphere and a climate of trust that promotes autonomy and challenge" Responsibilities : - Responsible...


  • India Adglobal360 Full time

    We are seeking an experienced Senior Software Engineer to join our team as a Web Crawling Expert. In this role, you will be responsible for designing and developing robust web crawling solutions using Python and related libraries.">The ideal candidate will have strong expertise in web scraping and data extraction using Python, proficiency in web scraping...

  • Web Data Analyst

    4 days ago


    India MrScraper Full time

    Job OverviewWe are seeking a highly skilled Web Data Analyst to join our innovative team at MrScraper. As a key member of our data extraction team, you will play a crucial role in helping us extract, process, and analyze large-scale data from the web.ResponsibilitiesDesign, build, and maintain web scraping scripts, bots, and crawlers to collect data from...


  • India Angel & Genie Full time

    Job DescriptionJob Summary:We are seeking a skilled Web Scraping Developer to design and develop scalable and efficient web scraping applications to extract data from various websites. The ideal candidate should be proficient in PHP/Python, web scraping frameworks (Scrapy, Beautiful Soup), MySQL, HTML/CSS and Javascript. This role requires strong analytical...


  • India YipitData Full time

    About YipitData:We are a fast-growing technology company backed by The Carlyle Group and Norwest Venture Partners. Our offices are located in various cities around the world, but this role can be fully remote based in India with standard work hours from 11am to 8pm IST.We cultivate a people-centric culture focused on mastery, ownership, and transparency. As...


  • India MrScraper Full time

    Large Scale Data EngineerAt MrScraper, we are seeking a seasoned Large Scale Data Engineer to join our team of experts in data extraction and automation. As a critical member of our engineering team, you will play a pivotal role in designing, building, and maintaining complex web scraping solutions.ResponsibilitiesDesign and develop large-scale web scraping...


  • India YipitData Full time

    Job DescriptionAbout YipitData:YipitData is the leading market research and analytics firm for the disruptive economy and recently raised up to $475M from The Carlyle Group at a valuation over $1B.We analyze billions of alternative data points every day to provide accurate, detailed insights on ridesharing, e-commerce marketplaces, payments, and more. Our...

  • Data Engineers

    2 weeks ago


    India Taiyō.AI Full time

    Python Developer (Data Specialization)Company OverviewTaiyō.AI is a pioneering data intelligence platform that aggregates and standardizes thousands of infrastructure data sources worldwide. Our mission is to make global infrastructure intelligence accessible and impactful for organizations in construction, engineering, and development sectors. Founded by...


  • India Alternative Path Full time

    Manager – Data Engineer – (Web Scraping) Alternative Path is seeking an experience Technical Manager to engage and mentor a team of skilled Data engineer who collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across the team and various company departments to shape and innovate new products...


  • India Forage AI Full time

    Forage AI is seeking a skilled Web Crawling Specialist to join our remote team. As a Web Crawling Specialist, you will play a crucial role in developing and refining crawlers using Python-based tools and frameworks. This involves building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality.To excel in this role,...


  • India Forage AI Full time

    We are seeking a highly skilled Data Extraction Engineer to join our team at Forage AI. In this role, you will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality.As a key member of our data acquisition team, you will develop and refine crawlers using Python-based tools and frameworks...