
Web Scraping Engineer
6 days ago
We're looking for a Web Scraping Engineer to join our team and help us extract, process, and analyze large-scale data from the web. You'll be responsible for developing efficient, scalable, and resilient scraping solutions to gather structured and unstructured data from diverse online sources. If you thrive on solving complex problems, working with automation, and building data pipelines, this role is for you
Key Responsibilities:- Design, build, and maintain web scraping scripts, bots, and crawlers to collect data from websites, APIs, and other online sources.
- Ensure scrapers are efficient, scalable, and resilient to website structure changes and anti-bot mechanisms.
- Develop data parsing, cleaning, and structuring workflows for downstream applications.
- Optimize scraping performance, handling concurrency, rate-limiting, and distributed architectures.
- Work with stakeholders to understand data needs and translate them into automated scraping solutions.
- Monitor, troubleshoot, and maintain scrapers to ensure data accuracy and integrity.
- Implement best practices for ethical web scraping and comply with legal requirements (e.g., robots.txt, data privacy regulations).
- Programming: Proficiency in Python and libraries like Scrapy, BeautifulSoup, Selenium, or Playwright.
- Web Technologies: Strong understanding of HTML, CSS, JavaScript, and browser behavior.
- Networking & HTTP: Familiarity with REST APIs, proxies, headers, and cookies.
- Data Processing: Experience with pandas, regex, JSON, XML, and database integration (SQL/NoSQL).
- Automation & Scaling: Knowledge of asynchronous programming, multiprocessing, and distributed computing frameworks (e.g., Celery, Kafka).
- Security & Compliance: Awareness of anti-scraping mechanisms (CAPTCHAs, bot detection) and ethical scraping guidelines.
- Work on cutting-edge data extraction and automation projects.
- Flexible work environment with remote opportunities.
- Competitive salary, benefits, and career growth opportunities.
- Join a passionate, innovative team that values data-driven decision-making.
IF ANYONE IS INTERESTED, PLEASE SEND YOUR DESIRED CV AND PORTFOLIO TO HELLO@PENATEAM.COM. CANDIDATES WHO DID NOT SUBMIT THIS, WILL NOT BE CONSIDERED
-
Web Scraping Engineer
4 days ago
India MrScraper Full timeJob Overview: We are seeking a skilled Web Scraping Engineer to help us extract, process, and analyze large-scale data from the web. This role involves developing efficient, scalable, and resilient scraping solutions to gather structured and unstructured data from diverse online sources. Key Responsibilities: Design, build, and maintain web scraping scripts,...
-
Python Web Scraping Developer
3 hours ago
India Forage AI Full timeWe are seeking a talented Python Web Scraping Developer to join our team at Forage AI. As a Python Web Scraping Developer, you will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality.You will develop and refine crawlers using Python-based tools and frameworks such as Scrapy, and...
-
Web Scraping Expert
5 days ago
India YipitData Full timeCrafting Resilient Web ScrapersYipitData is seeking a skilled Web Scraping Expert to join our Data Solutions team. In this transformative role, you will design, refactor, and maintain web scrapers that drive critical reports across our organization, ensuring data ingestion processes are resilient, efficient, and scalable.About the JobCollaborate with...
-
Web Scraping Expert
7 days ago
India brightedge Full timeData Engineer Job DescriptionThe successful candidate will be responsible for developing and maintaining data collection pipelines to efficiently ingest and store structured and unstructured data. They will also implement web scraping techniques and APIs for real-time and batch data retrieval. Additionally, they will optimize data storage and query...
-
Data Scientist
4 days ago
India Adglobal360 Full timeWe are seeking a highly skilled Data Scientist to join our team as a Web Scraping Specialist. In this role, you will be responsible for designing and developing robust web scraping solutions using Python and related libraries.">The ideal candidate will have strong expertise in web scraping and data extraction using Python, proficiency in web scraping...
-
India brightedge Full timeWe are seeking a Backend Engineer to design, implement, and optimize data collection, storage, and processing pipelines. The ideal candidate will have expertise in handling structured and unstructured data, optimizing query performance, and ensuring efficient data retrieval. This role involves working with APIs, cloud logging, data streaming, and anomaly...
-
Web Crawling Engineer
4 days ago
India Forage AI Full timeWe are seeking a Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.Key...
-
15h Left Web Data Scraping Engineer
1 week ago
India hotsourced Full timeJob DescriptionSummaryWere a fully female-founded company on a mission to change the way people search and shop online for fashionforever Were going to spark a new era of fashion discovery, igniting confidence in everybody and every body, and to create a world where fashion confidence starts with us.We are at the beginning of an exciting journey and were...
-
Web Scraping Developer
6 days ago
Anywhere in India/Multiple Locations ALTERNATIVE PATH PRIVATE LIMITED Full timeCompany Overview">The Web Scraping Team at Alternative Path Private Limited is a dynamic and innovative group focused on delivering high-quality proprietary datasets, timely notifications, and exceptional service to support clients' daily investment decisions.Job DescriptionWe are seeking a creative and detail-oriented developer to contribute to client...
-
Python Web Scraping Developer
7 days ago
India Petras Solutions Private Limited Full timeAbout the Job : In full growth, particularly internationally, we are looking for new collaborators to join our fabulous team A young but experienced, dynamic and complementary team: a resolutely start-up spirit Real job and career opportunities A friendly atmosphere and a climate of trust that promotes autonomy and challenge" Responsibilities : - Responsible...
-
Senior Software Engineer
4 days ago
India Adglobal360 Full timeWe are seeking an experienced Senior Software Engineer to join our team as a Web Crawling Expert. In this role, you will be responsible for designing and developing robust web crawling solutions using Python and related libraries.">The ideal candidate will have strong expertise in web scraping and data extraction using Python, proficiency in web scraping...
-
Web Data Analyst
4 days ago
India MrScraper Full timeJob OverviewWe are seeking a highly skilled Web Data Analyst to join our innovative team at MrScraper. As a key member of our data extraction team, you will play a crucial role in helping us extract, process, and analyze large-scale data from the web.ResponsibilitiesDesign, build, and maintain web scraping scripts, bots, and crawlers to collect data from...
-
India Angel & Genie Full timeJob DescriptionJob Summary:We are seeking a skilled Web Scraping Developer to design and develop scalable and efficient web scraping applications to extract data from various websites. The ideal candidate should be proficient in PHP/Python, web scraping frameworks (Scrapy, Beautiful Soup), MySQL, HTML/CSS and Javascript. This role requires strong analytical...
-
Web Scraping Engineer
5 days ago
India YipitData Full timeAbout YipitData:We are a fast-growing technology company backed by The Carlyle Group and Norwest Venture Partners. Our offices are located in various cities around the world, but this role can be fully remote based in India with standard work hours from 11am to 8pm IST.We cultivate a people-centric culture focused on mastery, ownership, and transparency. As...
-
Large Scale Data Engineer
3 days ago
India MrScraper Full timeLarge Scale Data EngineerAt MrScraper, we are seeking a seasoned Large Scale Data Engineer to join our team of experts in data extraction and automation. As a critical member of our engineering team, you will play a pivotal role in designing, building, and maintaining complex web scraping solutions.ResponsibilitiesDesign and develop large-scale web scraping...
-
Web Crawling Specialist
5 days ago
India YipitData Full timeJob DescriptionAbout YipitData:YipitData is the leading market research and analytics firm for the disruptive economy and recently raised up to $475M from The Carlyle Group at a valuation over $1B.We analyze billions of alternative data points every day to provide accurate, detailed insights on ridesharing, e-commerce marketplaces, payments, and more. Our...
-
Data Engineers
2 weeks ago
India Taiyō.AI Full timePython Developer (Data Specialization)Company OverviewTaiyō.AI is a pioneering data intelligence platform that aggregates and standardizes thousands of infrastructure data sources worldwide. Our mission is to make global infrastructure intelligence accessible and impactful for organizations in construction, engineering, and development sectors. Founded by...
-
Manager – Data Engineer –
4 weeks ago
India Alternative Path Full timeManager – Data Engineer – (Web Scraping) Alternative Path is seeking an experience Technical Manager to engage and mentor a team of skilled Data engineer who collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across the team and various company departments to shape and innovate new products...
-
Web Crawling Specialist
6 hours ago
India Forage AI Full timeForage AI is seeking a skilled Web Crawling Specialist to join our remote team. As a Web Crawling Specialist, you will play a crucial role in developing and refining crawlers using Python-based tools and frameworks. This involves building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality.To excel in this role,...
-
Data Extraction Engineer
11 hours ago
India Forage AI Full timeWe are seeking a highly skilled Data Extraction Engineer to join our team at Forage AI. In this role, you will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality.As a key member of our data acquisition team, you will develop and refine crawlers using Python-based tools and frameworks...