Senior Web Scraping Engineer
2 weeks ago
About us
We're building a product that relies heavily on collecting structured data from a number of known websites. We need someone experienced who can own this part of the system end-to-end; from writing scrapers to making sure they scale and stay reliable.
This is a senior role. We're looking for someone who has already dealt with the real-world problems of web scraping and can work independently.
What you'll do
- Build and maintain scrapers for 24-30 websites (starting with 6).
- Handle authenticated sites, sessions, cookies, tokens, etc.
- Work with search endpoints and dynamic content.
- Architect the data ingestion pipeline, caching, retries, and queues.
- Make the scrapers stable and resilient even as websites change.
- Collaborate closely with our backend team to define and use APIs.
- Handle scaling, rate limits, and performance.
What we're looking for (hard requirements)
- 4–6+ years of building production-grade web scrapers
- Strong experience with both API-based scraping and browser automation (Playwright/Puppeteer)
- Familiarity with proxies/IP rotation and basic anti-bot handling
- Comfortable with auth flows (cookies, sessions, tokens)
- Ability to design pipelines: ingestion → clean → store → cache
- Can work independently and own the scraper system end-to-end
Nice to have (not required)
- Familiarity with Cloudflare's ecosystem.
- Python, (TypeScipt only), or Go; any backend language is fine, TypeScript is preferred.
Who will do well here
- You enjoy scrapers that work reliably, and predictably.
- You think in terms of systems, not quick patches.
- You can work remotely and manage your own time.
- You appreciate straightforward communication and minimal bureaucracy.
Compensation
- Base:
₹3,00,000 - 4,00,000 per month LPA). - Equity/options are available if you choose to take a lower cash component.
-
Python Web Scraping Engineer
1 week ago
Delhi, Delhi, India Cadin360 Full timeResponsibilities:• Build ethical web scrapers using Python (BeautifulSoup/Scrapy/Selenium)• Extract and clean data using Pandas & JSON• Work with YouTube API for metadata and comments• Prepare datasets for AI/LLM training
-
Senior Product Engineer – Web Platforms
1 day ago
Delhi, Delhi, India Weekday (YC W21) Full timeThis role is for one of our clientsIndustry: IT Services and IT ConsultingSeniority level: Mid-senior levelMin Experience: 4 yearsLocation: NCRJobType: full-timeWe may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment...
-
Senior Software and Web Developer
2 weeks ago
Delhi, Delhi, India Ecomdemic Full time US$ 6,00,000 - US$ 12,00,000 per yearCompany DescriptionWe suggest you enter details here.Role DescriptionThis is a full-time on-site role for a Senior Software and Web Developer, based in New Delhi. The role involves designing and developing web applications, ensuring functionality and user-friendliness. You will work on front-end and back-end development, collaborate with cross-functional...
-
Lead Research Associate
2 weeks ago
Delhi, Delhi, India ITIO Innovex Pvt. Ltd. Full time ₹ 8,000 - ₹ 25,000 per yearJob description:Location: Noida sector - 62 / Kaushambi GhaziabadTraining Salary: ₹8,000 – ₹10,000After Training Salary: ₹18,000 – ₹25,000 (Based on skills)Role Type: Full-time (Office)About the RoleLead Generation Trainees will work closely with the Business Development team to identify, research, and qualify leads using platforms...
-
Senior Data Engineer
5 days ago
Delhi, Delhi, India WIN Home Inspection Full timeABOUT THE PYTHON DATA ENGINEER ROLE:We are looking for a skilled Python Data Engineer to join our team and work on building high-performance applications and scalable data solutions. In this role, you will be responsible for designing, developing, and maintaining robust Python-based applications, optimizing data pipelines, and integrating various APIs and...
-
Senior Web Designer in Delhi
7 days ago
Delhi, Delhi, India iSearch Solution Pvt. Ltd. Full timeWe are seeking a dynamic and talented Senior Web Designer to join our team at Isearch Solution Pvt. Ltd. As a Senior Web Designer, you will be responsible for creating visually appealing and user-friendly websites and applications that meet our clients' needs and expectations.Key ResponsibilitiesDevelop creative and innovative design concepts for websites...
-
Content Analyst
3 days ago
Delhi, Delhi, India Alphasense Full timeThe Role: Content Analyst will work with our Content team to expand AlphaSense s content offering and support the needs of our clients. You will work together with a team of: product managers, content analysts, software engineers, and content vendors. Tasks will include: identifying and onboarding new content, configuring automatic content ingestion...
-
Senior Software Engineer
1 day ago
Delhi, Delhi, India Wundrsight Health Full timeJob Title: Senior Software EngineerLocation:Delhi NCR (On-site)Job Type:Full-TimeExperience:5+ years (Product-based companies)Compensation:Competitive salary + ESOPsAbout Wundrsight HealthWundrsight Health builds AI and XR-driven digital therapeutics for mental health, neuro-rehab, and behavioral healthcare. Our clinician-facing platform powers AI...
-
Web Developer
2 weeks ago
Delhi, Delhi, India Total Shape Full time ₹ 13,80,000 - ₹ 16,50,000 per yearThe Role: Job Summary:We're offering a competitive salary of $9,500 to $11,500 monthly for our next Web DeveloperWe're looking for someone with the talent and experience to help us create and maintain a top-notch website for our customers. You'll be in charge of developing both the front and the back end of the website, ensuring everything runs smoothly and...
-
Senior Development Engineer
1 day ago
Delhi, Delhi, India Avizva Full timeBackdropAVIZVA is a Healthcare Technology Organization that harnesses technology to simplify, accelerate, & optimize the way healthcare enterprises deliver care. Established in 2011, we have served as strategic enablers for healthcare enterprises, helping them enhance their overall care delivery.With over 14 years of expertise, we have engineered more than...