Data engineer/Web scapper

6 days ago

Lucknow, Uttar Pradesh, India QuickBid Systems Pvt. Ltd. Full time ₹ 9,00,000 - ₹ 12,00,000 per year

Job Title: Web Scraper / Data Engineer

Location: Remote | Experience: 1+ years

Responsibilities:

Design, develop, and maintain robust web scrapers for structured and dynamic data sources.

Implement headless browser automation using Playwright, Puppeteer, or Selenium to extract content reliably.

Manage session handling, cookies, CAPTCHAs, and anti-bot mechanisms to ensure uninterrupted data collection.

Transform, clean, and structure extracted data into usable formats such as JSON, CSV, or directly into databases.

Integrate scraper outputs with internal APIs, analytics pipelines, or downstream applications.

Monitor scrapers for failures, performance bottlenecks, and data inconsistencies; implement automated alerting and recovery mechanisms.

Ensure ethical scraping practices, compliance with site policies, and respect for data usage rules.

Optimize scraping performance and minimize resource usage, including efficient handling of large-scale data.

Skills & Experience:

Proficient in or Python with strong programming fundamentals.

Hands-on experience with browser automation frameworks (Playwright, Puppeteer, Selenium).

Deep understanding of HTTP, REST APIs, JSON parsing, and web protocols.

Knowledge of proxy rotation, anti-detection techniques, rate limiting, and IP management.

Experience with relational (PostgreSQL) and NoSQL (MongoDB) databases.

Strong problem-solving skills and ability to debug complex automation scenarios.

Bonus: Experience with cloud platforms (Azure/AWS) and building real-time data pipelines.

Familiarity with containerization (Docker) and CI/CD pipelines is a plus.

Perks:

Fully remote and flexible work environment.

Opportunity to work on high-scale, real-time data automation and analytics projects.

Collaborate with a dynamic team solving complex challenges in data engineering.

  • Data Extraction Analyst

    19 hours ago

    Lucknow, Uttar Pradesh, India Ardem Data Services Full time ₹ 2,00,000 - ₹ 4,00,000 per year

    This is a data extraction project focused on researching and collecting information related to Business Lawsuits, Law Firms, Consumer Debt, and Debt Buyers. The work involves extracting structured data from publicly available legal databases.Key Responsibilities:Use the provided website url: to perform data lookups.Extract and record accurate information...

  • Web Development Intern

    6 days ago

    Lucknow, Uttar Pradesh, India Innovative Modern Engineers Pvt. Ltd. Full time ₹ 4,00,000 - ₹ 6,00,000 per year

    Company DescriptionInnovative Modern Engineers Pvt. Ltd. specializes in developing software solutions and online services, particularly in the field taxi/cabs and food catering through online network management. By fostering research and development, the company aims to advance software and online support systems. Innovative Modern Engineers Pvt. Ltd. is...

  • Data Engineer

    1 week ago

    Lucknow, Uttar Pradesh, India CloudKeeper Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    DescriptionAbout CLOUDKEEPER :CloudKeeper is a cloud cost optimization partner that combines the power of group buying & commitments management, expert cloud consulting & support, and an enhanced visibility & analytics platform to reduce cloud cost & help businesses maximize the value from AWS, Microsoft Azure, & Google Cloud. A certified AWS Premier...

  • Data Quality Engineer

    1 week ago

    Lucknow, Uttar Pradesh, India Ditstek Innovation Pvt. Ltd. Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Job Title : Data Quality Engineer (Data QE) Job Description :We are seeking a skilled and experienced Data Quality Engineer (Data QE) to join our team. The ideal candidate will have a strong background in Quality Engineering, ETL and database testing, and hands-on expertise with Python, PySpark, SQL, and Databricks. You will be responsible for designing...

  • React Frontend Developer

    2 days ago

    Lucknow, Uttar Pradesh, India Ardem Data Services Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    React Frontend Developer Mobile Responsive UI & TestingLocation: RemoteShift: EST hoursCompany: ARDEM IncorporatedPosition Overview:ARDEM is seeking an experienced Frontend Developer to enhance our existing React-based loan amortization desktop application by extending it with full mobile responsiveness and comprehensive testing. This role focuses on...

  • Search Engine Optimization Specialist

    1 week ago

    Lucknow, Uttar Pradesh, India Vicdigit Technologies Pvt Ltd Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Company DescriptionVICDIGIT Technologies, based in Lucknow, is a leader in providing innovative IT solutions. We offer a wide range of services, including software development, app development, web development, SEO, web design, social media, PPC, email marketing, web hosting, and content writing. Our expert team ensures client satisfaction by delivering...

  • Search Engine Optimization Specialist

    4 days ago

    Lucknow, Uttar Pradesh, India eMarketters Full time ₹ 2,50,000 - ₹ 7,50,000 per year

    Job Title: SEO Specialist - Onsite OnlyLocation:Onsite – LucknowExperience Required:2+ YearsCompany:eMarkettersAbout eMarketterseMarketters is a fast-growing digital marketing agency based in Lucknow, delivering result-oriented solutions to a diverse range of clients. We specialize in performance marketing, branding, content creation, and strategic digital...

  • Search Engine Optimization Executive

    22 hours ago

    Lucknow, Uttar Pradesh, India Triangle Mind Technologies Private Limited Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    NO ONLINE INTERVIEWNO WORK FROM HOMECompany DescriptionTriangle Mind Technologies Private Limited is an India based IT company specializing in ERP, PeopleSoft, Oracle Apps, web designing, web-applications, and IT training services. With expertise in various technologies like PeopleSoft, Oracle Apps, and ERP, we provide customized web and software solutions...

  • Lead Software Engineer

    20 hours ago

    Lucknow, Uttar Pradesh, India Agoda Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    About AgodaAgoda is an online travel booking platform for accommodations, flights, and more. We build and deploy cutting-edge technology that connects travelers with a global network of 4.7M hotels and holiday properties worldwide, plus flights, activities, and more. Based in Asia and part of Booking Holdings, our 7,100+ employees representing 95+...

  • Staff Data Scientist I

    2 days ago

    Lucknow, Uttar Pradesh, India InMobi Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    InMobi Advertising is a global technology leader helping marketers win the moments that matter. Our advertising platform reaches over 2 billion people across 150+ countries and turns real-time context into business outcomes, delivering results grounded in privacy-first principles. Trusted by 30,000+ brands and leading publishers, InMobi is where...