Senior Data Pipeline Engineer
2 weeks ago
Our company is seeking a Python Web Scraping Automation Specialist with 3-10 years of experience.Job Description:We require a skilled individual to develop and maintain high-scale automated scraping workflows for dynamic websites.The ideal candidate will implement browser automation solutions using Playwright, Selenium, or Puppeteer for complex user flows and asynchronous rendering.They must integrate CAPTCHA-solving services, proxy rotation systems, and advanced anti-detection mechanisms.A robust ETL-style data pipeline ensuring data quality, monitoring, retries, and error handling is essential.Required Skills and Qualifications:Strong hands-on experience in managing cookies, sessions, and local storage to maintain state and bypass authentication/anti-bot systems.Ability to solve CAPTCHAs programmatically using third-party or AI-based solutions.Proven expertise in proxy rotation, IP management, and fingerprinting techniques to avoid detection and rate limits.Familiarity with SQL/NoSQL databases for structured data storage and experience working with AWS/GCP/Azure is preferred.Excellent debugging, analytical, and problem-solving skills are necessary.Benefits:This full-time role offers a competitive salary and benefits package.The opportunity to work on challenging projects and contribute to the growth of our organization.How to Apply:Please submit your resume and cover letter outlining your relevant experience and qualifications.We look forward to reviewing your application and discussing this opportunity further.
-
Senior Data Pipeline Developer
3 days ago
nagpur, India beBeeDataEngineering Full timeKey Data Engineering ResponsibilitiesWe seek a skilled Data Engineer to design and develop scalable data pipelines using modern data engineering tools and frameworks.Pipeline Development: Develop efficient ETL/ELT workflows for structured, semi-structured, and unstructured data.Data Ingestion: Implement ingestion, transformation, storage, and processing...
-
Data Pipeline Irc173004
20 hours ago
Nagpur, India GlobalLogic Full timeJob: - IRC173004- Location: - India - Nagpur- Designation: - Associate Consultant- Experience: - 5-10 years- Function: - Engineering**Description**: seeking an experienced data pipelines engineer for our data engineering organization. As a member of our team, you will develop highly-performant, scalable and available data pipelines for bp internal - and...
-
Data Pipelines Architect
5 days ago
nagpur, India beBeeDataIntegration Full timeData Integration ProfessionalWe are seeking a highly skilled Data Integration Professional to join our team. In this role, you will be responsible for analyzing legacy mainframe data flows, documenting source-to-target mappings, and supporting the migration of data pipelines to cloud-based platforms.The key responsibilities include:Analyzing mainframe JCL...
-
Data Engineering Lead
2 weeks ago
nagpur, India beBeeDataEngineer Full timeSenior Data Engineer Position OverviewWe are seeking a highly skilled Senior Data Engineer to lead the design and implementation of scalable data pipelines and cloud-based solutions. The ideal candidate will have extensive experience in data engineering, including 9-12 years of designing and building large-scale data pipelines.
-
Senior Data Engineer
1 week ago
Nagpur, India Coditas Full timeWe are looking for Senior Data Engineers who have the right attitude, aptitude, skills, empathy, compassion, and hunger for learning. Build products in the data analytics space. A passion for shipping high-quality data products, interest in the data products space; curiosity about the bigger picture of building a company, product development and its people....
-
Cloud-Native Data Pipeline Specialist
2 weeks ago
nagpur, India beBeeDataEngineer Full timeWe are seeking experienced Data Engineers to join our cloud-native data platform and analytics teams in fully remote freelance roles.Job OverviewWe are hiring multiple positions for Data Engineers who can design, build, and maintain scalable data pipelines using modern data engineering tools and frameworks. These long-term freelance roles are ideal for...
-
nagpur, India beBeeDataEngineer Full timeSenior Data Engineer Job OpportunityWe are seeking a highly skilled Senior Data Engineer with expertise in Telecommunications data migration to lead initiatives from on-premises environments to cloud platforms.Data Migration Strategy & Execution: Design, develop, and implement the data migration strategy from on-prem data warehouses to cloud platforms.Data...
-
nagpur, India beBeeDataEngineering Full timeSenior Data Engineer Role OverviewAs a senior data engineer, you will be responsible for designing, building, and scaling robust data solutions on Microsoft Azure. This involves owning modern data pipelines and models that power analytics and reporting across the business.Key ResponsibilitiesDesign and implement scalable data solutions using Microsoft Azure...
-
Advanced Data Pipeline Developer
2 weeks ago
nagpur, India beBeeDataEngineer Full timeAbout the RoleWe are seeking a skilled Data Engineer to design, build and maintain scalable data pipelines, transformations and applications within the Palantir Foundry ecosystem.The ideal candidate will have experience working with Palantir AIP features such as RAG workflows, copilots or agent-based applications.
-
Senior Data Pipeline Specialist
1 week ago
nagpur, India beBeeData Full timeETL Tester Job DescriptionAs a seasoned ETL tester, you play a pivotal role in ensuring the seamless operation of an organization's data pipelines.The ideal candidate for this position will have strong analytical skills, attention to detail, and exceptional problem-solving abilities.Identify and report defects in the ETL processDevelop and maintain...