Lead Data Engineer – Python

3 days ago

Bareilly, India Whatjobs IN C2 Full time

Job Title: Lead Data Engineer – Python & GCP Experience: 10+ Years Location: Hyderabad Interview Mode: Face-to-Face About the Role We are seeking an experienced Lead Data Engineer with deep expertise in Python and Google Cloud Platform (GCP) . This role involves designing and building scalable data solutions, developing ETL/ELT pipelines, and leading the architecture and implementation of complex data workflows. You’ll collaborate with business and technical teams to deliver high-quality data solutions that support analytics, reporting, and business intelligence needs. Experience Level: 10+ years in IT, with hands-on experience in data engineering and cloud-based systems. Key Responsibilities Design, develop, and maintain robust ETL/ELT data pipelines using Python. Architect enterprise-grade data solutions using GCP services and other technologies. Work extensively with GCP services such as Dataflow, BigQuery, Cloud Functions, Composer (Airflow), GCS, IAM, and Cloud Run. Build APIs using Python (FastAPI Framework). Handle real-time and batch data processing using Apache Kafka and Apache Spark. Utilize storage systems like MongoDB, Redis, and Bigtable. Manage deployments on GKE and Cloud Run with CI/CD integration (GitHub). Ensure data quality, validation, and transformation for accurate, consistent data delivery. Collaborate closely with data scientists, analysts, and cross-functional teams. Write advanced SQL queries for data extraction and validation. Document designs, data flow diagrams, and standard operating procedures. Required Skills Strong programming experience in Python. Solid understanding of GCP services (Dataflow, BigQuery, Cloud Functions, Composer, etc.). Hands-on experience with Apache Spark, Airflow, Kafka, and Redis. Proficiency with SQL and database systems like SQL Server, Oracle, or PostgreSQL. Knowledge of CI/CD practices and version control (GitHub). Expertise in designing and managing large-scale data architectures. Experience with cloud data migrations from on-premise sources. Good to Have Experience with Snowflake and Databricks. Familiarity with Azure Data Factory (ADF) and other Azure tools.

  • Enterprise Data Architect

    2 days ago

    Bareilly, India Aptus Data Labs Full time

    Company DescriptionAptus Data Labs is a leading Data and AI company specializing in Pharma, Manufacturing & Supply Chain, Banking & FinTech, and Technology domains. We offer innovative analytical solutions and consulting services to help businesses make quick, data-driven decisions essential for growth and sustainability in evolving industries. Leveraging...

  • Enterprise Data Architect

    2 days ago

    Bareilly, India Aptus Data Labs Full time

    Company DescriptionAptus Data Labs is a leading Data and AI company specializing in Pharma, Manufacturing & Supply Chain, Banking & FinTech, and Technology domains. We offer innovative analytical solutions and consulting services to help businesses make quick, data-driven decisions essential for growth and sustainability in evolving industries. Leveraging...

  • Backend Python Engineer

    2 weeks ago

    Bareilly, India Turing Full time

    Role Overview:We’re looking for experienced Python engineers to collaborate with one of the world’s top Large Language Model (LLM) companies. Your work will directly help improve how AI models think, reason, and code.In this role, you’ll generate and evaluate high-quality data used to fine-tune and benchmark LLMs. You’ll design prompts, analyze model...

  • Python Developer

    6 days ago

    Bareilly, India MyRemoteTeam Inc Full time

    About UsMyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent. We empower businesses by providing world-class software engineers, operations support, and infrastructure to help them grow faster and better.Position: Senior Data Engineer (Python Coder)Location: India ( Remote )Work Commitment: 40 Hrs...

  • Data Engineer

    2 weeks ago

    Bareilly, India Recro Full time

    Key Responsibilities include: Data Pipeline Engineering: Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake. AI Data Enablement: Collaborate with AI Consultants to prepare and manage datasets for supervised, unsupervised, and synthetic learning workflows. POC Support:...

  • Sr data engineer

    4 weeks ago

    Bareilly, India Mitchell Martin Inc. Full time

    Job Title: Senior Data EngineerLocation: Remote in India Employment Type: Full-timeAbout the RoleWe are looking for a Senior Data Engineer to design, build, and optimize data pipelines and systems that power our analytics, reporting, and data-driven decision-making. The ideal candidate will have hands-on experience with Python, Dagster (or equivalent...

  • Senior Data Engineer

    1 day ago

    bareilly, India Intellify Solutions Full time

    Title: Senior Data EngineerLocation: Remote, IndiaTotal years of experience: 10+ years Requirements:● Required: Bachelor's degree or relevant experience● Required: Prior experience as a software engineer or data engineer in afast-paced, technical, problem-solving environment● Required: Cloud Data Warehouse experience - Snowflake● Required: Expert in...

  • Only 24h Left! Data Engineer

    4 weeks ago

    Bareilly, India Tata Consultancy Services Full time

    Dear Candidate, Greetings from TCS !!! TCS is hiring for AWS Data engineer, please find the below JD.. Experience range – 6 to 8 years Location - Hyderabad/ Pune/ Indore Skills Required - AWS, Pyspark, EMR, Glue Role& Responsibilities – · AMI Rehydration, Cloud formation, EMR, Glue, S3, Lambda/Python, ELB and other AWS components knowledge · AWS...

  • Senior Data Integration Engineer

    3 weeks ago

    Bareilly, India Indsafri Full time

    Job Title- Senior Data Integration Engineer Experience -6+YearsJob Summary We are seeking a highly experienced and strategic Senior Data Integration Engineer to architect, build, and manage the data pipelines that power our customer intelligence ecosystem. In this critical role, you will be the subject matter expert responsible for designing and implementing...

  • Data Science Lead

    2 days ago

    bareilly, India beBeeMachine Full time

    Job OverviewWe are seeking an experienced Senior Data Scientist to develop and implement cutting-edge machine learning models for our intelligent decision systems.Design and deploy AI-driven solutions for dynamic pricing, availability forecasting, and real-time hotel demand optimization.Develop and maintain large-scale data pipelines and feature stores to...