GCP Lead Data Engineer

7 days ago

bangalore, India People Prime Worldwide Full time

Lead Data Engineer – Python & GCPLocation - 10+ YearsJob Overview:We are seeking an experienced Lead Data Engineer with strong expertise in Python and Google Cloud Platform (GCP). The ideal candidate will lead the end-to-end data engineering lifecycle—from requirement gathering and solution design to development, deployment, and post-delivery support. This role involves designing scalable ETL/ELT pipelines, architecting cloud-native solutions, implementing data ingestion & transformation processes, and ensuring data quality across systems.Experience Level:10+ years of relevant IT experience in data engineering and backend development.Key Responsibilities:Design, develop, test, and maintain scalable ETL/ELT data pipelines using Python.Architect enterprise-grade data solutions using technologies such as Kafka, GKE, multi-cloud services, load balancers, APIGEE, DBT, LLMs, and DLP tools.Work extensively with GCP services, including:Dataflow – real-time & batch processingCloud Functions – serverless computeBigQuery – data warehousing & analyticsCloud Composer – workflow orchestration (Airflow)GCS – scalable storageIAM – access control & securityCloud Run – containerized workloadsBuild APIs using Python FastAPI.Work with Big Data and processing technologies:Apache Spark, Kafka, Airflow, MongoDB, Redis/BigtablePerform data ingestion, transformation, cleansing, and validation to ensure high data quality.Implement and enforce data quality checks, monitoring, and validation rules.Collaborate with data scientists, analysts, and engineering teams to understand data needs and deliver solutions.Use GitHub for version control and support CI/CD deployments.Write complex SQL queries for relational databases such as SQL Server, Oracle, and PostgreSQL.Document data pipeline designs, architecture diagrams, and operational procedures.Required Skills:10+ years of hands-on experience with Python in data engineering or backend development.Strong working knowledge of GCP services (Dataflow, BigQuery, Cloud Functions, Cloud Composer, GCS, Cloud Run).Deep understanding of data pipeline architecture, ETL/ELT processes, and data integration patterns.Experience with Apache Spark, Kafka, Airflow, FastAPI, Redis/Bigtable.Strong SQL skills with at least one enterprise RDBMS (SQL Server, Oracle, PostgreSQL).Experience in on-prem to cloud data migrations.Knowledge of GitHub and CI/CD best practices.Good to Have:Experience with Snowflake.Hands-on knowledge of Databricks.Familiarity with Azure Data Factory (ADF) or other Azure data tools.

  • Lead GCP Data Engineer

    4 days ago

    Bangalore, India Impetus Full time

    Job Title: Lead Data Engineer – GCP (BigQuery • Composer • Python • PySpark) Location: Bengaluru Experience: 8+ years (data engineering / analytics engineering), with previous lead responsibilities About the Role: You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform. You will manage a team of data...

  • Lead GCP Data Engineer

    1 day ago

    bangalore, India Impetus Full time

    Job Title: Lead Data Engineer – GCP (BigQuery • Composer • Python • PySpark)Location: BengaluruExperience: 8+ years (data engineering / analytics engineering), with previous lead responsibilitiesAbout the Role:You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform. You will manage a team of data...

  • Lead GCP Data Engineer

    1 day ago

    bangalore district, India Impetus Full time

    Job Title: Lead Data Engineer – GCP (BigQuery • Composer • Python • PySpark) Location: Bengaluru Experience: 8+ years (data engineering / analytics engineering), with previous lead responsibilities About the Role: You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform. You will manage a team of data...

  • Lead Data Engineer GCP

    4 weeks ago

    bangalore, India Egen Full time

    Job Overview: We are looking for a skilled and motivated Lead Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team. The ideal candidate will be responsible for requirements gathering, designing, architecting the solution, developing, and maintaining robust and scalable ETL (Extract,...

  • Lead GCP Data Engineer

    7 days ago

    bangalore, India People Prime Worldwide Full time

    Important Note (Please Read Before Applying)🚫 Do NOT apply if: • You have less than 10 years of relevant IT experience • You lack hands-on Python + GCP project experience • You have no real-time experience with Dataflow, BigQuery, Airflow, or Spark • You are not experienced in architecting or designing data pipelines • You cannot work in the...

  • Lead Data Engineer – Python

    7 days ago

    bangalore, India People Prime Worldwide Full time

    Job Title: Lead Data Engineer – Python & GCPExperience: 10+ YearsLocation: HyderabadInterview Mode: Face-to-FaceAbout the RoleWe are seeking an experienced Lead Data Engineer with deep expertise in Python and Google Cloud Platform (GCP). This role involves designing and building scalable data solutions, developing ETL/ELT pipelines, and leading the...

  • GCP Data Engineer

    3 days ago

    bangalore, India Adastra Full time

    Job DescriptionWe are looking for a proactive and solution-oriented GCP Data Engineer to join our team. This role requires hands-on experience in Google Cloud Platform (GCP), especially with BigQuery and Airflow, alongside strong SQL and Python skills. The ideal candidate is a self-starter who thrives in solving problems independently, communicates...

  • GCP Data Engineer

    3 days ago

    bangalore, India Adastra Full time

    Job Description We are looking for a proactive and solution-oriented GCP Data Engineer to join our team. This role requires hands-on experience in Google Cloud Platform (GCP), especially with BigQuery and Airflow, alongside strong SQL and Python skills. The ideal candidate is a self-starter who thrives in solving problems independently, communicates...

  • GCP Data engineer

    1 week ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from Tata Consulting Services TCS is Hiring for GCP Data engineer Experience : 5-10 years Location: PAN INDIA Please find the JD below Required Technical Skill - GCP Data engineer, Pyspark, Data proc, Data flow

  • GCP Lead Engineer

    2 days ago

    bangalore, India Impetus Full time

    Qualifications8-11 years of IT experience range is preferred.BE/B.Tech/MCA/MS-IT/M.Tech or any other Engineering degrees in related fieldsThe candidate should have extensive production experience (5 Years) with GCP, Other cloud experience would be a strong bonus.Strong background in Data engineering 4-5 Years of exp in Big Data technologies including,...