GCP Lead Data Engineer

4 days ago

Hyderabad, India People Prime Worldwide Full time

Lead Data Engineer – Python & GCPLocation - 10+ YearsJob Overview:We are seeking an experienced Lead Data Engineer with strong expertise in Python and Google Cloud Platform (GCP). The ideal candidate will lead the end-to-end data engineering lifecycle—from requirement gathering and solution design to development, deployment, and post-delivery support. This role involves designing scalable ETL/ELT pipelines, architecting cloud-native solutions, implementing data ingestion & transformation processes, and ensuring data quality across systems.Experience Level:10+ years of relevant IT experience in data engineering and backend development.Key Responsibilities:Design, develop, test, and maintain scalable ETL/ELT data pipelines using Python.Architect enterprise-grade data solutions using technologies such as Kafka, GKE, multi-cloud services, load balancers, APIGEE, DBT, LLMs, and DLP tools.Work extensively with GCP services, including:Dataflow – real-time & batch processingCloud Functions – serverless computeBigQuery – data warehousing & analyticsCloud Composer – workflow orchestration (Airflow)GCS – scalable storageIAM – access control & securityCloud Run – containerized workloadsBuild APIs using Python FastAPI.Work with Big Data and processing technologies:Apache Spark, Kafka, Airflow, MongoDB, Redis/BigtablePerform data ingestion, transformation, cleansing, and validation to ensure high data quality.Implement and enforce data quality checks, monitoring, and validation rules.Collaborate with data scientists, analysts, and engineering teams to understand data needs and deliver solutions.Use GitHub for version control and support CI/CD deployments.Write complex SQL queries for relational databases such as SQL Server, Oracle, and PostgreSQL.Document data pipeline designs, architecture diagrams, and operational procedures.Required Skills:10+ years of hands-on experience with Python in data engineering or backend development.Strong working knowledge of GCP services (Dataflow, BigQuery, Cloud Functions, Cloud Composer, GCS, Cloud Run).Deep understanding of data pipeline architecture, ETL/ELT processes, and data integration patterns.Experience with Apache Spark, Kafka, Airflow, FastAPI, Redis/Bigtable.Strong SQL skills with at least one enterprise RDBMS (SQL Server, Oracle, PostgreSQL).Experience in on-prem to cloud data migrations.Knowledge of GitHub and CI/CD best practices.Good to Have:Experience with Snowflake.Hands-on knowledge of Databricks.Familiarity with Azure Data Factory (ADF) or other Azure data tools.

  • Lead GCP Data Engineer

    2 weeks ago

    Hyderabad, India People Prime Worldwide Full time

    Job Title: Lead Data Engineer – Python & GCP Location: (Hyderabad) Experience: 10–12 Years (STRICTLY) Employment Type: Permanent Notice Period: Immediate / ≤ 15 Days About the Company Our client is a leading technology-driven organization specializing in advanced data solutions, cloud modernization, and scalable enterprise analytics. They empower...

  • Lead GCP Data Engineer

    2 weeks ago

    Hyderabad, India People Prime Worldwide Full time

    Job Title: Lead Data Engineer – Python & GCP Location: [Hyderabad] Experience: 10–12 Years (STRICTLY) Employment Type: Permanent Notice Period: Immediate / ≤ 15 Days About the Company Our client is a leading technology-driven organization specializing in advanced data solutions, cloud modernization, and scalable enterprise analytics. They empower...

  • GCP Lead Data Engineer

    4 weeks ago

    Hyderabad, India people prime worldwide pvt ltd Full time

    Important Note (Please Read Before Applying)🚫 Do NOT apply if:You have less than 10 years of GCP Lead Data Engineer experience.You do not have hands-on GCP, Python, Python Fast API, Kafka, Dataflow, MongoDB, Redis/Bigtable, Big Query, Git Hubexperience.You are on a notice period longer than 15 days.You are looking for remote only (role is hybrid in...

  • GCP Lead Data Engineer

    4 weeks ago

    Hyderabad, India people prime worldwide pvt ltd Full time

    Important Note (Please Read Before Applying)🚫 Do NOT apply if:You have less than 10 years of GCP Lead Data Engineer experience.You do not have hands-on GCP, Python, Python Fast API, Kafka, Dataflow, MongoDB, Redis/Bigtable, Big Query, Git Hubexperience.You are on a notice period longer than 15 days.You are looking for remote only (role is hybrid in...

  • GCP Lead Data Engineer

    4 weeks ago

    Hyderabad, India people prime worldwide pvt ltd Full time

    Important Note (Please Read Before Applying) Do NOT apply if: You have less than 10 years of GCP Lead Data Engineer experience. You do not have hands-on GCP, Python, Python Fast API, Kafka, Dataflow, MongoDB, Redis/Bigtable, Big Query, Git Hub experience. You are on a notice period longer than 15 days. You are looking for remote only (role is hybrid in...

  • GCP Lead Data Engineer

    4 weeks ago

    Hyderabad, India people prime worldwide pvt ltd Full time

    Important Note (Please Read Before Applying)Do NOT apply if:- You have less than 10 years of GCP Lead Data Engineer experience.You do not have hands-on GCP, Python, Python Fast API, Kafka, Dataflow, MongoDB, Redis/Bigtable, Big Query, Git Hubexperience.- You are on a notice period longer than 15 days.- You are looking for remote only (role is hybrid in...

  • GCP Lead Data Engineer

    3 days ago

    Hyderabad, India People Prime Worldwide Full time

    Lead Data Engineer – Python & GCP Location - 10+ Years Job Overview: We are seeking an experienced Lead Data Engineer with strong expertise in Python and Google Cloud Platform (GCP). The ideal candidate will lead the end-to-end data engineering lifecycle—from requirement gathering and solution design to development, deployment, and post-delivery support....

  • Lead Data Engineer

    6 days ago

    hyderabad, India People Prime Worldwide Full time

    Important Note (Please Read Before Applying)🚫 Do NOT apply if:You have less than 10 years or more than 12 years of total IT experienceYou do not have hands-on experience with Python and GCP Data Engineering projectsYou lack real-world experience in building and deploying ETL/ELT pipelines using GCP services (Dataflow, BigQuery, Composer, etc.)You have no...

  • Lead Data Engineer

    6 days ago

    Hyderabad, India People Prime Worldwide Full time

    Important Note (Please Read Before Applying)🚫 Do NOT apply if:You have less than 10 years or more than 12 years of total IT experienceYou do not have hands-on experience with Python and GCP Data Engineering projectsYou lack real-world experience in building and deploying ETL/ELT pipelines using GCP services (Dataflow, BigQuery, Composer, etc.)You have no...

  • Lead Data Engineer

    6 days ago

    Hyderabad, India People Prime Worldwide Full time

    Important Note (Please Read Before Applying)🚫 Do NOT apply if:You have less than 10 years or more than 12 years of total IT experienceYou do not have hands-on experience with Python and GCP Data Engineering projectsYou lack real-world experience in building and deploying ETL/ELT pipelines using GCP services (Dataflow, BigQuery, Composer, etc.)You have no...