GCP Lead Data Engineer

3 days ago

Bharatpur, India People Prime Worldwide Full time

Lead Data Engineer – Python & GCPLocation - 10+ YearsJob Overview:We are seeking an experienced Lead Data Engineer with strong expertise in Python and Google Cloud Platform (GCP). The ideal candidate will lead the end-to-end data engineering lifecycle—from requirement gathering and solution design to development, deployment, and post-delivery support. This role involves designing scalable ETL/ELT pipelines, architecting cloud-native solutions, implementing data ingestion & transformation processes, and ensuring data quality across systems.Experience Level:10+ years of relevant IT experience in data engineering and backend development.Key Responsibilities:Design, develop, test, and maintain scalable ETL/ELT data pipelines using Python.Architect enterprise-grade data solutions using technologies such as Kafka, GKE, multi-cloud services, load balancers, APIGEE, DBT, LLMs, and DLP tools.Work extensively with GCP services, including:Dataflow – real-time & batch processingCloud Functions – serverless computeBigQuery – data warehousing & analyticsCloud Composer – workflow orchestration (Airflow)GCS – scalable storageIAM – access control & securityCloud Run – containerized workloadsBuild APIs using Python FastAPI.Work with Big Data and processing technologies:Apache Spark, Kafka, Airflow, MongoDB, Redis/BigtablePerform data ingestion, transformation, cleansing, and validation to ensure high data quality.Implement and enforce data quality checks, monitoring, and validation rules.Collaborate with data scientists, analysts, and engineering teams to understand data needs and deliver solutions.Use GitHub for version control and support CI/CD deployments.Write complex SQL queries for relational databases such as SQL Server, Oracle, and PostgreSQL.Document data pipeline designs, architecture diagrams, and operational procedures.Required Skills:10+ years of hands-on experience with Python in data engineering or backend development.Strong working knowledge of GCP services (Dataflow, BigQuery, Cloud Functions, Cloud Composer, GCS, Cloud Run).Deep understanding of data pipeline architecture, ETL/ELT processes, and data integration patterns.Experience with Apache Spark, Kafka, Airflow, FastAPI, Redis/Bigtable.Strong SQL skills with at least one enterprise RDBMS (SQL Server, Oracle, PostgreSQL).Experience in on-prem to cloud data migrations.Knowledge of GitHub and CI/CD best practices.Good to Have:Experience with Snowflake.Hands-on knowledge of Databricks.Familiarity with Azure Data Factory (ADF) or other Azure data tools.

  • Data Quality Engineer

    1 week ago

    Bharatpur, India Whatjobs IN C2 Full time

    Looking for Data Quality Engineer| Bangalore to join a team of rockstar developers. The candidate should have a minimum of 8+yrs of experience. About CodeVyasa: CodeVyasa is a mid-sized product engineering company that works with top-tier product/solutions companies such as McKinsey, Walmart, RazorPay, Swiggy, and others. We are about 550+ people strong and...

  • Data Engineer Ii

    1 week ago

    Bharatpur, India Whatjobs IN C2 Full time

    Job Summary: Building on the foundation of the SDE-I role, the DE- II position takes on a greater level of responsibility and leadership. You'll play a crucial role in driving the evolution and efficiency of our data collection and analytics platform, capable of handling terabyte-scale data and billions of data points. Key Responsibilities Lead the design,...

  • Lead Software Development Engineer

    5 days ago

    Bharatpur, India RevX Full time

    Software Lead Software EngineerAbout RevXRevX helps app businesses acquire and reengage users via programmatic to retain, monetize, and accelerate revenue. We're all about taking your app businesses to a new growth level. We rely on data science, innovative technology, and AI, and a skilled team, to create and deliver seamless ad experiences to delight your...

  • Data/ML Engineer

    4 weeks ago

    Bharatpur, India Lilo Full time

    Lilo is revolutionizing procurement for Commercial Real Estate (CRE) businesses by providing a single platform to automate and optimize procurement processes using AI. From hotels to gyms, schools, and senior living homes, we streamline workflows like invoicing, vendor management, and price comparisons, saving our clients both time and money. We are already...

  • Data Engineer

    1 week ago

    Bharatpur, India Whatjobs IN C2 Full time

    Job Title: Azure Data Engineer Location: Bangalore (Hybrid) Experience: 5-10 Years (STRICTLY) Employment Type: Full-Time / Permanent Notice Period: Immediate to 15 Days Preferred Interview Mode : F2F About the Company Our client is a leading financial technology transformation partner helping global banks and financial institutions streamline their lending...

  • Network Team Lead

    3 days ago

    Bharatpur, India Yotta Data Services Private Limited Full time

    Role OverviewWe are looking for an energetic and dynamic Network Team Lead with 7 plus years of experience in a data center networking domain with 1-3 years of experience in Software Defined Networking(SDN) specifically in a private cloud & virtualization domain.ResponsibilitiesYour responsibilities in this role will be centered around the...

  • Software Development Engineer II

    1 week ago

    Bharatpur, India ClearDemand Full time

    Job Summary: Data is the foundation of our business, and your work will ensure that we continue to deliver high-quality competitive intelligence at scale. Web platforms are constantly evolving, deploying sophisticated anti-bot measures—your job is to stay ahead of them. If you thrive on solving complex technical challenges and enjoy working with real-world...

  • DevOps Engineer

    1 day ago

    Bharatpur, India Recro Full time

    We’re looking for high-impact DevOps Engineers and Backend Software Engineers to join an exciting, fast-moving project. If you’re someone who loves building secure, scalable, cloud-native platforms—and can own systems end-to-end—this is for you.🔧 What We’re Looking ForDevOps EngineersHands-on experts with:Kubernetes (local &...

  • Chief Technology Officer

    3 weeks ago

    Bharatpur, India Whatjobs IN C2 Full time

    About the Role We are seeking an experienced Technology Director / Head of Engineering to lead our Data Engineering and Tech team. This role requires a blend of hands-on technical depth, architectural leadership, and strategic vision . You will play a key role in scaling our technology, building robust infrastructure, and shaping the future of our platform....

  • Devops Engineer

    9 hours ago

    Bharatpur, India Whatjobs IN C2 Full time

    Job Title: DevOps Cloud Engineer Location: Noida/Onsite Experience: 2-3 years We're seeking a DevOps Cloud Engineer with Python programming and some basic frontend or backend experience for our AI product-based client. If you're skilled in AWS, DevOps tools, and Python, and want to grow in a fast-paced AI company, this role is for you. As the DevOps Cloud...