Gcp Data Engineer

Job Summary: We are seeking a Senior Data Engineer to join a Global Data Analytics & Insights team focused on transforming enterprise data management and democratizing access to insights. This role will lead a large-scale data migration from external agency-managed environments into an internal Google Cloud Platform (GCP) ecosystem. You will drive discovery, define and execute migration strategy, rebuild ingestion pipelines, and ensure secure, compliant, always-on access for internal teams and approved external partners. The work directly supports global analytics initiatives by delivering scalable, high-quality, governed data products Roles and Responsibility Lead discovery and deep analysis of existing third-party data warehouses (e.G., Snowflake or hybrid setups), including data assets, lineage, dependencies, and integration patterns. Define and execute end-to-end migration strategy to move data assets and workflows into internal GCP environments. Handle and migrate third-party API-based ingestion flows so inbound data lands in GCP data lakes/warehouses. Partner with product lines, business stakeholders, and downstream consumers (dashboards, CRM, analytics) to capture requirements and ensure smooth data consumption. Design, build, and maintain scalable ingestion and transformation pipelines using Python, SQL, PySpark, and DBT/Dataform. Develop and manage GCP-native data solutions using BigQuery, Dataflow, Pub/Sub, Cloud Functions, Cloud Run, Dataproc, etc. Implement strong data governance, security, access controls, and compliance practices using GCP security capabilities. Integrate DevSecOps quality and security checks (e.G., SonarQube, FOSSA) into CI/CD pipelines, and respond to findings. Orchestrate workflows with Apache Airflow/Astronomer and provision infrastructure via Terraform (IaC best practices). Monitor and optimize performance, scalability, reliability, and cost-efficiency of pipelines and storage. Promote engineering best practices, reusable patterns, automation, and continuous improvement across teams. Produce clear documentation and communicate technical decisions effectively to technical and non-technical audiences. Required Skills: Expert proficiency in Python (NumPy, Pandas), SQL, and PySpark. Strong production-grade GCP experience with BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Functions, Cloud Run, Data Fusion/Data Prep. Proven ability to build scalable ELT/ETL ingestion and curation pipelines. Hands-on experience with DBT and/or Dataform for transformations. Workflow orchestration expertise with Apache Airflow and/or Astronomer. Proven experience integrating and managing third-party APIs for data ingestion. CI/CD and DevOps exposure (e.G., Tekton) and strong GitHub-based version control. Infrastructure as Code with Terraform. Solid knowledge of data governance, encryption, masking, access management, and cloud security best practices. Strong understanding of modern data ecosystems: data warehouses/lakes, metadata, meshes/fabrics, and analytics/AI use cases. Experience with Agile delivery, user stories, and cross-functional collaboration. Willingness to work from office (Chennai location). Preferred Skills: Direct experience with Snowflake migration, exploration, and optimization. Experience with Java. Familiarity with Master Data Management (MDM) concepts/tools. Exposure to additional security/performance tools (e.G., Checkmarx, Dynatrace). Working knowledge of GDPR and data privacy impact on architecture. Required Experience: 3-7 years in Data Engineering or Software Engineering with data-intensive systems. 2+ years building and deploying production-scale cloud data platforms on GCP. Demonstrated leadership in delivering migration or large data engineering programs. Strong track record of optimising compute/storage cost and performance in cloud environments. Education: Bachelor’s degree in Computer Science, Information Technology, Data Analytics, or related field (or equivalent practical experience).