GCP Data Lead

4 weeks ago

Bengaluru, India Cygnus Professionals Inc. Full time

Role: Google Cloud Platform (GCP) Data Lead

Location: Bangalore, India (Day 1 Onsite)

Job Type: Contract


Job Description:


  • Strong GCP experience
  • Prior experience in building pipelines using dataflow, data fusion, data proc, etc
  • Very strong python coding skill
  • Experience in optimising GCP resources for cost and performance
  • Experience in containerisation of applications
  • CI/CD using GIThub actions
  • Exposure to DataPlex, Data Catalog, and metadata management
  • GCP IAM and role management
  • Design and implement end-to-end data architectures leveraging GCP services (e.g., Big Query, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer) for large-scale data ingestion and processing.
  • Build and optimize large-scale data pipelines using Apache Spark on GCP (via Dataproc or other Spark services). Ensure high performance and scalability in Spark-based data processing workloads.
  • Develop and manage scalable data ingestion pipelines for structured and unstructured data using tools like Cloud Dataflow, Cloud Pub/Sub, and Apache Spark.
  • Provide guidance for designing secure, scalable, and efficient data solutions on the Google Cloud Platform, integrating with on-premise/cloud systems like SAP S/4HANA.
  • Implement both real-time streaming and batch processing pipelines using Apache Spark, Dataflow, and other GCP services to meet business requirements.
  • Implement data governance, access controls, and security best practices to ensure the integrity, confidentiality, and compliance of data across systems.
  • Collaborate with business stakeholders, data scientists, and engineering teams to define data requirements, ensuring the architecture aligns with business goals.
  • Optimize Apache Spark jobs for performance, scalability, and cost-efficiency, ensuring that the architecture can handle growing data volumes.
  • Provide technical leadership to the data engineering team, mentoring junior engineers in data architecture, Apache Spark development, and GCP best practices.


Technical Expertise:


  • Expert-level programming proficiency in Python, Java, and Scala.
  • Extensive hands-on experience with big data technologies, including Spark, Hadoop, Hive, Yarn, MapReduce, Pig, Kafka, and PySpark.
  • Proficient in Google Cloud Platform services, such as BigQuery, Dataflow, Cloud Storage, Dataproc, and Cloud Composer Google Pub/Sub, and Google Cloud Functions.Expertise in Apache Spark for both batch and real-time processing, as well as proficiency in Apache Beam, Hadoop, or other big data frameworks. Experienced in using Cloud SQL, BigQuery, and Looker Studio (Google Data Studio) for cloud-based data solutions.
  • Skilled in orchestration and deployment tools like Cloud Composer, Airflow, and Jenkins for continuous integration and deployment (CI/CD).
  • Expertise in designing and developing integration solutions involving Hadoop/HDFS, real-time systems, data warehouses, and analytics solutions.
  • Experience with DevOps practices, including version control (Git), CI/CD pipelines, and infrastructure-as-code (e.g., Terraform, Cloud Deployment Manager). Strong background in working with relational databases, NoSQL databases, and in-memory databases.
  • Experience managing large datasets within Data Lake and Data Fabric architectures.
  • Strong knowledge of security best practices, IAM, encryption mechanisms, and compliance frameworks (GDPR, HIPAA) within GCP environments.
  • Experience in implementing data governance, data lineage, and data quality frameworks.
  • In-depth knowledge of web technologies, application programming languages, OLTP/OLAP technologies, data strategy disciplines, relational databases, data warehouse development, and big data solutions.
  • Led end-to-end processes for the design, development, deployment, and maintenance of data engineering projects.
  • Excellent debugging and problem-solving skills.
  • Retail and e-commerce domain knowledge is a plus.
  • Positive attitude with strong analytical skills and the ability to guide teams effectively.


Preferred Qualifications:


  • GCP Certifications: Such as Professional Data Engineer / Apache Spark and Python certifications.
  • Experience with Data visualization tools like Tableau, Power BI etc.


Nice to have (but not mandatory):


  • Experience in integration with S4 Hana
  • Experience with integration with Salesforce

  • GCP Lead Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a skilled GCP Lead Engineer to join our team at NTT DATA. This role is ideal for someone with expertise in designing and implementing Google Cloud Platform (GCP) solutions.Key ResponsibilitiesAssess, design, and implement GCP solutions, including compute, network, storage, identity, security, and disaster recovery.Develop and...

  • GCP Lead Engineer

    1 month ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 298250 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a GCP Lead Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Mandatory...

  • GCP Lead Engineer

    2 months ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 298250 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a GCP Lead Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Mandatory Skills:Technical...

  • Gcp Data Science

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    **Statistical Modeling**: Strong understanding of statistical techniques and experience leveraging BigQuery ML and Vertex AI for model development. **Machine Learning Algorithms**: Experience with GCP's pre-built ML APIs (e.g., Vision AI, Natural Language API) and custom model building using Vertex AI. **Feature Engineering**: Ability to create meaningful...

  • Gcp data lead

    4 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • GCP Data Lead

    4 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • GCP Data Lead

    4 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • GCP Data Lead

    4 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • Lead engineer

    2 weeks ago

    Bengaluru, India Live Connections Full time

    Lead Engineer - GCP Data ProtectionExperience: 12 to 16 YearsLocation: Bangalore / HyderabadNotice Period: Maximum 30 DaysOrganization: One of the leading GCCsCTC: 60 LPA or 30% on current CTC, whichever is lower.We are looking for a Lead Engineer - GCP Data Protection to spearhead cutting-edge cloud transformation initiatives. This role is ideal for...

  • GCP Lead Engineer

    5 months ago

    Bengaluru, India Kognivera IT Solution Full time

    Functional Title: Lead GCP EngineerLocation: MG RoadStart Date: ASAPWork mode: 2 days/week in officeJob Description:About the Role: We are seeking a Lead GCP Data Engineer who can demonstrate a broad range of technical skills across Google Cloud Platform (GCP) and third-party technologies. As a pivotal member of our team, you will be responsible for...

  • Gcp data architect

    3 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    GCP Data Architect, BQ, Pub/Sub, GKE, Terraform, Python, Cloud Dev OpsExp-5 to 10Location-Pan India• Main SkillExperience in cloud architecture or systems engineering• Experience with identity management and networking segmentation across multiple VPC networks and projects• Experience leading large-scale network migrations to GCP• Experience...

  • GCP Data Stream

    3 weeks ago

    Bengaluru, India Tekgence Inc Full time

    Mandatory Skills: Azure CI/CD pipeline, GCP Cloud, Terraform, Kubernetes, Dataflow, Big Query , GCP DataStream.Azure Devops (CI/CD pipelines), GCP Cloud, Terraform, Docker, Kubernetes, GitHub actions, scriptingADO, Jenkins, GitHubLocation- Pan India, hybrid mode 3 days in a weekEXP range- 7-12Job Role: Senior Cloud Infrastructure Engineer (GCP & Terraform...

  • GCP Data Stream

    3 weeks ago

    Bengaluru, India Tekgence Inc Full time

    Mandatory Skills: Azure CI/CD pipeline, GCP Cloud, Terraform, Kubernetes, Dataflow, Big Query, GCP DataStream.Azure Devops (CI/CD pipelines), GCP Cloud, Terraform, Docker, Kubernetes, GitHub actions, scripting ADO, Jenkins, GitHubLocation- Pan India, hybrid mode 3 days in a weekEXP range- 7-12 Job Role: Senior Cloud Infrastructure Engineer (GCP & Terraform)...

  • GCP Data Stream

    3 weeks ago

    Bengaluru, India Tekgence Inc Full time

    Mandatory Skills: Azure CI/CD pipeline, GCP Cloud, Terraform, Kubernetes, Dataflow, Big Query, GCP DataStream.Azure Devops (CI/CD pipelines), GCP Cloud, Terraform, Docker, Kubernetes, GitHub actions, scripting ADO, Jenkins, GitHubLocation- Pan India, hybrid mode 3 days in a weekEXP range- 7-12 Job Role: Senior Cloud Infrastructure Engineer (GCP & Terraform)...

  • GCP Data Stream

    3 weeks ago

    Bengaluru, India Tekgence Inc Full time

    Mandatory Skills: Azure CI/CD pipeline, GCP Cloud, Terraform, Kubernetes, Dataflow, Big Query , GCP DataStream. Azure Devops (CI/CD pipelines), GCP Cloud, Terraform, Docker, Kubernetes, GitHub actions, scripting ADO, Jenkins, GitHub Location- Pan India, hybrid mode 3 days in a week EXP range- 7-12 Job Role: Senior Cloud Infrastructure Engineer (GCP &...

  • GCP Data Architect

    3 months ago

    Bengaluru, India Tata Consultancy Services Full time

    GCP Data Architect, BQ, Pub/Sub, GKE, Terraform, Python, Cloud DevOpsExp-5 to 10Location-Pan India• Main SkillExperience in cloud architecture or systems engineering• Experience with identity management and networking segmentation across multiple VPC networks and projects• Experience leading large-scale network migrations to GCP• Experience...

  • GCP Data Architect

    3 months ago

    Bengaluru, India Tata Consultancy Services Full time

    GCP Data Architect, BQ, Pub/Sub, GKE, Terraform, Python, Cloud DevOpsExp-5 to 10Location-Pan India• Main SkillExperience in cloud architecture or systems engineering• Experience with identity management and networking segmentation across multiple VPC networks and projects• Experience leading large-scale network migrations to GCP• Experience...

  • Gcp data engineer

    5 days ago

    Bengaluru, India Apptad Inc. Full time

    Job Title: GCP Data Engineer - Teradata, ETLJob Location: BangaloreJob Type: PermanentKey Responsibilities:Data Engineering on Google Cloud Platform (GCP):Design, develop, and maintain data pipelines using GCP services such as Big Query, Cloud Storage, Cloud Dataflow, and Pub/Sub.Implement and optimize ETL processes to ensure efficient data transformation...

  • ▷ (Only 24h Left) GCP Data Lead

    3 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:- Strong GCP experience- Prior experience in building pipelines using dataflow, data fusion, data proc, etc- Very strong python coding skill- Experience in optimising GCP resources for cost and performance- Experience in containerisation of...

  • GCP Data Engineer

    3 weeks ago

    Bengaluru / Bangalore, Kolkata, India Zenfinet Solutions Opc Private Limited Full time

    Please find the below JD:Job Title: Data EngineerJob Type: Permanent RoleLocation:Bangalore/Hyderabad/Kolkata/Pune/Noida Inviting applications for the role of Principal / Senior Principal Consultant Google Cloud Platform (GCP) Data Engineer Experience Required - 8 yrs to 12 yrs. In this role, As a GCP Data Engineer a, you will be a key player in...