Lead GCP Data Engineer

2 months ago

Bangalore Urban, India Recro Full time

Job Title: Lead GCP Data Engineer

Location: Bengaluru, Karnataka, India

Experience: 7-10 years


Responsibilities:


Cloud Analytics Platform:

  • Build and optimize a cloud analytics platform on Google Cloud Platform (GCP) to meet evolving business needs using lean Agile practices.
  • Design and implement solutions for transformation and modernization on GCP.

Data Engineering:

  • Analyze and manipulate large datasets to support enterprise data needs.
  • Design and develop solutions for landing data from source applications to GCP.
  • Build and operationalize data warehouses, data lakes, and analytics platforms on GCP.

Solution Architecture:

  • Design solutions with appropriate combinations of GCP and third-party technologies.
  • Deep dive into complex performance issues to improve efficiency and reduce customer incidents.
  • Provide automation and orchestration expertise for workloads on GCP or other public clouds.

Automation:

  • Develop and maintain scripts, workflows, and automation using tools like Cloud Composer or Cloud Functions.
  • Implement configuration management systems such as Chef, Puppet, Ansible, or Salt.

Software Development:

  • Write code in Python or Go for data engineering tasks.
  • Implement CI/CD pipelines using tools such as Jenkins, TeamCity, or Spinnaker.

Continuous Learning and Research:

  • Stay updated with the latest advancements in data engineering and cloud technologies.
  • Explore new features and services offered by GCP to improve data engineering processes.
  • Conduct research, attend conferences, and engage with the data engineering community.

Collaboration and Communication:

  • Work effectively with global teams and stakeholders.
  • Communicate complex technical concepts to non-technical stakeholders.

Automating Tasks:

  • Develop solutions to automate repetitive or time-consuming data processes.
  • Improve efficiency and productivity by streamlining data workflows.


Required Skills and Experience:


Work Experience:

  • Minimum 7+ years of experience in development or migration projects.
  • 3+ years of experience with GCP.

GCP Data Engineering:

  • Experience with GCP-based Big Data deployments (Batch/Real-Time).
  • Proficiency with GCP data services like BigQuery, Bigtable, Google Cloud Storage, Pub/Sub, Data Fusion, Dataflow, Dataproc, and Airflow.
  • Experience building data lake solutions leveraging GCP data products (e.g., DataProc, AI Building Blocks, Looker, Cloud Data Fusion, Dataprep, etc.), Hive, and Spark.

Programming:

  • Strong coding skills in Python, PySpark, or Scala.
  • Experience with source control tools like GitHub and related development processes.

Workflow Management:

  • Experience with workflow scheduling tools like Airflow.

Data Engineering Knowledge:

  • Strong understanding of data structures and algorithms.
  • Experience building relational SQL and NoSQL solutions.

Soft Skills:

  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration skills.
  • Ability to work independently and as part of a global team.
  • Passionate about data solutions.
  • Self-motivated and able to work in a fast-paced environment.
  • Detail-oriented and committed to delivering high-quality work.

Certifications:

  • Google Certified Professional Cloud Architect is preferred.


Preferred Skills:

  1. Databricks:
  • Experience using Databricks, including its Data Engineering and Delta Lake components.

  • Lead GCP Data Engineer

    2 months ago

    Bangalore Urban, India Recro Full time

    Job Title: Lead GCP Data Engineer Location: Bengaluru, Karnataka, India Experience: 7-10 years Responsibilities: Cloud Analytics Platform : Build and optimize a cloud analytics platform on Google Cloud Platform (GCP) to meet evolving business needs using lean Agile practices. Design and implement solutions for transformation and modernization on GCP. ...

  • GCP Data Engineer

    1 week ago

    Bangalore Urban, India Tech Mahindra Full time

    We are seeking a highly skilled GCP Data Engineer to join our team. The ideal candidate will have extensive experience with Google Cloud Platform (GCP) services, particularly Dataflow, BigQuery, Pub/Sub, and related services. Additionally, the candidate should be proficient in provisioning infrastructure on GCP using Infrastructure as Code (IaC) with...

  • Lead GCP Data Engineer

    2 months ago

    bangalore, India Recro Full time

    Job Title: Lead GCP Data EngineerLocation: Bengaluru, Karnataka, IndiaExperience: 7-10 yearsResponsibilities:Cloud Analytics Platform :Build and optimize a cloud analytics platform on Google Cloud Platform (GCP) to meet evolving business needs using lean Agile practices.Design and implement solutions for transformation and modernization on GCP.Data...

  • GCP Data Engineer

    4 weeks ago

    Bangalore Urban, India Impetus Full time

    Immediate Joiners Preferred 3-7 years of IT experience range is preferred.Strong experience in Java programming – Java 8+, SpringBoot, Microservices, REST, Webservices, Hybernate/JPAGood to have knowledge on - Kubernetes basicsAble to effectively use GCP managed services e.g. VMs, Dataproc, Dataflow, pub/sub, Cloud functions, Big Query, GCS - At least 4 of...

  • GCP Data Engineer

    3 weeks ago

    Bangalore Urban, India Impetus Full time

    Immediate Joiners Preferred 3-7 years of IT experience range is preferred. Strong experience in Java programming – Java 8+, SpringBoot, Microservices, REST, Webservices, Hybernate/JPA Good to have knowledge on - Kubernetes basics Able to effectively use GCP managed services e.g. VMs, Dataproc, Dataflow, pub/sub, Cloud functions, Big Query, GCS - At...

  • GCP Data Engineer

    3 weeks ago

    Bangalore Urban, India Impetus Full time

    Immediate Joiners Preferred 3-7 years of IT experience range is preferred.Strong experience in Java programming – Java 8+, SpringBoot, Microservices, REST, Webservices, Hybernate/JPAGood to have knowledge on - Kubernetes basicsAble to effectively use GCP managed services e.g. VMs, Dataproc, Dataflow, pub/sub, Cloud functions, Big Query, GCS - At least 4 of...

  • GCP Data Engineer

    1 week ago

    bangalore, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • GCP Data Engineer

    1 week ago

    bangalore, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • GCP Data Engineer

    3 weeks ago

    bangalore, India Tech Mahindra Full time

    Tech Mahindra hiring GCP Data Engineers for all across IndiaNotice- Immediate/ Early joiners only**Experience- 4+ yearsMust have skills:Linux, SQL, GCP BigQuery, CloudRun, Cloud Functions, GC Storage, PythonSend CVs to if eligible.

  • GCP Data Engineer

    1 week ago

    bangalore, India Tech Mahindra Full time

    We are seeking a highly skilled GCP Data Engineer to join our team. The ideal candidate will have extensive experience with Google Cloud Platform (GCP) services, particularly Dataflow, BigQuery, Pub/Sub, and related services. Additionally, the candidate should be proficient in provisioning infrastructure on GCP using Infrastructure as Code (IaC) with...

  • GCP Data Engineer

    4 weeks ago

    bangalore, India Alp Consulting Limited Full time

    Requirement:- Mandatory to have knowledge of Big Data Architecture Patterns and experience in delivery of BigData and Hadoop Ecosystems. Strong experience required in GCP . Must have done multiple large projects with GCP Big Query and ETL Experience working in GCP based Big Data deployments (Batch/Realtime) leveraging components like GCP Big...

  • GCP Data Engineer

    3 weeks ago

    bangalore, India Alp Consulting Limited Full time

    We are seeking a skilled Data Engineer with a specialisation in building robust ETL (Extract, Transform, Load) processes and a strong background in utilising Google Cloud Platform (GCP) services. The ideal candidate should have experience designing, developing, and maintaining data pipelines that facilitate the collection, processing, and storage of large...

  • Data Engineering Manager

    1 week ago

    bangalore, India Publicis Sapient Full time

    As Manager, Data Engineering, you will be responsible for translating client requirements into design, architecting, and implementing GCP Cloud based big data solutions for clients. Your role will be focused on delivering high quality solutions by independently driving design discussions related to Data Ingestion, Transformation & Consumption, Data...

  • Senior Data Engineer

    1 week ago

    bangalore, India TecHHire Global Full time

    Job Description : We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP : Must be able to write quality code and build secure, highly available...

  • Senior Data Engineer

    2 months ago

    bangalore, India TecHHire Global Full time

    Job Description : We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP : Must be able to write quality code and build secure, highly available...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, India TecHHire Global Full time

    Job Description : We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP : Must be able to write quality code and build secure, highly available...

  • Senior Data Engineer

    2 months ago

    Bangalore, India TecHHire Global Full time

    Job Description : We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP : Must be able to write quality code and build secure, highly available...

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP :Must be able to write quality code and build secure, highly available...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP :Must be able to write quality code and build secure, highly available...

  • GCP Data Engineer

    3 weeks ago

    bangalore, India Impetus Full time

    Immediate Joiners Preferred3-7 years of IT experience range is preferred.Strong experience in Java programming – Java 8+, SpringBoot, Microservices, REST, Webservices, Hybernate/JPAGood to have knowledge on - Kubernetes basicsAble to effectively use GCP managed services e.g. VMs, Dataproc, Dataflow, pub/sub, Cloud functions, Big Query, GCS - At least 4 of...