See more Collapse

GCP Data Engineer

2 months ago

Bengaluru, India Coforge Full time

Job Description

Position: GCP Data Engineer

Location: Bangalore

Duration: Full Time

MUST HAVE SKILLS:

1: Scala

2: Python

3: GCP Cloud (Big Query, Big Table, Google Cloud Storage)

4: ETL Pipelines and Data Pipelines

RESPONSIBILITIES:

Build data pipelines on cloud GCP Environment. Implement tasks for Automatic job Schedular for task/workflow orchestration. Extract data from different data source systems and ingest data into GCS - Hive Tables on GCP Environment. Build data Pipelines to read data from GCS/Hive tables ingest data into big query Tables (fact and Dimension). Develop efficient ETL/ELT pipelines using ETL standards using Bigdata eco system/Data warehouse.

EXPERIENCE REQUIRED:

5+ years of data pipeline ETL/ELT development experience required. 3 years of GCP experience. Experience working in GCP based Big Data deployments (Batch/Real-Time) leveraging Big Query, Big Table, Google Cloud Storage, Data Proc, Pub Sub, Data Fusion, Dataflow, Dataproc, Automic/Airflow. 4 + years expert coding skills in Scala/Python using developer tools like IntelliJ or PyCharm as mandate. Work with data team to analyze data, build models, and integrate massive datasets from multiple data sources for data modeling. Extracting, Loading, Transforming, cleaning, and validating data + Designing pipelines and architectures for data processing. Implementation of Basic Data quality rules during ETL process and adopt the best industry optimized coding standards for next generation data and analytics platforms on GCP cloud. Experience in working with Jira Agile and Lean methodologies. Experience working with either a Map Reduce or an MPP system on any size/scale - Spark Ecosystem Having good knowledge on Retail domain would add advantage (plus) Posted On

Position: GCP Data Engineer

Location: Bangalore

Duration: Full Time

 

MUST HAVE SKILLS:

1: Scala

2: Python

3: GCP Cloud (Big Query, Big Table, Google Cloud Storage)

4: ETL Pipelines and Data Pipelines

RESPONSIBILITIES:

  • Build data pipelines on cloud GCP Environment.
  • Implement tasks for Automatic job Schedular for task/workflow orchestration.
  • Extract data from different data source systems and ingest data into GCS - Hive Tables on GCP Environment.
  • Build data Pipelines to read data from GCS/Hive tables ingest data into big query Tables (fact and Dimension).
  • Develop efficient ETL/ELT pipelines using ETL standards using Bigdata eco system/Data warehouse.

 

EXPERIENCE REQUIRED:

  • 5+ years of data pipeline ETL/ELT development experience required.
  • 3 years of GCP experience.
  • Experience working in GCP based Big Data deployments (Batch/Real-Time) leveraging Big Query, Big Table, Google Cloud Storage, Data Proc, Pub Sub, Data Fusion, Dataflow, Dataproc, Automic/Airflow.
  • 4 + years expert coding skills in Scala/Python using developer tools like IntelliJ or PyCharm as mandate.
  • Work with data team to analyze data, build models, and integrate massive datasets from multiple data sources for data modeling.
  • Extracting, Loading, Transforming, cleaning, and validating data + Designing pipelines and architectures for data processing.
  • Implementation of Basic Data quality rules during ETL process and adopt the best industry optimized coding standards for next generation data and analytics platforms on GCP cloud.
  • Experience in working with Jira Agile and Lean methodologies.
  • Experience working with either a Map Reduce or an MPP system on any size/scale - Spark Ecosystem
  • Having good knowledge on Retail domain would add advantage (plus)
Skills Required

SCALA AND SPARK

Posted On SCALA AND SPARK Location

Bangalore

Posted On Bangalore Desirable Skills

HIVE

Posted On HIVE

We have other current jobs related to this field that you can find below

  • Gcp Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Tech Mahindra Full time

    We are seeking a highly skilled GCP Data Engineer to join our team.The ideal candidate will have extensive experience with Google Cloud Platform (GCP) services, particularly Dataflow, Big Query, Pub/Sub, and related services.Additionally, the candidate should be proficient in provisioning infrastructure on GCP using Infrastructure as Code (Ia C) with...

  • Gcp Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Impetus Full time

    Location: Bangalore/Gurgaon4-7 years of IT experience range is preferred.Able to effectively use GCP managed services e.G. Dataproc, Dataflow, pub/sub, Cloud functions, Big Query, GCS At least 4 of these Services.Good to have knowledge on Cloud Composer, Cloud SQL, Big Table, Cloud Function.Strong experience in Big Data technologies – Hadoop, Sqoop, Hive...

  • GCP Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world's biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We're harnessing the power of technology and humanity to create...

  • GCP Data Engineer

    4 weeks ago

    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • GCP Data Engineer

    4 weeks ago

    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • GCP Data Engineer

    3 days ago

    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • GCP Data Engineer

    4 weeks ago

    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • GCP Data Engineer

    4 weeks ago

    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • GCP Data Engineer

    5 days ago

    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • GCP Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Tech Mahindra Full time

    Tech Mahindra hiring GCP Data Engineers for all across IndiaNotice- Immediate/ Early joiners only**Experience- 4+ yearsMust have skills:Linux, SQL, GCP BigQuery, CloudRun, Cloud Functions, GC Storage, PythonSend CVs to if eligible.

  • GCP Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Tech Mahindra Full time

    Tech Mahindra hiring GCP Data Engineers for all across India Notice- Immediate/ Early joiners only** Experience- 4+ years Must have skills: Linux, SQL, GCP BigQuery, CloudRun, Cloud Functions, GC Storage, Python Send CVs to if eligible.

  • GCP Data Engineer

    2 weeks ago

    Bengaluru, India Tech Mahindra Full time

    Tech Mahindra hiring GCP Data Engineers for all across India Notice- Immediate/ Early joiners only** Experience- 4+ years Must have skills: Linux, SQL, GCP BigQuery, CloudRun, Cloud Functions, GC Storage, Python Send CVs to if eligible.

  • GCP Data Engineer

    1 month ago

    Bengaluru, India Tech Mahindra Full time

    Tech Mahindra hiring GCP Data Engineers for all across IndiaNotice- Immediate/ Early joiners only**Experience- 4+ yearsMust have skills:Linux, SQL, GCP BigQuery, CloudRun, Cloud Functions, GC Storage, PythonSend CVs to KO00759472@TechMahindra.com if eligible.

  • GCP Data Engineer

    3 weeks ago

    Bengaluru, India Tech Mahindra Full time

    Tech Mahindra hiring GCP Data Engineers for all across IndiaNotice- Immediate/ Early joiners only**Experience- 4+ yearsMust have skills:Linux, SQL, GCP BigQuery, CloudRun, Cloud Functions, GC Storage, PythonSend CVs to KO00759472@TechMahindra.com if eligible.

  • GCP Data Engineer

    1 month ago

    Bengaluru, India Tech Mahindra Full time

    Tech Mahindra hiring GCP Data Engineers for all across IndiaNotice- Immediate/ Early joiners only**Experience- 4+ yearsMust have skills:Linux, SQL, GCP BigQuery, CloudRun, Cloud Functions, GC Storage, PythonSend CVs to KO00759472@TechMahindra.com if eligible.

  • GCP Data Engineer

    2 weeks ago

    Bengaluru, India Tech Mahindra Full time

    Tech Mahindra hiring GCP Data Engineers for all across IndiaNotice- Immediate/ Early joiners only**Experience- 4+ yearsMust have skills:Linux, SQL, GCP BigQuery, CloudRun, Cloud Functions, GC Storage, PythonSend CVs to KO00759472@TechMahindra.com if eligible.

  • Lead GCP Data Engineer

    1 month ago

    Bengaluru, India Recro Full time

    Job Title: Lead GCP Data EngineerLocation: Bengaluru, Karnataka, IndiaExperience: 7-10 yearsResponsibilities:Cloud Analytics Platform:Build and optimize a cloud analytics platform on Google Cloud Platform (GCP) to meet evolving business needs using lean Agile practices.Design and implement solutions for transformation and modernization on GCP.Data...

  • Lead GCP Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Recro Full time

    Job Title: Lead GCP Data Engineer Location: Bengaluru, Karnataka, India Experience: 7-10 years Responsibilities: Cloud Analytics Platform : Build and optimize a cloud analytics platform on Google Cloud Platform (GCP) to meet evolving business needs using lean Agile practices. Design and implement solutions for transformation and modernization on GCP. ...

  • GCP Data Engineer

    2 months ago

    Bengaluru, India CIEL HR Services Full time

    Hi,Total years of experience: 5+Description:Job Description:GCP Data Engineer Engineers :Prepare DAGPrepare Airflow based ETL JobsDocument architecture & technical designAnalyze and implement Securities Services use caseImplement required Securities for product data acquisition

  • Lead GCP Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Recro Full time

    Job Title: Lead GCP Data EngineerLocation: Bengaluru, Karnataka, IndiaExperience: 7-10 yearsResponsibilities:Cloud Analytics Platform :Build and optimize a cloud analytics platform on Google Cloud Platform (GCP) to meet evolving business needs using lean Agile practices.Design and implement solutions for transformation and modernization on GCP.Data...