Lead Data Engineer

3 months ago

Bengaluru, India KOGNIVERA IT SOLUATION Full time

Functional Title : Lead GCP Engineer

Location : MG Road

Start Date : ASAP

Work mode : 2 days/week in office

Job Description :

About the Role: We are seeking a Lead GCP Data Engineer who can demonstrate a broad range of technical skills across Google Cloud Platform (GCP) and third-party technologies. As a pivotal member of our team, you will be responsible for designing robust data architectures, developing efficient data pipelines, and optimizing data processing workflows on GCP.


Your expertise will be instrumental in driving our data engineering initiatives forward, ensuring scalability, reliability, and performance.

Key Responsibilities :

1. Data Architecture Design :


- Design scalable and efficient data architectures on GCP that meet the organization's data processing and analysis requirements.


- Collaborate closely with data scientists, business analysts, and stakeholders to define effective data models and structures.

2. Data Pipeline Development :


- Develop and implement data pipelines using GCP services such as Google Cloud Storage, BigQuery, Dataflow, and Pub/Sub.


- Ensure data quality, reliability, and governance throughout the data lifecycle.

3. Data Transformation and Integration :


- Utilize technologies like Apache Beam, Apache Spark, and Cloud Dataproc to transform and integrate data from diverse sources.


- Perform data cleansing, aggregation, enrichment, and normalization to support downstream applications and analytics.

4. Performance Optimization :


- Optimize data processing workflows to enhance performance and efficiency.


- Monitor pipelines, identify bottlenecks, and implement optimizations such as improved data partitioning, sharding, and leveraging GCP's autoscaling capabilities.

5. Continuous Improvement :


- Stay abreast of advancements in data engineering and cloud technologies.


- Explore and implement new GCP features and services to enhance data processing capabilities and drive innovation within the team.

6. Research and Innovation :


- Conduct research on emerging data engineering technologies, tools, and best practices.


- Evaluate new methodologies to improve data engineering processes and bring innovative solutions to the organization.

7. Task Automation :


- Automate data engineering tasks using scripting, workflows, or tools like Cloud Composer and Cloud Functions.


- Streamline data ingestion, transformation, monitoring, and other operational processes to improve efficiency and reduce manual effort.

Attributes & Competencies :

Education : BE/BTech, MTech, or MCA.

Experience :


- Minimum 7+ years in development/migration projects, with at least 3 years focused on GCP.


- Experience in GCP-based Big Data deployments (batch/real-time) using BigQuery, BigTable, Google Cloud Storage, Pub/Sub, Data Fusion, Dataflow, Dataproc, and Airflow.

Technical Skills :

- Google Certified Professional Cloud Architect with experience automating and orchestrating workloads on GCP or other public clouds.

- Proficiency in at least one configuration management system (Chef, Puppet, Ansible, Salt, etc.).

- Strong programming skills in Python, Go, with expertise in Git and Git workflows.

- Demonstrated proficiency in CI/CD tools such as Jenkins, TeamCity, or Spinnaker

(ref:hirist.tech)

  • Lead Engineer

    2 months ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 275332 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Engineer to join our team in Bengaluru/Remote, Karnātaka (IN-KA), India (IN).Ability to...

  • Sr data engineer

    2 months ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 286326 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Sr data engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Senior data...

  • Data Engineer

    1 month ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 287747 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Mandatory Skills and...

  • Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Mandatory Skills and...

  • Data Engineer

    1 month ago

    Bengaluru, India NTT DATA Services Full time

    About NTT DATANTT DATA is a $30+ billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success. As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem of established and...

  • Data Engineer

    4 weeks ago

    Bengaluru, India NTT DATA Services Full time

    About NTT DATA NTT DATA is a $30+ billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success. As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem of...

  • Data Specialist

    24 hours ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 287764 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Specialist / Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Data EngineerDesign...

  • Data Engineering Lead

    7 days ago

    Bengaluru, India ofi Full time

    Data Engineering LeadAt ofi , we are on a mission to leverage data to drive innovation and business transformation. We are looking for a Data Engineering Lead to spearhead our data engineering team and contribute to our mission of delivering data-driven solutions. As a Data Engineering Lead, you will be responsible for managing the data engineering team and...

  • Digital Engineering Lead Engineer

    2 months ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 274724 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Lead Engineer to join our team in Bengaluru, Karnātaka (IN-KA), India...

  • Sr data engineer

    1 month ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 286326 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Sr data engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Senior data engineer  Strong...

  • Lead Engineer

    2 months ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 285317 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN)."Minimum 5+...

  • Digital Engineering Lead Engineer

    2 months ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 288920 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Lead Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Once You Are...

  • Data Engineering Lead

    2 weeks ago

    Bengaluru, India ofi Full time

    Data Engineering LeadAt ofi, we are on a mission to leverage data to drive innovation and business transformation. We are looking for a Data Engineering Lead to spearhead our data engineering team and contribute to our mission of delivering data-driven solutions. As a Data Engineering Lead, you will be responsible for managing the data engineering team and...

  • Data Engineering Lead

    7 days ago

    Bengaluru, India ofi Full time

    Data Engineering Lead At ofi , we are on a mission to leverage data to drive innovation and business transformation. We are looking for a Data Engineering Lead to spearhead our data engineering team and contribute to our mission of delivering data-driven solutions. As a Data Engineering Lead, you will be responsible for managing the data engineering team...

  • Data Engineer

    2 months ago

    Bengaluru, India NTT DATA Full time

    Job Description Data Engineer Grade 07ResponsibilitiesAssure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements using Azure data factory and Azure databricksDevelop and maintain innovative Azure Azure data factory and Azure databricksSolution design using Microsoft Azure...

  • Digital Engineering Lead Engineer

    2 months ago

    Bengaluru, India NTT DATA Full time

    Job Description NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Lead Engineer to join our team in bangalore, Karnātaka (IN-KA), India (IN).Java-5+ yrs,...

  • Data Engineering Lead

    7 days ago

    Bengaluru, India ofi Full time

    Data Engineering LeadAtofi , we are on a mission to leverage data to drive innovation and business transformation. We are looking for a Data Engineering Lead to spearhead our data engineering team and contribute to our mission of delivering data-driven solutions. As a Data Engineering Lead, you will be responsible for managing the data engineering team and...

  • Data Engineering Lead

    7 days ago

    Bengaluru, India ofi Full time

    Atofi , we are on a mission to leverage data to drive innovation and business transformation. We are looking for a Data Engineering Lead to spearhead our data engineering team and contribute to our mission of delivering data-driven solutions. As a Data Engineering Lead, you will be responsible for managing the data engineering team and overseeing the design...

  • Data Engineering Lead

    3 weeks ago

    Bengaluru, India ofi Full time

    At ofi, we are on a mission to leverage data to drive innovation and business transformation. We are looking for a Data Engineering Lead to spearhead our data engineering team and contribute to our mission of delivering data-driven solutions. As a Data Engineering Lead, you will be responsible for managing the data engineering team and overseeing the design...

  • Data Engineering Lead

    3 weeks ago

    Bengaluru, India ofi Full time

    At ofi, we are on a mission to leverage data to drive innovation and business transformation. We are looking for a Data Engineering Lead to spearhead our data engineering team and contribute to our mission of delivering data-driven solutions. As a Data Engineering Lead, you will be responsible for managing the data engineering team and overseeing the design...