Senior Data Engineer

3 weeks ago

Bengaluru, India TecHHire Global Full time

Job Description :


We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP :

  • Must be able to write quality code and build secure, highly available systems.
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc with the guidance.
  • Create data tools for analytics and data scientist team members that assist them in building and optimising our product into an innovative industry leader.
  • Monitoring performance and advising any necessary infrastructure changes.
  • Defining data retention policies.
  • Implementing the ETL process and optimal data pipeline architecture
  • Build analytic tools that utilise the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
  • Create design documents that describe the functionality, capacity, architecture, and process.
  • Develop, test, and implement data solutions based on finalised design documents.
  • Work with data and analytics experts to strive for greater functionality in our data systems.
  • Proactively identify potential production issues and recommend and implement solutions

Requirements :

  • Should have at least 2 years of relevant experience as GCP Data Engineer
  • Educational Qualifications - Bachelor's/Master's
  • Good understanding of optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and GCP big data' technologies
  • Strong in SQL and good understanding and experience in writing stored procedures
  • Good experience in GCP
  • Good experience in Hadoop, Big data
  • Proficient understanding of distributed computing principles
  • Experience in working with batch processing/ real-time systems using various open-source technologies like NoSQL, Spark, Pig, Hive, Apache Airflow.
  • Implemented complex projects dealing with the considerable data size (PB).
  • Optimisation techniques (performance, scalability, monitoring, etc.)
  • Experience with integration of data from multiple data sources preferably AWS
  • Experience with NoSQL databases, such as HBase, Cassandra, MongoDB, etc.
  • Knowledge of various ETL techniques and frameworks, such as Flume
  • Experience with various messaging systems, such as Kafka or RabbitMQ
  • Good understanding of Lambda Architecture, along with its advantages and drawbacks
  • Creation of DAGs for data engineering
  • Expert at Python /Scala programming, especially for data engineering/ ETL purposes
(ref:hirist.tech)

  • Senior Data Engineer

    4 days ago

    Bengaluru, India NTT DATA Full time

    Job Description Lead Data Engineering Project Team, Expertise in Databricks,Expertise in Data Management & Data Governance,Expertise in Azure Cloud and its services, Experienced in Data Modelling and Data Pipelines,Experienced in Healthcare projects, Experience implementing FHIR Data model and FHIR Server Apply from local Career site Listen to the story...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, India Giant Eagle GCC Full time

    Sr Data Engineer Data Analytics Grow. Learn. Thrive. Giant Eagle is where your career can soar high. At Giant Eagle, you are so much more than an employee. You are part of the Family. About the Company Since our founding in 1931, Giant Eagle, Inc. has evolved into one of the top 40 largest private corporations in the U.S. and one of the country’s...

  • Senior Data Engineer

    2 days ago

    Bengaluru, India Giant Eagle GCC Full time

    Senior Software Engineer - Data AnalyticsGrow. Learn. Thrive. Giant Eagle is where your career can soar high. At Giant Eagle, you are so much more than an employee. You are part of the Family.About the CompanySince our founding in 1931, Giant Eagle, Inc. has evolved into one of the top 40 largest private corporations in the U.S. and one of the country’s...

  • Senior Data Engineer

    2 days ago

    Bengaluru, India Giant Eagle GCC Full time

    Senior Software Engineer - Data AnalyticsGrow. Learn. Thrive. Giant Eagle is where your career can soar high. At Giant Eagle, you are so much more than an employee. You are part of the Family.About the CompanySince our founding in 1931, Giant Eagle, Inc. has evolved into one of the top 40 largest private corporations in the U.S. and one of the country’s...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, India Twenty20 Systems Full time

    We are seeking a talented and experienced Senior Data Engineer to join our dynamic team at Twenty20 Systems. As a Senior Data Engineer, you will play a pivotal role in designing, building, and maintaining data pipelines that empower our organization to make data-driven decisions. You will collaborate closely with cross-functional teams to gather...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, India Huron Full time

    The OpportunityHuron helps its clients drive growth, enhance performance and sustain leadership in the markets they serve. We help healthcare organizations build innovation capabilities and accelerate key growth initiatives, enabling organizations to own the future, instead of being disrupted by it. Together, we empower clients to create sustainable growth,...

  • Senior Data Engineer

    2 months ago

    Bengaluru, India Giant Eagle GCC Full time

    Sr Data Engineer Data AnalyticsGrow. Learn. Thrive. Giant Eagle is where your career can soar high. At Giant Eagle, you are so much more than an employee. You are part of the Family.About the CompanySince our founding in 1931, Giant Eagle, Inc. has evolved into one of the top 40 largest private corporations in the U.S. and one of the country’s largest food...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, India Mitchell Martin Inc. Full time

    JOB DESCRIPTION The client is building the Data Center of Excellence team in Bangalore, India. The client is looking to hire Senior Data Engineers. The Client Data and Commercial Technology Team is looking for Senior Data Engineers with a passion for building robust, scalable, efficient, and high-quality Data Engineering solutions to join our Engineering...

  • Senior Data Engineer

    2 months ago

    Bengaluru, India Giant Eagle GCC Full time

    Sr Data Engineer Data Analytics Grow. Learn. Thrive. Giant Eagle is where your career can soar high. At Giant Eagle, you are so much more than an employee. You are part of the Family. About the Company Since our founding in 1931, Giant Eagle, Inc. has evolved into one of the top 40 largest private corporations in the U.S. and one of the country’s...

  • Senior Data Engineer

    2 months ago

    Bengaluru, India Giant Eagle GCC Full time

    Sr Data Engineer Data AnalyticsGrow. Learn. Thrive. Giant Eagle is where your career can soar high. At Giant Eagle, you are so much more than an employee. You are part of the Family. About the CompanySince our founding in 1931, Giant Eagle, Inc. has evolved into one of the top 40 largest private corporations in the U.S. and one of the country’s largest...

  • Senior Data Engineer

    2 months ago

    Bengaluru, India Giant Eagle GCC Full time

    Sr Data Engineer Data AnalyticsGrow. Learn. Thrive. Giant Eagle is where your career can soar high. At Giant Eagle, you are so much more than an employee. You are part of the Family. About the CompanySince our founding in 1931, Giant Eagle, Inc. has evolved into one of the top 40 largest private corporations in the U.S. and one of the country’s largest...

  • Senior Data Engineer

    5 days ago

    Bengaluru, India New Relic, Inc. Full time

    Senior Data EngineerSenior Data EngineerSenior Data Engineer Req ID FY|S&M|#8 Location(s) Bangalore, India; Work arrangement(s) Hybrid (works from home and New Relic office throughout the week), Office (works primarily in a New Relic office) Your opportunity Do you love the transformative impact data can have on a business? Are you motivated to push for...

  • Senior Data Engineer

    2 months ago

    Bengaluru, India Mitchell Martin Inc. Full time

    JOB DESCRIPTION The client is building the Data Center of Excellence team in Bangalore, India. The client is looking to hire Senior Data Engineers. The Client Data and Commercial Technology Team is looking for Senior Data Engineers with a passion for building robust, scalable, efficient, and high-quality Data Engineering solutions to join our Engineering...

  • Senior Data Engineer

    2 months ago

    Bengaluru, India Mitchell Martin Inc. Full time

    JOB DESCRIPTIONThe client is building the Data Center of Excellence team in Bangalore, India. The client is looking to hire Senior Data Engineers.The Client Data and Commercial Technology Team is looking for Senior Data Engineers with a passion for building robust, scalable, efficient, and high-quality Data Engineering solutions to join our Engineering...

  • Senior Data Engineer

    2 months ago

    Bengaluru, India Mitchell Martin Inc. Full time

    JOB DESCRIPTIONThe client is building the Data Center of Excellence team in Bangalore, India. The client is looking to hire Senior Data Engineers.The Client Data and Commercial Technology Team is looking for Senior Data Engineers with a passion for building robust, scalable, efficient, and high-quality Data Engineering solutions to join our Engineering...

  • Senior Data Engineer

    6 days ago

    Bengaluru, India Twenty20 Systems Full time

    We are seeking a talented and experienced Senior Data Engineer to join our dynamic team at Twenty20 Systems. As a Senior Data Engineer, you will play a pivotal role in designing, building, and maintaining data pipelines that empower our organization to make data-driven decisions. You will collaborate closely with cross-functional teams to gather...

  • Senior Data Engineer

    7 days ago

    Bengaluru, India Twenty20 Systems Full time

    We are seeking a talented and experienced Senior Data Engineer to join our dynamic team at Twenty20 Systems. As a Senior Data Engineer, you will play a pivotal role in designing, building, and maintaining data pipelines that empower our organization to make data-driven decisions. You will collaborate closely with cross-functional teams to gather...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, India Twenty20 Systems Full time

    We are seeking a talented and experienced Senior Data Engineer to join our dynamic team at Twenty20 Systems. As a Senior Data Engineer, you will play a pivotal role in designing, building, and maintaining data pipelines that empower our organization to make data-driven decisions. You will collaborate closely with cross-functional teams to gather...

  • Senior Data Engineer

    5 days ago

    Bengaluru, India New Relic, Inc. Full time

    Senior Data Engineer Senior Data Engineer Senior Data Engineer Req ID FY|S&M|#8 Location(s) Bangalore, India; Work arrangement(s) Hybrid (works from home and New Relic office throughout the week), Office (works primarily in a New Relic office) Your opportunity Do you love the transformative impact data can have on a business? Are you motivated to push for...

  • Senior Data Engineer

    2 months ago

    Bengaluru, India Mitchell Martin Inc. Full time

    JOB DESCRIPTIONThe client is building the Data Center of Excellence team in Bangalore, India. The client is looking to hire Senior Data Engineers.The Client Data and Commercial Technology Team is looking for Senior Data Engineers with a passion for building robust, scalable, efficient, and high-quality Data Engineering solutions to join our Engineering...