Senior Data Engineer

2 weeks ago

BangaloreKarnataka, India Kloud9 Full time

Job Description :


As a Senior Data Engineer, you will partner with Software Engineers, Data Scientists and Business Intelligence Engineers. You will gain a deep understanding of our services and the data they produce and become our resident expert in how to transform that data into a format that is useful for analytics and business intelligence. You will proactively help to identify new data for integration with the platform and propose and implement new technologies to help us better understand our data.

- Collaborate with cross-functional teams including data scientists, analysts, and software engineers to understand data requirements and design optimal data solutions on AWS.

- Design, develop, and maintain data pipelines and ETL processes to move data from various sources into our data lake and data warehouse on AWS.


- Implement efficient and reliable data transformation, cleansing, and enrichment processes to ensure high- quality data for downstream analytics and reporting.

- Optimize data pipelines for performance, scalability, and cost-effectiveness using AWS services such as Amazon Redshift, Amazon S3, AWS Glue, AWS Lambda, etc.

- Ensure data security and compliance by implementing appropriate data access controls, encryption mechanisms, and auditing processes. Monitor and troubleshoot data pipeline issues, performance bottlenecks, and data quality problems, and implement solutions to address them.

- Design and implement product features in collaboration with business and Technology stakeholders. Anticipate, identify, and solve issues concerning data management to improve data quality.

- Design and build reusable components, frameworks, and libraries at scale to support analytics products. Clean, prepare and optimize data at scale for ingestion and consumption.

- Clean, prepare and optimize data at scale for ingestion and consumption. Drive the implementation of new data management projects and re-structure of the current data architecture.

- Implement complex automated workflows and routines using workflow scheduling tools. Build continuous integration, test-driven development, and production deployment frameworks.

- Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards. Analyse and profile data for the purpose of designing scalable solutions.

- Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues. Mentor and develop other data engineers in adopting best practices.

Education :

- BE / BTECH | MTech | MCA

- Minimum 4+ years of work experience in development / Migration project.

- Experience working with distributed technology tools for developing Batch and Streaming pipelines using SQL, Spark, Python, Airflow, Scala

- Experience in Cloud Computing, e.g., AWS, GCP, Azure, etc.

- Able to quickly pick up new programming languages, technologies, and frameworks.

- Strong skills building positive relationships across Product and Engineering.

- Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders

- Experience with creating/ configuring Jenkins pipeline for smooth CI/CD process for Managed Spark jobs, build Docker images, etc.

- Working knowledge of Data warehousing, Data modelling, Governance and Data Architecture

Good to have :

- Experience working with Data platforms, including EMR, Airflow, Data bricks (Data Engineering & Delta Lake components, and Lakehouse Medallion architecture), etc.

- Experience working in Agile and Scrum development process.

- Experience in EMR/ EC2, Data bricks etc.

- Experience working with Data warehousing tools, including SQL database, Presto, and Snowflake

- Experience architecting data product in Streaming, Server less and Microservices Architecture and platform.

(ref:hirist.tech)

  • Senior data engineer

    1 day ago

    Bangalore, India Rakuten Symphony Full time

    Job Title: Senior Data Engineer Location: Bangalore, India Company: Rakuten Symphony, India Overview Rakuten Symphony is seeking an experienced and strategic Head/Senior Data Engineer to lead our data engineering initiatives and transform mobile connectivity through innovative data solutions. This senior leadership role requires deep expertise in...

  • Data engineer

    2 months ago

    Bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost...

  • Senior Data Engineer

    2 weeks ago

    Bangalore, India Rakuten Symphony Full time

    Job Title: Senior Data Engineer Location: Bangalore, India Company: Rakuten Symphony, India Overview Rakuten Symphony is seeking an experienced and strategic Head/Senior Data Engineer to lead our data engineering initiatives and transform mobile connectivity through innovative data solutions. This senior leadership role requires deep expertise...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, India Rakuten Symphony Full time

    Job Title: Senior Data Engineer Location: Bangalore, India Company: Rakuten Symphony, India Overview Rakuten Symphony is seeking an experienced and strategic Head/Senior Data Engineer to lead our data engineering initiatives and transform mobile connectivity through innovative data solutions. This senior leadership role requires deep expertise...

  • Data Engineer

    4 weeks ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. - Lead the architecture, design, and delivery of enterprise grade microservices. - Deliver assigned projects within budgeted time, cost and...

  • Data Engineer

    4 weeks ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and...

  • Data Engineer

    2 months ago

    Bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and...

  • Data Engineer

    1 month ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data EngineerLocation: Yemalur, BangaloreYour Key Responsibilities:Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices.Deliver assigned projects within budgeted time, cost and agreed...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, India ThoughtFocus Full time

    Job Description: Senior Data Engineer Position: Senior Data Engineer Experience: 8-13 Years Location: Bangalore Employment Type: Full-Time Must have experience in Capital Market/Investment banking/BFSI domain. ONLY IMMEDIATE JOINERS (0-15 DAYS) Summary: We are looking for a highly skilled Senior Data Engineer with 8-13 years of...

  • Senior Data Engineer

    1 month ago

    bangalore, India ThoughtFocus Full time

    Job Description: Senior Data Engineer Position: Senior Data Engineer Experience: 8-10 Years Location: Bangalore Employment Type: Full-Time Must have experience in Capital Markets, Investment banking domain. --- Summary: We are looking for a highly skilled Senior Data Engineer with 8-10 years of experience in SQL, Azure Data Factory (ADF), SQL Server...

  • Senior Data Engineer

    1 month ago

    bangalore, India ThoughtFocus Full time

    Job Description: Senior Data Engineer Position: Senior Data Engineer Experience: 8-10 Years Location: Bangalore Employment Type: Full-Time Must have experience in Capital Markets, Investment banking domain. --- Summary: We are looking for a highly skilled Senior Data Engineer with 8-10 years of experience in SQL, Azure Data Factory (ADF), SQL Server...

  • Senior Data Engineer

    1 month ago

    Bangalore, India ThoughtFocus Full time

    Job Description: Senior Data Engineer Position: Senior Data Engineer Experience: 8-10 Years Location: Bangalore Employment Type: Full-Time Must have experience in Capital Markets, Investment banking domain. --- Summary: We are looking for a highly skilled Senior Data Engineer with 8-10 years of experience in SQL, Azure Data Factory (ADF),...

  • Senior Data Engineer

    1 month ago

    bangalore, India ThoughtFocus Full time

    Job Description: Senior Data EngineerPosition: Senior Data EngineerExperience: 8-10 YearsLocation: BangaloreEmployment Type: Full-TimeMust have experience in Capital Markets, Investment banking domain.---Summary:We are looking for a highly skilled Senior Data Engineer with 8-10 years of experience in SQL, Azure Data Factory (ADF), SQL Server Integration...

  • Data Engineer

    3 weeks ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and...

  • Data Engineer

    2 months ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and...

  • Data Engineer

    2 months ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and agreed...

  • Senior Data Engineer

    2 months ago

    Bangalore, India Talent Corner HR Services Full time

    Job Title: Senior Data EngineerJob Summary:We are seeking a skilled and experienced Data Engineer to join our dynamic team at Talent Corner HR Services. As a Senior Data Engineer, you will be responsible for designing, implementing, and maintaining our data infrastructure, pipelines, and systems.Key Responsibilities:Lead the design and implementation of...

  • Senior Data Engineer

    4 weeks ago

    Bangalore, India Capco Full time

    Greetings from Capco... We are hiring for Senior Data Engineer for Bangalore Location. Qualifications: Proven 5-8 years of experience as a Senior Data Engineer or similar role. • Experience with big data tools: Hadoop, Spark, Scala, Hive, etc. • Expert level SQL skills for data manipulation (DML) and validation (DB2). • Experience with...

  • Senior Data Engineer

    4 weeks ago

    bangalore, India Capco Full time

    Greetings from Capco...We are hiring for Senior Data Engineer for Bangalore Location.Qualifications:Proven 5-8 years of experience as a Senior Data Engineer or similar role.• Experience with big data tools: Hadoop, Spark, Scala, Hive, etc.• Expert level SQL skills for data manipulation (DML) and validation (DB2).• Experience with data pipeline and...

  • Senior Data Engineer

    1 month ago

    bangalore, India Synechron Full time

    Good-day, We have an immediate opportunity for the Senior Data Engineer Job Role: Senior Data Engineer Job Location: Bangalore BCIT Experience- 7 to 10 years Notice : upto 15 Days . About Company: At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative...