Data Engineer

4 weeks ago

Bengaluru, Karnataka, India The Metromax Group Full time

Job Summary

Data Engineer - Cloud Data Architect

We are seeking a highly skilled Data Engineer - Cloud Data Architect to join our team at The Metromax Group. As a key member of our data engineering team, you will be responsible for designing and developing cloud-based data pipelines using AWS Glue, EMR, Spark Scala, and S3 to support both batch and real-time data processing needs.

Key Responsibilities

- Design and develop cloud-based data pipelines using AWS Glue, EMR, Spark Scala, and S3 to support both batch and real-time data processing needs.

- Implement ETL processes to extract, transform, and load data from various sources (structured and unstructured) into the data lake.

- Leverage Apache Spark on EMR for big data processing and transformations, using Spark Scala.

- Manage and optimize data storage on S3, ensuring proper data partitioning, file formats (Parquet, ORC, Avro), and lifecycle policies for cost-effective storage solutions.

- Monitor, troubleshoot, and optimize EMR clusters for performance, scalability, and cost efficiency.

- Collaborate with data architect and analysts to ensure seamless data integration and support advanced analytics and machine learning models.

- Automate data workflows using AWS Step Functions, Lambda, and other orchestration tools.

Required Qualifications

- Bachelor's or master's degree in computer science, Data Engineering, or a related technical field.

- 3-5 years of experience in data engineering, particularly using AWS services (EMR, Glue, S3, Lambda).

- Strong expertise in Apache Spark for distributed data processing, with hands-on coding experience in Scala and Python.

- Experience with building ETL pipelines and working with big data in a cloud-based Lakehouse environment.

- Deep understanding of data formats (Parquet, Avro, ORC) and file optimization techniques.

- Familiarity with data modeling principles, including partitioning, bucketing, and schema management in AWS Glue Data Catalog.

- Strong knowledge of SQL and query optimization for working with large datasets.

- Experience with AWS security services such as IAM, KMS (Key Management Service), and encryption best practices.

- Proficiency in troubleshooting and performance tuning of Spark and EMR clusters for large-scale data processing.

- Familiarity with CI/CD pipelines and infrastructure-as-code (Terraform, CloudFormation) for managing AWS environments.

Preferred Qualifications

- AWS Certified Data Analytics, Developer, or Solutions Architect certification.

- Experience with streaming data technologies such as Kinesis or Kafka for real-time data ingestion.

- Knowledge of serverless computing and experience with AWS Lambda, Step Functions, and DynamoDB.

- Familiarity with DevOps and automation tools (e.g., Jenkins, Git, Docker).

Soft Skills

- Strong problem-solving and analytical thinking skills.

- Ability to work collaboratively in a fast-paced, cross-functional environment.

- Excellent communication skills to explain complex technical issues to both technical and non-technical stakeholders.

  • Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job DescriptionData Engineer - Diverse Data Engineering Activities on Azure Data PlatformMain Responsibilities:Design and develop big data processing notebooks using Azure DatabricksLead the data identification, data analysis efforts for data sources and work directly with data owner teamsDevelop data requirements for new data sourcesDesign and development...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Description Data Engineer - Cloud Solutions Diverse Data Engineering activities on Azure Data platform. Build & Deliver Data pipeline connecting various enterprise data sources. Develop data mappings to existing data sources. Design and develop big data processing notebooks using Azure Databricks. Scripting and programming using programming languages...

  • Data Visualization Engineer

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data Visualization EngineerJob Summary:Data Visualization Engineer - NTT DATAData visualization engineers utilize expertise in information visualization and design to create and deploy new platforms and services for visualization initiatives. Visualization specialists build interactive and complex data visualizations and analysis tools. They are...

  • Data Engineering Expert

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a skilled Data Engineer to join our team in Bangalore, India. As a key member of our data engineering team, you will play a crucial role in designing and implementing automated data pipelines.

  • Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job DescriptionReq ID: 301470NTT DATA is seeking a skilled Data Engineer to partner with stakeholders across business units to develop analyses and documentation in a collaborative way.The ideal candidate will have three or more years of experience with Python, SQL, and data visualization/exploration tools, as well as familiarity with the AWS ecosystem.About...

  • Data Engineering Specialist

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job SummaryReq ID: 302206NTT DATA is seeking a highly skilled Senior Data Engineer to join our team in Bangalore, India.Job PurposeThe Senior Data Engineer will play a key role within the Data & Analytics team, designing, building, and maintaining systems and infrastructure for processing, collecting, storing, and analyzing large volumes of data.Key...

  • Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at NTT DATA Services. As a Data Engineer, you will be responsible for designing and developing data pipelines utilizing Cloud technologies Azure/AWS/GCP.Key ResponsibilitiesDesign and develop data pipelines using Cloud technologies Azure/AWS/GCPLead a team of Data Engineers and act...

  • Data Systems Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleNTT DATA is seeking a highly skilled Senior Data Engineer to join our team in Bangalore, India. The ideal candidate will have a strong background in designing, building, and maintaining data systems.Key ResponsibilitiesDesign and develop data warehouses and data models to support business intelligence and analytics.Create and maintain data...

  • Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job DescriptionKey Skills and Qualifications:Cloud Platforms: Proficient in AWS.Programming Languages: Expert in SQL and Python.ETL Tools: Experience with ETL tools and frameworks.Requirements:Education: Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.Skills:Proven experience in using Snowflake for data...

  • Senior Data Engineer

    1 month ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job OpportunityReq ID: 286326 NTT DATA Services is a global leader in IT and business services, committed to helping clients succeed in a rapidly changing world. We are seeking a talented Sr Data Engineer to join our team in Bangalore, India.This role requires a strong background in AWS cloud services, including Lambda, Glue, S3, EMR, SNS, SQS, CloudWatch,...

  • Data Engineering Cloud Specialist

    5 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data Engineering Cloud SpecialistAbout the RoleWe are seeking an experienced Data Engineer to join our team at NTT DATA. This is a fantastic opportunity to work on diverse data engineering activities on Azure Data platform and contribute to building a robust data pipeline connecting various enterprise data sources.About YouTo be successful in this...

  • Data Engineer Lead

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    Data Engineer LeadData Engineers within NTT DATA Services drive the creation of innovative data solutions that benefit our clients. As a Data Engineer Lead, you will be responsible for leading diverse data engineering activities on the Azure Data platform, including the development and delivery of data pipelines connecting various enterprise data sources,...

  • Power BI Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleNTT DATA is seeking a highly skilled Data Engineer for Power BI Development to join our team in India.The ideal candidate will have extensive experience with Power BI reporting development, a solid understanding of relational and distributed database systems, and strong analytical skills.Key Responsibilities:Develop and implement Power BI...

  • Data Engineering Specialist

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the JobThis position requires a highly skilled Data Engineer to design, build, and maintain scalable data pipelines using Databricks on AWS. The ideal candidate will have strong experience with Databricks, including Apache Spark development, and proficiency in Python or Scala for Spark job development.Key ResponsibilitiesDesign, build, and maintain...

  • Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    Job DescriptionWe are seeking a highly skilled Data Engineer to join our team at NTT DATA Services.About the Role:The ideal candidate will have a strong background in data engineering, with expertise in designing and implementing automated data pipelines, analyzing complex datasets, and providing actionable insights.Key Responsibilities:Develop and maintain...

  • Data Engineering Specialist

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    Job Requirements:NTT DATA Services is seeking a skilled Data Engineer to join our team in Bangalore, India. In this role, you will work closely with the GenAI architect to design and implement automated data pipelines.Key Responsibilities:Analyze and understand source data, preparing and managing data sets for actionable insights.Collaborate with Data...

  • Data Engineer

    1 month ago

    Bengaluru, Karnataka, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data EngineerLocation: Yemalur, BangaloreJob Summary:We are seeking a highly skilled Lead Data Engineer to join our team at Data Labs - LANDMARK GROUP. The successful candidate will be responsible for leading our engineering team in building and deploying cutting-edge analytics products.Key Responsibilities:Lead the architecture, design, and...

  • Data Engineer

    1 month ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data Engineer - AWS and Oracle ExpertDepartment: IT - Application DevelopmentJob Summary:We are seeking a highly skilled and experienced Data Engineer with expertise in data development in AWS and Oracle. The ideal candidate will have a minimum of 5 years of experience in data engineering, with a strong focus on AWS and Oracle technologies. The...

  • Data Engineer

    1 month ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data EngineerJob Summary:As a Data Engineer at NTT DATA, you will be responsible for designing, building, and maintaining large-scale data processing systems using Azure data factory and Azure databricks. You will work closely with cross-functional teams to ensure data quality, integrity, and security.Key Responsibilities:Design and implement data...

  • Senior Data Engineering Specialist

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a skilled Senior Data Engineering Specialist to join our team in Bangalore, India. The successful candidate will have a strong background in ETL development and data engineering using Azure concepts and technologies.Key ResponsibilitiesDesign and implement high and low-level design documents for ETL processes.Develop ETL code in...