Data Engineering Lead for Cloud-Based Big Data Solutions

2 weeks ago


Bengaluru, Karnataka, India The Nielsen Company Full time
About Nielsen

As a leader in content measurement and data technology, Nielsen empowers its employees to drive innovation and growth. Our company culture prioritizes career development, collaboration, and continuous improvement.

Job Description

We are seeking an experienced Senior/Lead Software Engineer to join our team. This role will focus on designing and implementing cloud-based big data solutions using various technologies such as Hadoop, Spark, and AWS services.

Responsibilities:
  • Design and develop scalable ETL processes for data processing and analytics requirements.
  • Coordinate with product teams to lead the development of technical solutions and manage scrum teams.
  • Implement automation scripts for cloud resource provisioning, unit testing, code coverage, and static code analysis.
  • Develop modular and maintainable software solutions using Python and Java.
  • Maintain and document software standards, best practices, and design patterns.
Qualifications and Skills:
  • 4-6 years of experience with ETL automation, SQL, and BI tools (Superset, Spotfire, etc.).
  • Programming experience in Python and Java, with a desire to write and review code 70% of the time.
  • Familiarity with relational and big data technology stacks such as Postgres, Hadoop, and columnar storage formats Parquet.
  • Hands-on experience with Spark, Spark SQL, Spark on K8s, and M/R Framework.
  • Experience developing cloud-hosted (AWS) applications and services is a plus.
  • AWS Components: EKS, EMR, EC2, S3 storage, Lambda, Relational Database Service, Simple Notification Service (SNS), Elasticache, etc.
Estimated Salary Range:

$120,000 - $180,000 per year, depending on location and experience.



  • Bengaluru, Karnataka, India NTT DATA Full time

    About the Role:As a Cloud Data Engineer, you will be responsible for designing and implementing large-scale data processing systems using GCP's BigQuery, Dataflow, and Dataproc technologies. Your expertise in ETL/ELT pipelines, data warehousing, and lakehouse solutions will enable us to efficiently process structured and unstructured data.Key...


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer to join our team in Bangalore, India. As a key member of our data engineering team, you will design, build, and maintain scalable data pipelines using Databricks on AWS.Job Responsibilities:Design and develop efficient data pipelines using Apache Spark and Databricks.Ingest data from multiple...


  • Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data Engineering Cloud SpecialistAbout the RoleWe are seeking an experienced Data Engineer to join our team at NTT DATA. This is a fantastic opportunity to work on diverse data engineering activities on Azure Data platform and contribute to building a robust data pipeline connecting various enterprise data sources.About YouTo be successful in this...

  • Cloud Engineer

    2 weeks ago


    Bengaluru, Karnataka, India NTT DATA Full time

    Job OverviewNTT DATA, a global leader in business and technology services, is seeking a highly skilled Cloud Engineer - Data Specialist to join our team in Bangalore, India.About the RoleWe are looking for an experienced Cloud Engineer with expertise in designing, building, and maintaining scalable data pipelines using Databricks on AWS. The ideal candidate...


  • Bengaluru, Karnataka, India Infogain Full time

    Job Title: Cloud Engineer for Big Data SolutionsAbout the Opportunity:We are seeking an experienced Cloud Engineer with expertise in big data solutions to join our team at Infogain.Key Responsibilities:Design, develop, and deploy scalable cloud-based big data architecturesCollaborate with cross-functional teams to integrate data engineering and analytics...


  • Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team in Bangalore, India. This role involves designing, building, and maintaining scalable data pipelines using Databricks on AWS.The ideal candidate will have experience with Databricks, including Apache Spark development, and proficiency in Python or Scala for Spark job development....


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    About the RoleWe are seeking an experienced Senior Cloud Data Architect to join our team at NTT DATA Services. As a key member of our data engineering team, you will be responsible for designing and implementing cloud-based data solutions on Azure Data platform.ResponsibilitiesDesign and develop big data processing notebooks using Azure DatabricksScripting...

  • Data Engineer

    1 week ago


    Bengaluru, Karnataka, India NTT DATA Full time

    Job OverviewWe are seeking a highly skilled Data Engineer to join our team in Bangalore, India. As a key member of our team, you will play a crucial role in designing and implementing data pipelines, ensuring high-quality data for business insights.About the RoleThis is an exciting opportunity to work with cutting-edge technology and contribute to the growth...


  • Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Data Engineer Cloud Migration Specialist to join our team in Bangalore, India.The successful candidate will be responsible for designing, building and maintaining scalable data pipelines using Databricks on AWS, as well as integrating data from multiple sources using AWS services such as S3, Lambda, Glue, and...


  • Bengaluru, Karnataka, India NTT DATA Full time

    NTT DATA is a global technology services company with a presence in over 50 countries.We have a wide range of job opportunities available for talented professionals like you.About the Job:Job DescriptionWe are seeking a Senior Cloud Data Architect to join our team in Bangalore, India. The successful candidate will be responsible for designing and...


  • Bengaluru, Karnataka, India S P A Enterprise Full time

    S P A Enterprise is a leading provider of big data engineering services, and we're looking for talented individuals to join our team.We're offering a highly competitive salary of $180,000 - $250,000 per year, depending on experience, plus benefits such as medical, dental, and vision insurance, 401(k) matching, and generous paid time off.Job DescriptionWe're...

  • Cloud Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India NTT DATA Services Full time

    We are seeking a skilled Cloud Data Engineer to join our team in Bangalore, India.About the JobThis role involves designing, building, and maintaining scalable data pipelines using Databricks on AWS. You will work with multiple data sources, including structured and unstructured data, and use AWS services such as S3, Lambda, Glue, and Kinesis for data...


  • Bengaluru, Karnataka, India The Nielsen Company Full time

    About the RoleThe Nielsen Company is looking for a skilled Data Engineering Specialist - Big Data Solutions to join our team. In this role, you will be responsible for designing, developing, and deploying large-scale data processing systems using big data technologies such as Apache Spark, Hadoop, and AWS Cloud.Key ResponsibilitiesDesign and develop scalable...


  • Bengaluru, Karnataka, India Mercedes-Benz Full time

    Job DescriptionAt Mercedes-Benz, we are seeking an experienced Data Engineer to join our team. As a key member of our cloud and big data team, you will be responsible for designing, developing, and maintaining large-scale data processing systems.About the RoleWe are looking for a highly skilled data engineer with experience in building and managing data...


  • Bengaluru, Karnataka, India The Nielsen Company Full time

    Job OverviewWe are seeking an experienced Staff Software Engineer to join our team at The Nielsen Company. This is a fantastic opportunity to work with cutting-edge technologies and collaborate with creative engineers on high-performance projects.About Nielsen Global MediaNielsen Global Media uses innovative technology and industry-leading data science to...


  • Bengaluru, Karnataka, India AppSierra Solutions Pvt Ltd Full time

    We are seeking a seasoned data engineer to lead the development and deployment of big data pipelines using AWS DataBricks.With 5-7 years of experience in software development on AWS, you will have strong hands-on knowledge of AWS DataBricks, DevOps, Kubernetes, and cloud services such as S3, Redshift, EC2, and Lambda. Additionally, you will have extensive...


  • Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly experienced and dynamic Strategic Data Architect Leader to join our Data, Analytics and AI team.Job SummaryThe ideal candidate will have a strong background in Data, Big Data and related technology and management, with a proven track record of leading innovation and architecture initiatives.Key ResponsibilitiesLead the...


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    Data Engineering Senior Specialist RoleDiverse and dynamic Data Engineering activities on Azure Data platform. The role involves building and delivering Data pipelines connecting various enterprise data sources, both RDBMS and NoSQL. Develop data mappings to existing data sources and design big data processing notebooks using Azure Databricks. Scripting and...

  • Data Engineer Lead

    1 day ago


    Bengaluru, Karnataka, India MonoSpear Technologies Pvt Ltd Full time

    Job SummaryWe are seeking a highly skilled Data Engineer Lead to join our team at MonoSpear Technologies Pvt Ltd. As a key member of our data engineering team, you will be responsible for designing and developing scalable data management solutions using cloud-based technologies. About the RoleThe ideal candidate will have 9+ years of experience in data...


  • Bengaluru, Karnataka, India Prime Search Consultant Full time

    Job DescriptionWe are seeking a highly skilled Big Data Architect and Engineering Lead to join our team at Prime Search Consultant. This role will be responsible for architecting and developing large-scale data systems, leading cross-functional teams, and collaborating with stakeholders to drive business outcomes.About the Role:This is an exciting...