Data Engineer III, Music Data Lake
5 months ago
If you love the challenges that come with big data then this role is for you. We collect billions of events a day, manage petabyte scale data on Redshift and S3, and develop data pipelines using Spark/Scala EMR, SQL based ETL, and Java services.
You are a talented, enthusiastic, and detail-oriented Data Engineer, Data Science, Business Intelligence, or Software Development who knows how to take on big data challenges in an agile way. Duties include big data design and analysis, data modeling, and development, deployment, and operations of big data pipelines. You will also help hire, mentor, and develop peers in the the Music Data Experience team including Data Scientists, Data Engineers, and Software Engineers. You'll help build Amazon Music's most important data pipelines and data sets, and expand self-service data knowledge and capabilities through an Amazon Music data university.
This role requires you to live at the cross section of data and engineering. You have a deep understanding of data, analytical techniques, and how to connect insights to the business, and you have practical experience in insisting on highest standards on operations in ETL and big data pipelines. With our Amazon Music Unlimited and Prime Music services, and our top music provider spot on the Alexa platform, providing high quality, high availability data to our internal customers is critical to our customer experiences.
Music Data Experience team develops data specifically for a set of key business domains like personalization and marketing and provides and protects a robust self-service core data experience for all internal customers. We deal in AWS technologies like Redshift, S3, EMR, EC2, DynamoDB, Kinesis Firehose, and Lambda. In 2020 your team will migrate Amazon Music's information model and data pipelines to a data exchange store (Data Lake) and EMR/Spark processing layer. You'll build our data university and partner with Product, Marketing, BI, and ML teams to build new behavioral events, pipelines, datasets, models, and reporting to support their initiatives. You'll also continue to develop big data pipelines.
The successful candidate will work with multiple global site leaders, Business Analysts, Business Intelligence Engineers, Software Developers, Database Engineers, Product Management in addition to stakeholders in sales, finance, marketing and service teams to create a coherent customer view. They will:
- Define and lead the data strategy of various analytical products owned with Amazon Music team.
- Develop and improve the current data architecture using AWS Redshift, AWS S3, and Hadoop/EMR/Spark.
- Improve upon the data ingestion models, ETL jobs, and alarming to maintain data integrity and data availability.
- Stay up-to-date with advances in data persistence and big data technologies and run pilots to design the data architecture to scale with the increased data sets of advertiser experience.
- Design and manage data models that serve multiple releases/feature launches and other business critical reporting.
Key job responsibilities
Build Data Platform and Data Lake solutions
Build Data Engineering tools
Build real time and micro batch data pipelines
Build and manage Data Pipelines
About the team
Data Engineering (IDEA) Team owns the foundational data model and datasets for Amazon Music, the Spark and Datanet ETL jobs and business logic to build them, org wide launch support (when required), the Executive Daily Summary (EDS), and batch dataset data quality and SLAs
BASIC QUALIFICATIONS
- 5+ years of data engineering experience- Experience with data modeling, warehousing and building ETL pipelines
- Experience with SQL
- Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
- Experience mentoring team members on best practices
PREFERRED QUALIFICATIONS
- Experience with big data technologies such as: Hadoop, Hive, Spark, EMR- Experience operating large data warehouses
-
Data Engineer III, Music Data Lake Specialist
4 weeks ago
Bengaluru, Karnataka, India ADCI - Karnataka Full timeAbout the RoleWe are seeking a highly skilled Data Engineer III to join our Music Data Experience team at Amazon Music. As a key member of our team, you will be responsible for designing and developing big data pipelines, data modeling, and data architecture using AWS technologies such as Redshift, S3, EMR, and Spark.Key Responsibilities:Build Data Platform...
-
Data Engineer III, Music Data Lake Specialist
1 month ago
Bengaluru, Karnataka, India ADCI - Karnataka Full timeAbout the RoleAs a Data Engineer III, Music Data Lake, you will be responsible for designing and developing large-scale data pipelines and architectures to support Amazon Music's data-driven initiatives. You will work closely with cross-functional teams to gather requirements, design and implement data solutions, and ensure high-quality data delivery to...
-
Data Engineer
4 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeJob DescriptionData Engineer - Diverse Data Engineering Activities on Azure Data PlatformMain Responsibilities:Design and develop big data processing notebooks using Azure DatabricksLead the data identification, data analysis efforts for data sources and work directly with data owner teamsDevelop data requirements for new data sourcesDesign and development...
-
Senior Data Engineer
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeJob Description Data Engineer - Cloud Solutions Diverse Data Engineering activities on Azure Data platform. Build & Deliver Data pipeline connecting various enterprise data sources. Develop data mappings to existing data sources. Design and develop big data processing notebooks using Azure Databricks. Scripting and programming using programming languages...
-
Data Engineering Cloud Specialist
1 week ago
Bengaluru, Karnataka, India NTT DATA Full timeJob Title: Data Engineering Cloud SpecialistAbout the RoleWe are seeking an experienced Data Engineer to join our team at NTT DATA. This is a fantastic opportunity to work on diverse data engineering activities on Azure Data platform and contribute to building a robust data pipeline connecting various enterprise data sources.About YouTo be successful in this...
-
Data Engineer
6 months ago
Bengaluru, India ADCI - Karnataka Full timeExternal job descriptionAmazon Music is an immersive audio entertainment service that deepens connections between fans, artists, and creators. From personalized music playlists to exclusive podcasts, concert livestreams to artist merch, Amazon Music is innovating at some of the most exciting intersections of music and culture. We offer experiences that serve...
-
Data Engineer Lead
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Services Full timeData Engineer LeadData Engineers within NTT DATA Services drive the creation of innovative data solutions that benefit our clients. As a Data Engineer Lead, you will be responsible for leading diverse data engineering activities on the Azure Data platform, including the development and delivery of data pipelines connecting various enterprise data sources,...
-
Senior Data Engineer
3 weeks ago
Bengaluru, Karnataka, India TE Connectivity Full timeData Engineer Role OverviewAt TE, we are looking for a Senior Data Engineer to join our team. As a Data Engineer, you will be responsible for designing and developing data lakes, managing data flows that integrate information from various sources into a common data lake platform through an ETL Tool.Key ResponsibilitiesDesign and develop data lakes on AWS...
-
Data Engineer for Scalable Data Lake Development
14 hours ago
Bengaluru, Karnataka, India Bosch Global Software Technologies Full timeAbout the Role:We are seeking an experienced Data Engineer to join our team at Bosch Global Software Technologies. As a key member of our data engineering team, you will be responsible for designing, developing, and deploying scalable data pipelines that bring data from multiple sources into our data lake.Key Responsibilities:Design and implement efficient...
-
Data Engineering Specialist
4 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the JobThis position requires a highly skilled Data Engineer to design, build, and maintain scalable data pipelines using Databricks on AWS. The ideal candidate will have strong experience with Databricks, including Apache Spark development, and proficiency in Python or Scala for Spark job development.Key ResponsibilitiesDesign, build, and maintain...
-
Bengaluru, Karnataka, India NR Consulting - India Full timeJob Title: Senior Data Engineer for Databricks and Data LakesLocation: BangaloreExperience: 8-11 yearsJob Description:- 9 to 12 years of experience in developing data lakes and data marts on Databricks, with expertise in ETL tools like Azure Data Factory, Informatica, and Spark.- Develop and maintain ETL processes using Databricks notebooks, implement and...
-
Data Engineer
1 month ago
Bengaluru, Karnataka, India ADCI - Karnataka Full timeAbout the RoleAs a Data Engineer on the Music Data Experience team at Amazon Music, you will be responsible for designing, building, and maintaining large-scale data pipelines and platforms that support our music streaming services. You will work closely with cross-functional teams to develop data solutions that meet the needs of our business and...
-
Data Engineer
3 weeks ago
Bengaluru, Karnataka, India ADCI - Karnataka Full timeAbout the RoleWe are seeking a highly skilled Data Engineer to join our Music Data eXperience team at Amazon Music. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data systems, including data pipelines, data warehouses, and data lakes.Key ResponsibilitiesDesign and develop data pipelines using big data...
-
Data Engineering Senior Specialist
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Services Full timeData Engineering Senior Specialist RoleDiverse and dynamic Data Engineering activities on Azure Data platform. The role involves building and delivering Data pipelines connecting various enterprise data sources, both RDBMS and NoSQL. Develop data mappings to existing data sources and design big data processing notebooks using Azure Databricks. Scripting and...
-
Data Engineer III
1 month ago
Bengaluru, Karnataka, India Mouser Electronics Full timeJob Title: Data Engineer IIIThe Data Engineer III will be responsible for designing, implementing, and maintaining robust, scalable, and efficient data pipelines ensuring seamless flow of data from various sources to destination systems.Key Responsibilities:Design and develop data pipelines using technologies like Apache Spark or Apache Kafka or ETL tools...
-
Azure Data Lake
5 months ago
Bengaluru, India NR Consulting - India Full time**Experience: 4+ Years** *** **Job Description: - ** - Azure Data lake Storage implementation, configuration, data creation & migration, - Creating the test data for Azure Data lake Storage covering the functional testing, performance. - Communicate project concepts and outcomes to client project stakeholders, - Conduct user training and actively promote...
-
Data Engineer
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Services Full timeJob DescriptionWe are seeking a highly skilled Data Engineer to join our team at NTT DATA Services.About the Role:The ideal candidate will have a strong background in data engineering, with expertise in designing and implementing automated data pipelines, analyzing complex datasets, and providing actionable insights.Key Responsibilities:Develop and maintain...
-
Snowflake Data Engineer and Architect
2 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the RoleWe are seeking a highly skilled Snowflake Data Engineer and Architect to join our team in Bangalore, India. This is a senior-level position that requires expertise in designing, developing, and maintaining large-scale data warehousing solutions using Snowflake.Key ResponsibilitiesDesign and implement efficient data models and architectures for...
-
Cloud Engineer
5 days ago
Bengaluru, Karnataka, India NTT DATA Full timeJob OverviewNTT DATA, a global leader in business and technology services, is seeking a highly skilled Cloud Engineer - Data Specialist to join our team in Bangalore, India.About the RoleWe are looking for an experienced Cloud Engineer with expertise in designing, building, and maintaining scalable data pipelines using Databricks on AWS. The ideal candidate...
-
Databricks Data Engineer
4 weeks ago
Bengaluru, India NTT DATA Full timeJob Description Req ID: 301996 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Databricks Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Job Duties: Key...