High Salary Lead Data Engineer

About Delta Air Lines About the Company Delta Air Lines NYSE DAL is the U S global airline leader in safety innovation reliability and customer experience Powered by our employees around the world Delta has for a decade led the airline industry in operational excellence while maintaining our reputation for award-winning customer service With our mission of connecting the people and cultures of the globe Delta strives to foster understanding across a diverse world and serve as a force for social good In 2024 Delta was recognized by Fortune with a No 11 placement on Fortune s Top 50 Most Admired Companies list The company s strong management and commitment to providing elevated experiences and premium products also secured a No 1 ranking out of the eight airlines on the list Delta landed on TIME s inaugural list of the World s Best Companies coming in at No 12 - the only U S airline in the top 155 TIME s award is based on three criteria employee satisfaction revenue growth and sustainability Delta s people-first culture continues to be recognized earning the airline a spot on Fortune s 100 Best Companies to Work For list for the fifth year Delta is the only airline included on the 2024 list Additionally Delta earned a coveted spot on Fast Company s list of the Most Innovative Companies climbing from its No 8 spot in 2023 to No 2 in the travel category The airline was recognized for its Wi-Fi revolution that is working to ensure the future of travel is connected About the Delta Technology Hub DTH Bangalore Delta has fast emerged as a customer-focused innovation-led technology-enabled business The Delta Technology Hub will contribute directly to these objectives About Delta Tech Hub Delta Air Lines NYSE DAL is the U S global airline leader in safety innovation reliability and customer experience Powered by our employees around the world Delta has for a decade led the airline industry in operational excellence while maintaining our reputation for award-winning customer service With our mission of connecting the people and cultures of the globe Delta strives to foster understanding across a diverse world and serve as a force for social good Delta has fast emerged as a customer-oriented innovation-led technology-driven business The Delta Technology Hub will contribute directly to these objectives It will sustain our long-term aspirations of delivering niche IP-intensive high-value and innovative solutions It supports various teams and functions across Delta and is an integral part of our transformation agenda working seamlessly with a global team to create memorable experiences for customers Responsibilities Marketing Automation Deployment Oversee the deployment of marketing automation workflows and jobs into production environments ensuring reliability scalability and performance Collaborate with cross-functional teams to manage application deployment and configuration adhering to DevOps best practices Technology Modernization Migration Lead the migration of legacy systems to modern technologies including Python and AWS promoting the adoption of scalable cloud-native solutions Support the team in transitioning from SAS UNIX-based systems to a more agile and maintainable Python AWS framework Automation Framework ETL Development Maintain and enhance the existing automation framework using SAS UNIX Python and AWS in partnership with developers and IT stakeholders Design develop and maintain robust ETL Extract Transform Load pipelines leveraging AWS services such as Glue Lambda and Step Functions Architecture Infrastructure Alignment Collaborate with solution architects and platform leads to ensure automation pipelines align with the organizations future-state marketing technology architecture Contribute to the evolution of the marketing platform by supporting integrations with Customer Data Platforms CDPs journey orchestration tools and AI-driven workflow engines Data Governance Documentation Ensure all solutions adhere to industry best practices for version control security and data governance Create and maintain comprehensive documentation of pipeline architecture data workflows and operational processes for both technical and non-technical stakeholders Agile Development Collaboration Actively participate in daily stand-ups sprint planning and retrospectives Work closely with team members to refine requirements test solutions and debug issues to meet evolving business and technical needs Innovation Continuous Improvement Stay abreast of industry trends emerging tools and best practices in data engineering and marketing technology Develop proof-of-concepts and propose innovative solutions to technical and non-technical teams driving continuous improvement in data workflows and marketing insights What you need to succeed minimum qualifications Education Bachelors degree in Computer Science Engineering Information Systems or a related field or an equivalent combination of education training and hands-on experience in computer systems engineering Experience Minimum 6 years of experience designing implementing and managing large-scale enterprise data warehouse solutions particularly in Teradata or Amazon Redshift environments Experience building and maintaining ETL pipelines using AWS services Technical Skills Strong understanding of relational and non-relational data warehouse architectures and best practices Proficient in Python SQL PySpark and Unix shell scripting with a focus on writing clean maintainable and well-documented code Hands-on experience with AWS services including but not limited to Amazon MWAA Managed Workflows for Apache Airflow AWS Glue AWS Lambda Amazon S3 Amazon Athena Design develop and maintain robust ETL Extract Transform Load processes leveraging AWS Serverless Services such as Glue Lambda and Step Functions Maintain and enhance the existing automation framework using SAS UNIX Python and AWS Serverless Services Data Expertise Proven ability to work with large complex datasets ensuring performance and scalability Strong analytical and problem-solving skills with a track record of delivering efficient and reliable data solutions Marketing Technology Exposure Experience with Customer Data Platforms CDPs real-time journey orchestration and AI-driven marketing systems such as Adobe Experience Platform and Adobe Journey Optimizer Experience supporting data integrations between cloud infrastructure and marketing engagement platforms Familiarity with Salesforce Marketing Cloud Automation Studio Journey Builder and integration methods APIs SFTP etc Communication Collaboration Excellent verbal and written communication skills with the ability to convey complex technical concepts to both technical and non-technical stakeholders What will give you a competitive edge preferred qualifications Knowledge or related experience in the airline industry AWS Certifications ID DELSL06