AWS Data Engineer

1 month ago

bangalore, India Lilly Full time

We’re looking for people who are determined to make life better for people around the world.


Job Description
As a Cloud Data Engineer, you will play a crucial role in designing, implementing, and maintaining our cloud-based data infrastructure. You will collaborate with cross-functional teams to ensure data availability, reliability, and security, enabling data-driven decision-making and insights.
Key Responsibilities
 

Design and develop data pipelines to extract, transform, and load (ETL) data from various sources into cloud-based data storage solutions. Implement data integration solutions using cloud services such as AWS (Amazon Web Service) Migrate Data pipelines previously developed with Informatica and similar tools to AWS Native data pipelines. Incorporate deep data management expertise into solutions, including permissions, recovery, security, and monitoring. Validate and qualify solutions. Identify, troubleshoot, debug, and resolve technical issues. Work with business and IT groups to understand source systems and enterprise infrastructure offerings. Work with data from multiple data sources to build integrated views that will drive decisions. Collaborate with data scientists, analysts, and software engineers to understand data requirements and ensure the availability of quality data. Optimize and tune data pipelines for performance and scalability. Implement data security and access controls to protect sensitive information. Monitor, troubleshoot, and resolve data pipeline and infrastructure issues. Stay current with emerging data engineering technologies and best practices. Document data engineering processes and standards. Work with stakeholders to define data architecture and modelling requirements. Provides technical leadership and consultancy to Global Services in close partnership with Global Services Architect to ensure the delivery of cloud solutions is aligned with the Enterprise IT (Information Technology) architecture. Creates and reviews system designs to leverage existing standards, to evaluate alternative technologies and to build a cost-effective solution. Develops a deep expertise on cloud platforms/technologies used by Lilly Solution Services to assist system custodians with life cycle management or complex changes. Ensures coding standards or technical implementation guides are in place for cloud platforms or technologies and educates External Service Providers to ensure they are followed. Helps to establish a DevOps to improve the quality, security, automation capabilities and stability of our cloud solutions. Identifies synergies and reuse opportunities to improve efficiency and cost savings.

Qualifications
 

Bachelor’s degree in computer science, Information Technology, or a related field (master’s preferred). Proven experience as a data engineer, preferably in a cloud environment. Prior experience in migrating from informatica workflow to AWS native processing. Strong experience with AWS ETL/File Movement tools (Glue, Athena, Lambda, and other AWS integration stack) Strong experience with Agile Development, CI/CD, Jenkins, GitHub, Automation platforms Strong experience with AWS database technologies (Redshift, Aurora, RDS, S3 & other AWS Data Service) covering security, policies, access management. Strong programming Experience with Lambda, Python and Spark Experience with Apache Airflow, Ansible & other automation stack. Familiarity with data modeling, data warehousing, and data architecture best practices. Strong problem-solving and analytical skills. Excellent communication and collaboration skills. Ability to work in a fast-paced and dynamic environment.
 

Technical Experience
Must Have
Postgres SQL
AWS Glue / Lambda
Python / Py Spark
AWS S3 / Redshift / Aurora DB / Athena
Airflow

Github Actions

PL/SQL

Agile
Good to Have
Power BI / Tableau

Informatica
AWS Step Functions/Event Bridge
AWS Glue Catalog
AWS CloudTrail
Oracle / MS SQL
 

Lilly does not discriminate on the basis of age, race, color, religion, gender, sexual orientation, gender identity, gender expression, national origin, protected veteran status, disability or any other legally protected status.

#WeAreLilly

  • Data Scientist, AWS Support Strategy Analytics and Data Science, Support Engineering

    4 days ago

    bangalore, India AWS India - Karnataka - D11 Full time

    AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success....

  • Aptus Data Labs

    2 months ago

    bangalore, India Aptus Data Labs Full time

    Aptus Data Labs is an innovative Data Engineering and Advanced Analytics company offering analytical solutions and consulting services.Role - Machine Learning EngineerExp - 3-5 YrsLocation - RemoteSkill - AWS Sagemaker , ML Pipeline, Machine Learning, AI, Python, NLP, , CI/CD , AWS Services - Lambda, Step functions and other AWS ServicesThe ideal...

  • Aptus Data Labs

    2 months ago

    Bangalore, India Aptus Data Labs Full time

    Aptus Data Labs is an innovative Data Engineering and Advanced Analytics company offering analytical solutions and consulting services.Role - Machine Learning EngineerExp - 3-5 YrsLocation - RemoteSkill - AWS Sagemaker , ML Pipeline, Machine Learning, AI, Python, NLP, , CI/CD , AWS Services - Lambda, Step functions and other AWS ServicesThe ideal...

  • AWS Data Engineer

    6 days ago

    bangalore, India EXL Full time

    Job Description: As an AWS Data Engineer, you will be responsible for designing, developing, and managing our cloud-based data infrastructure on Amazon Web Services (AWS). The ideal candidate will have extensive experience with AWS services, strong expertise in data engineering principles, and a passion for building robust and scalable data solutions. Key...

  • AWS Data Engineer

    2 weeks ago

    bangalore, India Torry Harris Full time

    Are you an AWS Data Engineer looking for an exciting new opportunity? Look no further than Torry Harris! We are a leading digital transformation company, and we are currently seeking a talented and experienced AWS Data Engineer to join our team.As an AWS Data Engineer at Torry Harris, you will be responsible for designing, implementing, and maintaining our...

  • Sr Data Scientist, AWS Support Strategy Analytics and Data Science, Support Engineering

    2 months ago

    bangalore, India AWS India - Karnataka - D11 Full time

    AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success....

  • Sr Data Scientist, AWS Support Strategy Analytics and Data Science, Support Engineering

    3 weeks ago

    bangalore, India AWS India - Karnataka - D11 Full time

    AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success....

  • Data Engineer

    4 weeks ago

    bangalore, India Concentrix Full time

    Job Description:Experience: 4 Years - 8 YearsJob Location: PAN IndiaNotice Period: Immediate/ 15 DaysShift Timings: 03:00 PM to 12:00 AM (IST)Skills: Data Engineering; data Pipeline; ETL, AWS (and frameworks), Pyspark/ scala; SQL & pythonJob Description:We are seeking an experienced Data Engineer to join our team. The ideal candidate should have a...

  • Data Engineer

    3 weeks ago

    bangalore, India Concentrix Full time

    Job Description:Experience: 4 Years - 8 YearsJob Location: PAN IndiaNotice Period: Immediate/ 15 DaysShift Timings: 03:00 PM to 12:00 AM (IST)Skills: Data Engineering; data Pipeline; ETL, AWS (and frameworks), Pyspark/ scala; SQL & pythonJob Description:We are seeking an experienced Data Engineer to join our team. The ideal candidate should have a...

  • AWS Data Engineer

    3 weeks ago

    bangalore, India PradeepIT Consulting Services Pvt Ltd Full time

    Job Description: As an AWS Data Engineer, you will be responsible for designing, building, and maintaining data pipelines and data storage solutions on Amazon Web Services (AWS). Your expertise will help us leverage data for critical business insights and analytics. Primary Skills: Experience in implementing at least 2 end-to-end Data Engineering projects....

  • AWS Data Engineer

    1 month ago

    bangalore, India PradeepIT Consulting Services Pvt Ltd Full time

    Job Description: As an AWS Data Engineer, you will be responsible for designing, building, and maintaining data pipelines and data storage solutions on Amazon Web Services (AWS). Your expertise will help us leverage data for critical business insights and analytics. Primary Skills: Experience in implementing at least 2 end-to-end Data Engineering projects....

  • Data Engineer

    2 weeks ago

    bangalore, India Programming.com Full time

    Job Description : This is a full-time on-site role as an AWS Data Engineer located in Bengaluru. The AWS Data Engineer will be responsible for daily tasks including data engineering, data modeling, Extract Transform Load (ETL), data warehousing, and data analytics. Qualifications : - Data Engineering and Data Modeling skills- Experience with Extract...

  • Data Engineer

    2 weeks ago

    Bangalore, India Programming.com Full time

    Job Description : This is a full-time on-site role as an AWS Data Engineer located in Bengaluru. The AWS Data Engineer will be responsible for daily tasks including data engineering, data modeling, Extract Transform Load (ETL), data warehousing, and data analytics. Qualifications : - Data Engineering and Data Modeling skills- Experience with Extract...

  • Data Engineer

    2 weeks ago

    Bangalore, Karnataka, India Programming.com Full time

    Job Description :This is a full-time on-site role as an AWS Data Engineer located in Bengaluru. The AWS Data Engineer will be responsible for daily tasks including data engineering, data modeling, Extract Transform Load (ETL), data warehousing, and data analytics. Qualifications :- Data Engineering and Data Modeling skills- Experience with Extract Transform...

  • Data Engineer

    4 weeks ago

    bangalore, India Concentrix Full time

    Job Description: Experience: 4 Years - 8 Years Job Location: PAN India Notice Period: Immediate/ 15 Days Shift Timings: 03:00 PM to 12:00 AM (IST) Skills: Data Engineering; data Pipeline; ETL, AWS (and frameworks), Pyspark/ scala; SQL & python Job Description: We are seeking an experienced Data Engineer to join our team. The ideal candidate should...

  • Data Engineer

    7 days ago

    bangalore, India Talhive HR Management LLP Full time

    About Loopio :Loopio supercharges the way companies respond to Requests for Proposals (RFPs), Due Diligence Questionnaires (DDQs), and Security Questionnaires. Our platform helps our 900 customers work faster, create higher-quality proposals, and win more business. Our mission is to empower people to focus on what matters. We do this by giving both our...

  • Data Engineer, Infra-Finance Business intelligence

    3 weeks ago

    bangalore, India AWS India - Karnataka Full time

    Do you have a desire to make a major contribution to the future in the rapid growth environment of Cloud Computing? Are you passionate about data and code? Does the prospect of dealing with massive volumes of data excite you? Do you want to build data engineering solutions that process millions of records? Amazon Web Services is looking for a Data Engineer...

  • Data Engineer, Infra-Finance Business intelligence

    4 weeks ago

    bangalore, India AWS India - Karnataka Full time

    Do you have a desire to make a major contribution to the future in the rapid growth environment of Cloud Computing? Are you passionate about data and code? Does the prospect of dealing with massive volumes of data excite you? Do you want to build data engineering solutions that process millions of records? Amazon Web Services is looking for a Data Engineer...

  • AWS Data Engineer

    7 days ago

    Bangalore Urban, India EXL Full time

    Job Description: As an AWS Data Engineer, you will be responsible for designing, developing, and managing our cloud-based data infrastructure on Amazon Web Services (AWS). The ideal candidate will have extensive experience with AWS services, strong expertise in data engineering principles, and a passion for building robust and scalable data solutions.Key...

  • Data Analytics Engineer, BizOps

    1 month ago

    bangalore, India AWS India - Karnataka Full time

    AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector.Amazon’s technology business has a history and tradition of leading the world in web-related technologies and services. Now, with...