Lead AWS Data Engineer

2 months ago

Hyderabad, India Manuh Technologies Full time

Position Lead : AWS Data Engineer.

Location : Pune/ Gurgaon/Hyderabad Hybrid.

Experience : 9+.

Job Description :

- 9+ years of experience as a Data Engineer on the AWS Stack PySpark.

- Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, EMR/Spark, RDS, Redshift, Data Sync, DMS, DynamoDB, Lambda, Step Functions, IAM, KMS, SM, Event Bridge, EC2, SQS, SNS, Lake Formation, Cloud Watch, Cloud Trail.

- Programming experience with Python, Shell scripting, and SQL.

- Responsible for building, test, QA & UAT environments using Cloud Formation.

- Build & implement CI/CD pipelines for the EDP Platform using Cloud Formation and Jenkins.

Good to Have :

- Implement high-velocity streaming solutions and orchestration using Amazon Kinesis, AWS Managed Airflow, and AWS Managed Kafka (preferred.

- Solid experience building solutions on AWS data lake/data warehouse.

- Analyze, design, Development , and implement data ingestion pipeline in AWS.

- Knowledge of implementing ETL/ELT for data solutions end to eng.

- Ingest data from Rest APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift.

- Perform the Peer Code Review and, perform code quality analysis, and associated tools end-to-end for Prudential's platform.

- Create detailed, comprehensive, and well-structured test cases that follow best practices and techniques.

- Estimate, prioritize, plan & coordinate quality testing activities.

- Understanding requirements, and data solutions (ingest, storage, integration, processing, access) on AWS.

- Knowledge of implementing RBAC strategy/solutions using AWS IAM and Redshift RBAC model.

- Knowledge of analyzing data using SQL Stored procedure.

- Build automated data pipelines to ingest data from relational database systems, file system , and NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift.

- Build Automated data pipelines to develop test plans, execute manual and automated test cases, help to identify the root causes, and articulate defects clearly.

- Recreate production issues to help determine the issue and verify any fixes.

- Conducting End to End verification and validation for the entire application.

- Creating Jenkins CI pipelines to integrate Sonar/Security scans and test automation scripts.

- Using Git/bitbucket for efficient remote team working, storing framework, and developing test scripts.

- Part of DevOps QA and AWS team focusing on building CI/CD pipeline.

- Part of the release/build team and mainly worked on release management, CI/CD pipeline.

- Deploy multiple instances by using cloud formation templates.

Responsibilities :

- Designing, building, and maintaining efficient, reusable, and reliable code.

- Ensure the best possible performance and quality of high-scale data applications and services.

- Participate in system design discussions.

- Independently perform hands-on Development and unit testing of the applications.

- Collaborate with the development team and build individual components into the enterprise data platform.

- Work in a team environment with the product, QE/QA, and cross-functional teams to deliver a project throughout the whole software development cycle.

- Responsible to identify and resolve any performance issues.

- Keep up to date with new technology development and implementation.

- Participate in code review to make sure standards and best practices are met.

(ref:hirist.tech)

  • Aws data engineer/lead

    2 months ago

    Hyderabad, India HGS - Hinduja Global Solutions Full time

    Hi, Position: AWS Data Engineer/Lead Location: Hyderabad , India Duration: Full time Job description: • Experience with working AWS (6-8 yrs)• Experience working with AWS Glue/EMR • Have sound hands on with AWS Lake formation • Specialized in AWS Athena - external or iceberg tables • Nice to have experience with AWS Cloud formation • Having...

  • AWS Data Engineer/Lead

    3 months ago

    hyderabad, India HGS - Hinduja Global Solutions Full time

    Hi,Position: AWS Data Engineer/LeadLocation: Hyderabad , IndiaDuration: Full timeJob description:• Experience with working AWS (6-8 yrs)• Experience working with AWS Glue/EMR• Have sound hands on with AWS Lake formation• Specialized in AWS Athena - external or iceberg tables• Nice to have experience with AWS Cloud formation• Having Good...

  • AWS Data Engineer/Lead

    3 months ago

    hyderabad, India HGS - Hinduja Global Solutions Full time

    Hi, Position: AWS Data Engineer/Lead Location: Hyderabad , India Duration: Full time Job description: • Experience with working AWS (6-8 yrs) • Experience working with AWS Glue/EMR • Have sound hands on with AWS Lake formation • Specialized in AWS Athena - external or iceberg tables • Nice to have experience with AWS Cloud formation • Having Good...

  • AWS Data Engineer/Lead

    3 months ago

    Hyderabad, India HGS - Hinduja Global Solutions Full time

    Hi, Position: AWS Data Engineer/Lead Location: Hyderabad , India Duration: Full time Job description: • Experience with working AWS (6-8 yrs) • Experience working with AWS Glue/EMR • Have sound hands on with AWS Lake formation • Specialized in AWS Athena - external or iceberg tables • Nice to have experience with AWS Cloud formation • Having Good...

  • AWS Data Engineer/Lead

    3 months ago

    hyderabad, India HGS - Hinduja Global Solutions Full time

    Hi,Position: AWS Data Engineer/LeadLocation: Hyderabad , IndiaDuration: Full timeJob description:• Experience with working AWS [6-8 yrs]• Experience working with AWS Glue/EMR• Have sound hands on with AWS Lake formation• Specialized in AWS Athena - external or iceberg tables• Nice to have experience with AWS Cloud formation• Having Good...

  • AWS Data Engineer/Lead

    3 months ago

    Hyderabad, India HGS - Hinduja Global Solutions Full time

    Hi,Position: AWS Data Engineer/LeadLocation: Hyderabad , IndiaDuration: Full timeJob description:• Experience with working AWS [6-8 yrs]• Experience working with AWS Glue/EMR• Have sound hands on with AWS Lake formation• Specialized in AWS Athena - external or iceberg tables• Nice to have experience with AWS Cloud formation• Having Good...

  • AWS Data Engineer/Lead

    3 months ago

    Hyderabad, India HGS - Hinduja Global Solutions Full time

    Hi,Position: AWS Data Engineer/LeadLocation: Hyderabad , IndiaDuration: Full timeJob description:• Experience with working AWS [6-8 yrs]• Experience working with AWS Glue/EMR• Have sound hands on with AWS Lake formation• Specialized in AWS Athena - external or iceberg tables• Nice to have experience with AWS Cloud formation• Having Good...

  • AWS Data Engineer/Lead Position

    1 month ago

    Hyderabad, Telangana, India HGS - Hinduja Global Solutions Full time

    Job Title: AWS Data Engineer/LeadJob Description:We are seeking a highly skilled AWS Data Engineer/Lead to join our team at HGS - Hinduja Global Solutions.Key Responsibilities:Design and implement data warehousing solutions using AWS Glue and EMR.Develop and maintain data pipelines using AWS Lake Formation.Work with AWS Athena to create and optimize external...

  • AWS Data Engineer/Lead

    2 months ago

    hyderabad, India HGS - Hinduja Global Solutions Full time

    Hi,Position: AWS Data Engineer/LeadLocation: Hyderabad , IndiaDuration: Full timeJob description:• Experience with working AWS (6-8 yrs)• Experience working with AWS Glue/EMR• Have sound hands on with AWS Lake formation• Specialized in AWS Athena - external or iceberg tables• Nice to have experience with AWS Cloud formation• Having Good...

  • Aws data engineer

    7 days ago

    Hyderabad, India Tata Consultancy Services Full time

    Role - AWS Data Engineer Required Technical Skill Set**AWS Glue, Lambda, Athena, Pyspark, S3, IAM, Step functions & Cloud FormationDesired Experience Range**5 + yearsLocation of RequirementChennai/Pune, preferable ChennaiDesired Competencies (Technical/Behavioral Competency)Must-Have**(Ideally should not be more than 3-5)Minimum 4 years of...

  • Aws data engineering

    2 weeks ago

    Hyderabad, India Bilvantis Technologies Full time

    We are looking for a skilled and experienced AWS Data Senior/Lead Engineer to join our Data team.About the jobPosition: AWS Data Engineering - Lead Programmer AnalystTotal Experience: Minimum 3 years to 8 years.Notice Period: ImmediateMust have skills: AWS Services, DW Concepts, ETL Tools, SQL, Python, Py Spark, Ia C, AWS CDK, Airflow.Job Description:...

  • AWS Data Engineering

    2 weeks ago

    Hyderabad, India Bilvantis Technologies Full time

    We are looking for a skilled and experienced AWS Data Senior/Lead Engineer to join our Data team.About the jobPosition:AWS Data Engineering - Lead Programmer AnalystTotal Experience:Minimum 3 years to 8 years.Notice Period:ImmediateMust have skills:AWS Services, DW Concepts, ETL Tools, SQL, Python, PySpark, IaC, AWS CDK, Airflow.Job Description: AWS Data...

  • AWS Data Engineer

    1 week ago

    Hyderabad, India Tata Consultancy Services Full time

    Role - AWS Data Engineer Required Technical Skill Set** AWS Glue, Lambda, Athena, Pyspark, S3, IAM, Step functions & Cloud Formation Desired Experience Range** 5 + years Location of Requirement Chennai/Pune, preferable Chennai Desired Competencies (Technical/Behavioral Competency) Must-Have** (Ideally should not be more than 3-5) Minimum 4...

  • AWS Data Engineer

    7 days ago

    Hyderabad, India Tata Consultancy Services Full time

    Role - AWS Data Engineer Required Technical Skill Set** AWS Glue, Lambda, Athena, Pyspark, S3, IAM, Step functions & Cloud Formation Desired Experience Range** 5 + years Location of Requirement Chennai/Pune, preferable Chennai Desired Competencies (Technical/Behavioral Competency) Must-Have** (Ideally should not be more than 3-5) Minimum 4 years of...

  • Lead AWS Data Engineer

    2 weeks ago

    Hyderabad, India Manuh Technologies Full time

    Position Lead : AWS Data Engineer. Location : Pune/ Gurgaon/Hyderabad Hybrid. Experience : 9+. Job Description : - 9+ years of experience as a Data Engineer on the AWS Stack PySpark. - Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, EMR/Spark, RDS, Redshift, Data Sync, DMS, DynamoDB, Lambda, Step Functions, IAM,...

  • AWS Data Engineering

    2 weeks ago

    Hyderabad, India Bilvantis Technologies Full time

    We are looking for a skilled and experienced AWS Data Senior/Lead Engineer to join our Data team. About the job Position: AWS Data Engineering - Lead Programmer Analyst Total Experience: Minimum 3 years to 8 years. Notice Period: Immediate Must have skills: AWS Services, DW Concepts, ETL Tools, SQL, Python, PySpark, IaC, AWS CDK,...

  • AWS Data Engineering

    2 weeks ago

    Hyderabad, India Bilvantis Technologies Full time

    We are looking for a skilled and experienced AWS Data Senior/Lead Engineer to join our Data team.About the jobPosition: AWS Data Engineering - Lead Programmer AnalystTotal Experience: Minimum 3 years to 8 years.Notice Period: ImmediateMust have skills: AWS Services, DW Concepts, ETL Tools, SQL, Python, PySpark, IaC, AWS CDK, Airflow.Job Description: AWS Data...

  • AWS Data Engineering

    2 weeks ago

    Hyderabad, India Bilvantis Technologies Full time

    We are looking for a skilled and experienced AWS Data Senior/Lead Engineer to join our Data team.About the jobPosition: AWS Data Engineering - Lead Programmer AnalystTotal Experience: Minimum 3 years to 8 years.Notice Period: ImmediateMust have skills: AWS Services, DW Concepts, ETL Tools, SQL, Python, PySpark, IaC, AWS CDK, Airflow.Job Description: AWS Data...

  • AWS Data Engineering

    2 weeks ago

    Hyderabad, India Bilvantis Technologies Full time

    We are looking for a skilled and experienced AWS Data Senior/Lead Engineer to join our Data team. About the job Position: AWS Data Engineering - Lead Programmer Analyst Total Experience: Minimum 3 years to 8 years. Notice Period: Immediate Must have skills: AWS Services, DW Concepts, ETL Tools, SQL, Python, PySpark, IaC, AWS CDK,...

  • AWS Data Engineering

    2 weeks ago

    Hyderabad, India Bilvantis Technologies Full time

    We are looking for a skilled and experienced AWS Data Senior/Lead Engineer to join our Data team. About the job Position: AWS Data Engineering - Lead Programmer Analyst Total Experience: Minimum 3 years to 8 years. Notice Period: Immediate Must have skills: AWS Services, DW Concepts, ETL Tools, SQL, Python, PySpark, IaC, AWS CDK, Airflow. Job...