AWS Data Engineer

3 weeks ago

Anywhere in IndiaMultiple Locations, IN Huquo Full time

Job Description :


- Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, EMR/Spark, RDS, Redshift, Data Sync, DMS, DynamoDB, Lambda, Step Functions, IAM, KMS, SM, Event Bridge, EC2, SQS, SNS, Lake Formation, Cloud Watch, Cloud Trail


- Programming experience with Python, Shell scripting, and SQL

- Responsible for building, test, QA & UAT environments using Cloud Formation.

- Build & implement CI/CD pipelines for the EDP Platform using Cloud Formation and Jenkins

Good to Have :


- Implement high-velocity streaming solutions and orchestration using Amazon Kinesis, AWS Managed Airflow, and AWS Managed Kafka (preferred)

- Solid experience building solutions on AWS data lake/data warehouse


- Analyze, design, Development , and implement data ingestion pipeline in AWS

- Knowledge of implementing ETL/ELT for data solutions end to end

- Ingest data from Rest APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift.

- Perform the Peer Code Review and, perform code quality analysis, and associated tools end-to-end for Prudential's platforms


- Create detailed, comprehensive, and well-structured test cases that follow best practices and techniques

- Estimate, prioritize, plan & coordinate quality testing activities

- Understanding requirements, and data solutions (ingest, storage, integration, processing, access) on AWS

- Knowledge of implementing RBAC strategy/solutions using AWS IAM and Redshift RBAC model

- Knowledge of analyzing data using SQL Stored procedures

- Build automated data pipelines to ingest data from relational database systems, file system , and NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift

- Build Automated data pipelines to develop test plans, execute manual and automated test cases, help to identify the root causes, and articulate defects clearly


- Recreate production issues to help determine the issue and verify any fixes


- Conducting End to End verification and validation for the entire application

- Creating Jenkins CI pipelines to integrate Sonar/Security scans and test automation scripts

- Using Git/bitbucket for efficient remote team working, storing framework, and developing test scripts


- Part of DevOps QA and AWS team focusing on building CI/CD pipeline

- Part of the release/build team and mainly worked on release management, CI/CD pipeline

- Deploy multiple instances by using cloud formation :


- Designing, building, and maintaining efficient, reusable, and reliable code

- Ensure the best possible performance and quality of high-scale data applications and services

- Participate in system design discussions

- Independently perform hands-on Development and unit testing of the applications

- Collaborate with the development team and build individual components into the enterprise data platform

- Work in a team environment with the product, QE/QA, and cross-functional teams to deliver a project throughout the whole software development cycle.

- Responsible to identify and resolve any performance issues

- Keep up to date with new technology development and implementation

- Participate in code review to make sure standards and best practices are met

- Project management : Agile developers take responsibility for estimating, planning, and managing all tasks and report on progress

- Software quality : The Agile developer is also responsible for the quality of the software he/she produces. The team takes responsibility for the quality of the work they produce instead of turning over some code to a separate and independent group for testing

- Teamwork : This includes collaboration with all other team members with the aim to take shared responsibility for the overall efforts

- Understanding user needs: This is about interacting with users as necessary to clarify requirements.

(ref:hirist.tech)

  • Aptus Data Labs

    1 month ago

    Anywhere in India/Multiple Locations, IN Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Aptus Data Labs

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Huquo Full time

    Position : AWS Data EngineerLocation : Pune/ Gurgaon/ HybridExperience : 5+Job Description :- 5+ years of experience as a Data Engineer on the AWS Stack- AWS Solutions Architect or AWS Developer Certification required- Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, EMR/Spark, RDS, Redshift, Data Sync, DMS,...

  • Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Huquo Full time

    Position : AWS Data EngineerLocation : Pune/ Gurgaon/ HybridExperience : 5+Job Description :- 5+ years of experience as a Data Engineer on the AWS Stack- AWS Solutions Architect or AWS Developer Certification required- Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, EMR/Spark, RDS, Redshift, Data Sync, DMS,...

  • Data Engineer

    4 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Talhive HR Management LLP Full time

    About Loopio :Loopio supercharges the way companies respond to Requests for Proposals (RFPs), Due Diligence Questionnaires (DDQs), and Security Questionnaires. Our platform helps our 900 customers work faster, create higher-quality proposals, and win more business. Our mission is to empower people to focus on what matters. We do this by giving both our...

  • Data Engineer

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Talhive HR Management LLP Full time

    About Loopio :Loopio supercharges the way companies respond to Requests for Proposals (RFPs), Due Diligence Questionnaires (DDQs), and Security Questionnaires. Our platform helps our 900 customers work faster, create higher-quality proposals, and win more business. Our mission is to empower people to focus on what matters. We do this by giving both our...

  • Lead Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Pro5.ai Full time

    Job Description :Technical delivery lead on platform and project builds for building and maintaining optimised and highly available data pipelines that facilitate data management and data analytics solutions. The role will drive the development of data processing frameworks and data architectural approaches that scale to handle the business's growing...

  • Lead Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Pro5.ai Full time

    Job Description :Technical delivery lead on platform and project builds for building and maintaining optimised and highly available data pipelines that facilitate data management and data analytics solutions. The role will drive the development of data processing frameworks and data architectural approaches that scale to handle the business's growing...

  • UsefulBI Corporation

    2 months ago

    Anywhere in India/Multiple Locations, IN USEFULBI CORPORATION Full time

    Job Description :Minimum 5+ years' Experience in Data Engineering.Must have good knowledge and experience in Python.Mush have good Knowledge of Pyspark or Spark.Mush have good Knowledge of Databricks.Must have good experience in AWS (Glue, EMR)Typically requires relevant analysis work and domain-area work experience.Expert in the management,...

  • Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Ridgeant Techonologies Full time

    Responsibilities :- Collaborate with stakeholders to understand business requirements and data needs, and translate them into scalable and efficient data engineering solutions using AWS Data Services.- Design, develop, and maintain data pipelines using AWS serverless technologies such as- Glue, S3, Lambda, DynamoDB, Athena, and RedShift.- Implement data...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Ridgeant Techonologies Full time

    Responsibilities :- Collaborate with stakeholders to understand business requirements and data needs, and translate them into scalable and efficient data engineering solutions using AWS Data Services.- Design, develop, and maintain data pipelines using AWS serverless technologies such as- Glue, S3, Lambda, DynamoDB, Athena, and RedShift.- Implement data...

  • AWS Cloud Engineer

    1 month ago

    Anywhere in India/Multiple Locations, IN Hirein5 Full time

    Immediate Joiners Preferred :We are looking for a talented AWS Cloud Engineer to design, deploy, and manage our infrastructure on Amazon Web Services (AWS). The ideal candidate will have a strong background in cloud computing, specifically with Amazon Web Services (AWS), and a proven track record of deploying and managing scalable, high-availability...

  • Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Coders Brain Technology Pvt. Ltd. Full time

    Position Name : Data Engineer (Integration)Experience Required : 10+yrsSalary : As per the Market Standard or Else whatever budget you haveNotice period : Immediate joiner Job Type : Full timeLocation : RemoteJob Description :- Understand all forms of data integration including APIs - Event-based, Streaming, REST and GraphQL API, CDC/Replication, Batch,...

  • Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Coders Brain Technology Pvt. Ltd. Full time

    Position Name : Data Engineer (Integration)Experience Required : 10+yrsSalary : As per the Market Standard or Else whatever budget you haveNotice period : Immediate joiner Job Type : Full timeLocation : RemoteJob Description :- Understand all forms of data integration including APIs - Event-based, Streaming, REST and GraphQL API, CDC/Replication, Batch,...

  • Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN ARAHAS TECHNOLOGIES PRIVATE LIMITED Full time

    We are currently hiring a Pyspark Data Engineer with AWS expertise with7 years of experience Job Description :- The candidate should have strong skills in Pyspark coding and have a background in AWS technologies such as Airflow- Experience working with complex SQL queries- Good with PySpark, Databricks, and Airflow- Bachelor's or Master's degree in...

  • Data Engineer

    1 week ago

    Anywhere in India/Multiple Locations, IN Circuit Compilers Full time

    Job Description :- 5 to 8 years in the build release management, software configuration, design, development, and cloud implementation. - Data Engineer with expertise in Python, Scala, PySpark, AWS S3, Glue, Redshift, Lambda, DBT, Serverless, Snowflake, SQL, Datalake, and Data Pipeline. - Leveraged AWS Glue for data cataloging, ETL job orchestration, and...

  • AWS Platform Engineer

    4 weeks ago

    Metros/Anywhere in India/Multiple Locations, IN Antal International Full time

    Job Description :We are seeking an experienced AWS Subject Matter Expert (SME) who will be able to design and build solutions for one of our Fortune 500 Client programs, with expertise in Lake Formation to join our team. The successful candidate will be responsible for designing, building, and maintaining our company's data lakes on AWS. This includes...

  • Cloud Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Circuit Compilers Full time

    Job Responsibilities :- Implement scalable, highly available, and secure AWS cloud infrastructure solutions.- Migrate existing on-premises workloads to the AWS cloud.- Implement security and compliance controls in the AWS environment.- Monitor and optimize AWS resources usage to ensure cost efficiency.- Automate infrastructure deployment, configuration...

  • AWS Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations/Gurgaon/Gurugram, IN Invitas Full time

    Job Position : AWS-Python- PySpark Engineer Job Type : Full-TimeLocation : GurgaonExperience : 5+ YearsMust Have : - Cloud Platforms (AWS), Automation Tools such as (Jenkins or Ansible)- Python and PySpark, EKS or AKS, Amazon EMR, Amazon Athena- AWS Glue, Amazon Lambda, Amazon EC2 and Amazon's Storage service like S3 and few other services like...

  • AWS Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations/Gurgaon/Gurugram, IN Invitas Full time

    Job Position : AWS-Python- PySpark Engineer Job Type : Full-TimeLocation : GurgaonExperience : 5+ YearsMust Have : - Cloud Platforms (AWS), Automation Tools such as (Jenkins or Ansible)- Python and PySpark, EKS or AKS, Amazon EMR, Amazon Athena- AWS Glue, Amazon Lambda, Amazon EC2 and Amazon's Storage service like S3 and few other services like...