Data Engineer

4 weeks ago


india Huquo Full time

Position : AWS Data Engineer

Location : Pune/ Gurgaon/ Hybrid

Experience : 5+

Job Description :

- 5+ years of experience as a Data Engineer on the AWS Stack

- AWS Solutions Architect or AWS Developer Certification required

- Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, EMR/Spark, RDS, Redshift, Data Sync, DMS, DynamoDB, Lambda, Step Functions, IAM, KMS, SM, Event Bridge, EC2, SQS, SNS, Lake Formation, Cloud Watch, Cloud Trail

- Programming experience with Python, Shell scripting, and SQL

- Responsible for building, test, QA & UAT environments using Cloud Formation.

- Build & implement CI/CD pipelines for the EDP Platform using Cloud Formation and Jenkins

Good to Have :

- Implement high-velocity streaming solutions and orchestration using Amazon Kinesis, AWS Managed Airflow, and AWS Managed Kafka (preferred)

- Solid experience building solutions on AWS data lake/data warehouse

- Analyze, design, Development , and implement data ingestion pipeline in AWS

- Knowledge of implementing ETL/ELT for data solutions end to end

- ingest data from Rest APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift.

- Perform the Peer Code Review and, perform code quality analysis, and associated tools end-to-end for Prudential's platforms

- Create detailed, comprehensive, and well-structured test cases that follow best practices and techniques

- Estimate, prioritize, plan & coordinate quality testing activities

- Understanding requirements, and data solutions (ingest, storage, integration, processing, access) on AWS

- Knowledge of implementing RBAC strategy/solutions using AWS IAM and Redshift RBAC model

- Knowledge of analyzing data using SQL Stored procedures

- Build automated data pipelines to ingest data from relational database systems, file system , and NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift

- Build Automated data pipelines to develop test plans, execute manual and automated test cases, help to identify the root causes, and articulate defects clearly

- Recreate production issues to help determine the issue and verify any fixes

- Conducting End to End verification and validation for the entire application

- Creating Jenkins CI pipelines to integrate Sonar/Security scans and test automation scripts

- Using Git/bitbucket for efficient remote team working, storing framework, and developing test scripts

- Part of DevOps QA and AWS team focusing on building CI/CD pipeline

- Part of the release/build team and mainly worked on release management, CI/CD pipeline

- Deploy multiple instances by using cloud formation templates

Responsibilities :

- Designing, building, and maintaining efficient, reusable, and reliable code

- Ensure the best possible performance and quality of high-scale data applications and services

- Participate in system design discussions

- Independently perform hands-on Development and unit testing of the applications

- Collaborate with the development team and build individual components into the enterprise data platform

- Work in a team environment with the product, QE/QA, and cross-functional teams to deliver a project throughout the whole software development cycle.

- Responsible to identify and resolve any performance issues

- Keep up to date with new technology development and implementation

- Participate in code review to make sure standards and best practices are met

- Project management: Agile developers take responsibility for estimating, planning, and managing all tasks and report on progress

- Software quality: The Agile developer is also responsible for the quality of the software he/she produces. The team takes responsibility for the quality of the work they produce instead of turning over some code to a separate and independent group for testing

- Teamwork: This includes collaboration with all other team members with the aim to take shared responsibility for the overall efforts

- Understanding user needs: This is about interacting with users as necessary to clarify requirements

Education :

Minimum of 15 years of formal education - BE/BTech or Graduate / Postgraduate in Computer Science / Information Technology

Every individual comes with a different set of skills and qualities so even if you don't tick all the boxes for the role today, we urge you to apply as there might be a suitable/unique role for you tomorrow

(ref:hirist.tech)
  • Operations Specialist

    1 month ago


    India Data Engineer Academy Full time

    Job Title: Operations Analyst Location : RemoteCompany Overview:Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary to...

  • Operations Specialist

    4 weeks ago


    india Data Engineer Academy Full time

    Job Title: Operations Analyst Location : Remote Company Overview: Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary...

  • Operations Specialist

    4 weeks ago


    india Data Engineer Academy Full time

    Job Title: Operations Analyst Location : Remote Company Overview: Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary...


  • India Data Engineer Academy Full time

    Job Title: Operations Analyst Location : Remote Company Overview: Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary...


  • india Lapis Data Analytics Full time

    Company Description Lapis Data Analytics is a dynamic and innovative data engineering company founded in 2020 and located in Chennai. Our primary focus is on designing, building, and maintaining robust data pipelines that convert raw data into actionable insights. With a team of seasoned experts and cutting-edge technology, we deliver customized data...


  • india K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...

  • Data Architect

    6 days ago


    India Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Aptus Data Labs

    1 month ago


    india Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • MLOps Engineer

    1 month ago


    india GOOGNU DATA SERVICES PRIVATE LIMITED Full time

    Job Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...

  • QA Engineer

    1 month ago


    india Data Unveil Full time

    Position Title: QA Engineer Level: Mid-Level Location: Hyderabad, Telangana. Hire Type: Full Time Start Date: Immediate Minimum 3+ years of QA experience in data projects. Collaborate with the Development/Business teams to understand the requirements. Analyzing Business Requirements/cases and developing/managing the positive/negative test scenarios cases....

  • Senior Engineer

    12 hours ago


    india NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at...

  • Tableau Developer

    3 days ago


    India The Data Family Full time

    Company Description The Data Family offers analytics and data visualization services designed to transform data into actionable insights for our clients. Through our service, customers can access expert analytics, robust data engineering, and captivating visualizations to drive better decision-making and uncover opportunities for growth. Our primary...

  • Tableau Developer

    3 days ago


    India The Data Family Full time

    Company DescriptionThe Data Family offers analytics and data visualization services designed to transform data into actionable insights for our clients. Through our service, customers can access expert analytics, robust data engineering, and captivating visualizations to drive better decision-making and uncover opportunities for growth. Our primary customers...


  • India NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...

  • Data Engineer

    4 hours ago


    India krtrimaIQ Cognitive Solutions Full time

    Company DescriptionkrtrimaIQ Cognitive Solutions is a technology company specializing in Cognitive Process Automation, Data Engineering, Statistical Analysis and Optimization, Data Science, Natural Language Processing, and Computer Vision. Our expertise allows us to transform organizations into intelligent enterprises.Role DescriptionThis is a full-time...

  • Data Engineer

    6 days ago


    India Paralleldots Full time

    Job Description : As a Senior Data Engineer at ParallelDots, you will be responsible for overseeing the design and development of advanced data solutions.You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure meets the highest standards of scalability, reliability, and : Lead the design,...

  • Data Engineer

    6 days ago


    India Perfecthire Full time

    Job Description : We are seeking a talented and experienced Data Engineer to join our team and play a key role in the development and implementation of our end-to-end 834 EDI solution. A robust level of experience or familiarity with 834 EDI (Electronic Data Interchange) transactions is critical for this role. As a Data Engineer, you will be responsible...

  • Data Engineer

    2 months ago


    india HTC Global Services Full time

    Position Description: • This role is for experienced Data Engineer that will be responsible for designing and building the foundational components required for our customer. This includes bringing disparate sources into our Data Factory and creating data products that will be used by the analytics and reporting teams. • Data engineers will work in small,...


  • india NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today’s ‘iNTTerconnected’ world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help...

  • Data Engineer

    6 days ago


    India Lingaro Full time

    Role: Data Engineer – Lead ConsultantLocation: IndiaAbout Lingaro:Lingaro Group is the end-to-end data services partner to global brands and enterprises. We lead our clients through their data journey, from strategy through development to operations and adoption, helping them to realize the full value of their data.Since 2008, Lingaro has been recognized...