AWS Data Engineer

Data Engineer Duties & Responsibilities

• Design and implement scalable data pipelines to process and transform large volumes of data using AWS services such as Glue, Kafka, S3, EMR, Athena.
• Collaborate with cross-functional teams including data scientists, analysts, and software engineers to understand data requirements and deliver solutions that meet business needs.
• Optimize data workflows for performance, reliability, and cost-efficiency on AWS cloud platform. Performance Tune ETL Pipelines, database queries.
• Develop and maintain ETL processes to ensure data quality and integrity throughout the data lifecycle.
• Implement best practices for data security, access control, and compliance in accordance with industry standards and regulations.
• Continuously monitor and troubleshoot data pipelines to identify and resolve issues in a timely manner.
• Stay updated with the latest AWS services and technologies and evaluate their potential for improving our data infrastructure.
• Document data architecture, design decisions, and technical specifications for reference and knowledge sharing.

Data Engineer – Experience & Qualifications:

• Design and implement scalable data pipelines to process and transform large volumes of data using AWS services such as Glue, Kafka, S3, EMR, Athena.
• Collaborate with cross-functional teams including data scientists, analysts, and software

engineers to understand data requirements and deliver solutions that meet business needs.

• Optimize data workflows for performance, reliability, and cost-efficiency on AWS cloud

platform. Performance Tune ETL Pipelines, database queries.

• Develop and maintain ETL processes to ensure data quality and integrity throughout the data lifecycle.
• Implement best practices for data security, access control, and compliance in accordance with industry standards and regulations.
• Continuously monitor and troubleshoot data pipelines to identify and resolve issues in a timely manner.
• Stay updated with the latest AWS services and technologies and evaluate their potential for improving our data infrastructure.
• Document data architecture, design decisions, and technical specifications for reference and knowledge sharing.

Data Engineer – Educational Qualifications & Skills

• Bachelor or Masters degree in Computer Science Engineering, or related field.
• Proven experience as a Data Engineer or similar role, with at least 2 years of experience in building and maintaining data infrastructure on AWS cloud platform.
• Strong proficiency in AWS services such as Glue, Redshift, Lambda, Athena, S3,QuickSight etc.
• Experience with big data technologies such as Hadoop, Spark, Kafka, etc.
• In-depth knowledge of data architecture principles, data modelling, and database design.
• Hands-on experience with SQL, Python, and any other programming languages for data manipulation and scripting.
• Familiarity with DevOps practices and tools for CI/CD, infrastructure as code, and automated deployment.
• Knowledge of machine learning concepts and techniques for data analysis and predictive modelling.
• Excellent problem-solving skills and attention to detail, with a focus on delivering high-quality solutions.
• Ability to work effectively in a fast-paced environment with tight deadlines and changing priorities.
• Strong communication and collaboration skills, with the ability to work effectively in a cross-functional team environment

Job Types: Full-time, Permanent

Pay: ₹1,200, ₹1,400,000.00 per year

Schedule:

• Day shift

Experience:

• Kafka: 3 years (Required)
• AWS: 3 years (Required)
• ETL: 3 years (Required)
• Spark: 3 years (Required)

Work Location: Remote