AWS Senior Data Engineer 6 to 7 years

23 hours ago

Chennai, Tamil Nadu, India connektitude pvt. ltd. Full time ₹ 16,00,000 - ₹ 18,00,000 per year

Job Description:

We are seeking an experienced AWS Data Engineer with a strong background in data processing, cloud technologies, and distributed computing. In this role, you will design, develop, and maintain scalable data pipelines, integrating various data sources, and enabling efficient data processing workflows using Python, PySpark, SQL, and AWS cloud-based services. The ideal candidate will also have exposure to Terraform and experience or familiarity with Snowflake and DBT.

Responsibilities:

Data Pipeline Development: Design, implement, and maintain scalable ETL pipelines using AWS Glue, PySpark, and Python to ingest, process, and transform data from multiple sources.

AWS Cloud Solutions: Utilize AWS services such as Lambda, S3, Athena, Glue, DynamoDB, and Step Functions to build and manage data workflows, ensuring robust and efficient data processing.

Data Transformation and Storage: Develop solutions for data transformation and ensure proper storage and retrieval using AWS S3, DynamoDB, and other related services.

Terraform Integration: Leverage Terraform to manage and provision AWS infrastructure as code, ensuring a streamlined deployment process.

Data Modeling and SQL Development: Write, optimize, and maintain complex SQL queries for data extraction, transformation, and loading (ETL) processes. Ensure data quality, integrity, and accuracy across transformations.

Collaboration and Communication: Work closely with data engineers, architects, analysts, and other stakeholders to understand data requirements and translate them into scalable data solutions.

Performance Optimization: Analyze and optimize PySpark and SQL scripts for better performance, ensuring smooth and efficient data processing at scale.

Documentation: Create and maintain detailed documentation of data engineering workflows, codebase, and infrastructure configurations.

Troubleshooting: Diagnose and resolve data-related issues, bottlenecks, and performance challenges in a timely manner.

Continuous Improvement: Stay current with emerging data technologies, tools, and best practices to continuously enhance data solutions and processes.

Required Skills and Qualifications:

Proficiency in Python and PySpark: Strong experience writing robust Python code and utilizing PySpark for distributed data processing, transformation, and working with large datasets.

AWS Data Engineering Expertise: Hands-on experience with AWS Glue, Lambda, S3, Athena, DynamoDB, and Step Functions to build and manage data workflows.

SQL Mastery: Deep knowledge of SQL for complex query writing, database design, and performance optimization.

Terraform Experience: Familiarity with Terraform for managing AWS infrastructure as code is essential.

Cloud Data Architecture: Experience designing and managing data solutions on AWS cloud platforms.

Problem-Solving Skills: Strong troubleshooting skills for data pipeline issues, optimization challenges, and system bottlenecks.

Preferred Qualifications:

Experience with Snowflake: Familiarity with Snowflake for data warehousing and integration with AWS.

DBT Knowledge: Experience with DBT for data transformations, testing, and modeling is a strong plus.

Collaboration and Communication: Strong interpersonal and collaboration skills for working in cross-functional teams.

Version Control: Proficiency with version control systems like Git for managing code changes, reviews, and collaboration.

Job Type: Contractual / Temporary

Pay: ₹1,600, ₹1,800,000.00 per year

Work Location: In person

  • AWS Data Engineer

    21 hours ago

    Chennai, Tamil Nadu, India Crescent Techservices Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role: AWS Data EngineerType: 6-12 month contractLocation: Chennai, Tamil Nadu (WFO)Experience: 6-8 YearsRequirements:Proficiency in Python and PySpark: Strong experience writing robust Python code and utilizing PySpark for distributed data processing, transformation, and working with large datasets.AWS Data Engineering Expertise: Hands-on experience with...

  • AWS Data Architect

    2 weeks ago

    Chennai, Tamil Nadu, India HCLTech Full time

    Dear Candidate, Greetings from HCL Tech We are hiring AWS requirement Kinldy share your availability If your resume is shortlisted we will schedule your interview accordingly. Senior AWS Data Engineer We are seeking a highly skilled Senior AWS Data Engineer to join our data engineering team. The ideal candidate will have deep expertise in building...

  • Aws Data Engineer- Chennai

    3 weeks ago

    Chennai, Tamil Nadu, India Diverse Lynx Full time

    HIRING FOR AWS DATA ENGINEER- CHENNAI Job Summary We are looking for a highly skilled AWS Data Engineer to join our growing data team In this role you will be responsible for designing building and maintaining scalable and reliable data pipelines and architectures on AWS to support analytics machine learning and business intelligence initiatives ...

  • Senior Data Engineering Expert

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking a highly skilled Senior Data Engineer to join our data engineering team. The ideal candidate will have deep expertise in building scalable data pipelines using Apache Spark, PySpark, SQL, and Python, along with hands-on experience in the AWS ecosystem.Key Responsibilities:Design, develop, and maintain robust ETL/ELT pipelines on AWS using...

  • AWS DevOps Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Talent Pro Full time

    Job Description : AWS DevOps EngineerRole : As an AWS DevOps Engineer, you will be responsible for designing, implementing, and managing the cloud infrastructure using Amazon Web Services. You will work closely with development and operations teams to automate manual tasks, improve system performance, and ensure the reliability and scalability of the system....

  • Lead Data Engineer

    23 hours ago

    Chennai, Tamil Nadu, India AABM Cloud Data Solutions Full time ₹ 12,00,000 - ₹ 18,00,000 per year

    About the RoleWe are seeking a Lead Data Engineer to join our team. In this role, you will be crucial in designing, building, and maintaining our data infrastructure. This is a hands-on position requiring a strong foundation in data warehousing, ETL processes, and database management. The ideal candidate will have a keen eye for detail and the ability to...

  • Senior Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India SecureKloud Technologies Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    About The RoleWe are seeking a highly skilled Senior Data Engineer to join our dynamic team. In this role, you will be instrumental in designing, building, and maintaining scalable, efficient, and reliable data pipelines and solutions. You will work closely with cross-functional teams to enable data-driven decision-making by ensuring the availability,...

  • Senior AWS Data Engineer

    22 hours ago

    Chennai, Tamil Nadu, India Virtusa Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    AWS Data Engineer Design and build scalable data pipelines using AWS services like AWS Glue, Amazon Redshift, and QS/SNS/Cloudwatch/Step function/CDK.Develop efficient ETL processes for data extraction, transformation, and loading into data warehouses and lakes.Create and manage applications using Python, Pyspark, SQL, Databricks, and various AWS...

  • Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India LTIMindtree Full time

    We are hiring for AWS Data EngineerExperience - 5 to 7 YearsLocation : ChennaiNotice period - immediate to 30 daysJob SummarySeeking a Specialist with 5 to 7 years of experience in AWS IoT Data Engineering and strong Python skills to design and implement scalable IoT data solutions4 years of experience in GCP BQ Dataflow Airflow and Python Should have...

  • AmpsTek - AWS Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India Ampstek Full time

    Job Role : AWS Data Location : Chennai/Bangalore/Hyderabad/Pune (Hybrid)Key Skills : AWS Glue, RedShift, S3, Lambda Athena.We are seeking a highly skilled AWS Data Engineer with 5 - 8 years of experience in designing, developing, and deploying cloud-based data solutions. The role involves building scalable ETL pipelines, data warehouses, and data...