AWS Data Engineer

2 weeks ago


Chennai, Tamil Nadu, India Roshini R (Proprietor of BluePrism Solutions) Full time
Job Description

We are looking for an experienced AWS Data Engineer with 6+ years of experience, including at least 2+ years of hands-on work with Databricks. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines using AWS services and Databricks. You will play a key role in processing large datasets, enabling analytics, and ensuring data security, performance, and reliability.

Key Responsibilities:

Data Pipeline Development:

- Design, develop, and manage scalable ETL/ELT pipelines using AWS services and Databricks.
- Ensure efficient data ingestion, processing, and transformation for analytical and reporting needs.

Data Integration & Transformation:

- Process structured, semi-structured, and unstructured data into AWS Data Lake or Databricks.
- Develop advanced data transformation workflows using PySpark, Databricks SQL, or Scala.

Databricks & AWS Architecture:

- Configure and optimize Databricks clusters, notebooks, and jobs for performance and cost efficiency.
- Design and implement solutions using AWS-native services like S3, Glue, Redshift, EMR, Lambda, Kinesis, and Athena.

Performance Tuning & Optimization:

- Optimize data pipelines, queries, and storage to ensure scalability, cost-effectiveness, and high availability.
- Implement best practices for monitoring, debugging, and troubleshooting issues in cloud-based data pipelines.

Collaboration & Documentation:

- Work closely with Data Analysts, Data Scientists, and Engineers to understand business needs and deliver data-driven solutions.
- Maintain comprehensive documentation for data pipelines, workflows, and architectures.

Security & Monitoring:

- Ensure data security, compliance, and governance across AWS environments.
- Implement monitoring solutions using CloudWatch, Datadog, or equivalent tools to track pipeline health and performance.

Required Skills & Qualifications:

6+ years of experience in Data Engineering, with at least 2+ years of hands-on experience in Databricks.

Strong expertise in AWS services like S3, Glue, Redshift, DynamoDB, Lambda, EMR, and Athena.

Proficiency in Python (PySpark), Scala, or SQL for data processing and transformation.

Extensive experience with Databricks Workspace, Delta Lake, and managing Databricks jobs and pipelines.

Strong knowledge of Apache Spark and Big Data processing frameworks.

Experience with modern data warehousing solutions like Redshift, Snowflake, or Databricks SQL.

Familiarity with Git, Terraform, and CI/CD pipelines for deploying data solutions.

Strong debugging, performance tuning, and monitoring experience using tools like CloudWatch and Datadog.

  • Chennai, Tamil Nadu, India Amazon Web Services (AWS) Full time

    Job DescriptionDescriptionAWS Utility Computing (UC) provides product innovations from foundational services such as Amazons Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWSs services and features apart in the industry. As a member of the UC organization, youll...

  • AWS Data Engineer

    5 days ago


    Chennai, Tamil Nadu, India Risk Resources LLP Full time

    Role : AWS Data EngineerRequired Technical Skill Set : ETL, SQL & AWSDesired Competencies (Technical/Behavioral Competency) :Must-Have :- Strong Knowledge of Aws Glue/AWS REDSHIFt/SQL/ETL.Good knowledge and experience in Pyspark for forming complex Transformation logic.Good-to-Have : PythonResponsibility of / Expectations from the Role :1. Good knowledge of...

  • AWS - Data Engineer

    4 weeks ago


    Chennai, Tamil Nadu, India Hexaware Technologies Full time

    We are looking for AWS Data Engineer for Chennai , Mumbai , Pune & Bangalore.Exp - 6-12 yearsPlease share your details at Saudaminic@hexaware.com.Name:Contact No:Email id :Current Company:Total Exp:Rel Exp in AWS:Notice period:Current CTC:Expected CTC:Current Location:Preferred Location:AWS Data EngineerPrimary Skillset:Proven experience as a Data Engineer...

  • AWS Data Engineer

    1 day ago


    Chennai, Tamil Nadu, India MNR Solutions Pvt Full time

    Role : AWS Data Engineer Experience Required : 6 to 11 Years Location : Bangalore/Chennai Skills Required : - Design, develop, and optimize scalable data pipelines on AWS. - Collaborate with cross-functional teams to collect, process, and analyze large datasets. - Implement and manage ETL processes to ensure data integrity and availability. - Monitor and...

  • AWS - Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India Kavi Software Technologies Private Limited Full time

    Data EngineerWe are seeking an experienced AWS Data Engineer with expertise in Databricks to design, build, and optimize large-scale data pipelines and data processing workflows on the AWS Cloud. The ideal candidate will have hands-on experience working with Databricks, big data technologies, and AWS-native services, ensuring efficient data ingestion,...

  • AWS - Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India Kavi Software Technologies Private Limited Full time

    Job Description Data EngineerWe are seeking an experienced AWS Data Engineer with expertise in Databricks to design, build, and optimize large-scale data pipelines and data processing workflows on the AWS Cloud. The ideal candidate will have hands-on experience working with Databricks, big data technologies, and AWS-native services, ensuring efficient data...

  • AWS Data Engineer

    2 weeks ago


    Chennai, Tamil Nadu, India MNR Solutions Pvt Full time

    Role : AWS Data EngineerExperience Required : 6 to 11 YearsLocation : Bangalore/ChennaiSkills Required : - Design, develop, and optimize scalable data pipelines on AWS.- Collaborate with cross-functional teams to collect, process, and analyze large datasets.- Implement and manage ETL processes to ensure data integrity and availability.- Monitor and improve...

  • AWS Data Engineer

    2 weeks ago


    Chennai, Tamil Nadu, India MNR Solutions Full time

    Currently looking for AWS Data Engineer hiring for a Singapore Based MNC on permanent payroll.Experience : 6+yearsNotice Period : Immediate/Feb serving maxMode : Hybrid ModeLocation : Bangalore/Chennai(No Relocation candidates)Qualification : B.Tech/B.E Or BCA/MCA ( No Civil or Mechanical)Job Description : - Design, develop, and optimize scalable data...

  • AWS Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    Job Title: AWS Data Engineer Experience: 4-8 years of experience in the field. Key Skills: Proficiency in Python programming and Pyspark Data Engineering expertise using AWS core services (Lambda, Glue, EMR, and RedShift) Familiarity with SQL, Airflow, Snowflake, Hive, and AWS S3 Responsibilities: Design, build, and test complex data pipelines (batch and...


  • Chennai, Tamil Nadu, India HCLTech Full time

    Job DescriptionWe are seeking an experienced Hadoop and AWS Data Engineer to join our team at HCLTech. As a key member of our team, you will be responsible for designing, building, and deploying scalable data architectures on AWS.Key Responsibilities:Design and implement data pipelines, storage, and security on AWSLead the migration of Hadoop workloads from...


  • Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    **Job Summary:**We are looking for an experienced AWS Python Data Engineer to join our team at Tata Consultancy Services. As a key member of our data engineering group, you will be responsible for designing and developing scalable data transformation pipelines using Snowflake, AWS, and Python.The ideal candidate will have strong experience in Snowflake, AWS,...


  • Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    Role: AWS Data EngineerJOB LOCATION : Chennai, Indore , PuneEXPERIENCE REQUIREMENT : 4+Required Technical Skill:Strong Knowledge of Aws Glue/AWS REDSHIFT/SQL/ETL. Good knowledge and experience in Pyspark for forming complex Transformation logic.AWS Data Engineer,SQL,ETL, DWH , Secondary : AWS Glue , AirflowMust-HaveGood Knowledge of SQL , ETLA minimum of 3 +...


  • Chennai, Tamil Nadu, India Giggso Full time

    **Job Summary**We are seeking a seasoned Cloud Data Engineer to join our team at Giggso. As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing data models and solutions for various business use cases. Your expertise in AWS, Snowflake, and SQL will enable you to create efficient data analysis processes...


  • Chennai, Tamil Nadu, India Coders Brain Technology Private Limited Full time

    About UsCoders Brain Technology Private Limited is a forward-thinking technology company in the fast-evolving cybersecurity field. We are committed to innovation and continuous learning and development. As a key member of our data engineering team, you will play a vital role in shaping our data architecture and driving impactful data solutions.Job...


  • Chennai, Tamil Nadu, India Amazon Web Services (AWS) Full time

    Job DescriptionDescriptionAWS Utility Computing (UC) provides product innovations from foundational services such as Amazons Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWSs services and features apart in the industry. As a member of the UC organization, youll...

  • Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India NTT DATA Full time

    Job Description Req ID: 303790 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Job Duties: As a data...

  • AWS Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India CBTS Full time

    Job Description Skills : Technical: solid understanding of Data Management, AWS tech stack like Python, Glue, Athena, Redshift, Lambda, data security, data governance, and privacy best practices to your AWS data pipelines Non-Tech: Solid communication, data design skills


  • Chennai, Tamil Nadu, India Millims Full time

    Experience: 6+ years in data engineering with a focus on AWS cloud technologies. Technical Skills: o Expertise in AWS Glue, Lambda, EMR, Redshift, Kinesis, and Step Functions. o Proficiency in SQL, Python, and PySpark for data transformations. o Strong understanding of ETL/ELT best practices and data warehousing concepts. o Experience with Apache Airflow...


  • Chennai, Tamil Nadu, India HCLTech Full time

    Job OverviewHCLTech is seeking highly skilled professionals to lead our efforts in Hadoop and AWS data engineering. We are looking for individuals with expertise in designing and implementing data architectures on AWS, including data pipelines, storage, and security.

  • AWS Data Engineer

    1 day ago


    Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    GreetingsTCS is now HiringJob Role- AWS Data EngineerExperience range- 5 to 8Job Location- ChennaiDesired Competencies:Involved in End to End implementations right from requirements gathering, business process analysis.Be able to work with the business to understand their needs, coach them on the appropriate solutions while at the same time work with...