AWS - Data Engineer

3 weeks ago


Chennai, Tamil Nadu, India Kavi Software Technologies Private Limited Full time
Job Description

Data Engineer

We are seeking an experienced AWS Data Engineer with expertise in Databricks to design, build, and optimize large-scale data pipelines and data processing workflows on the AWS Cloud. The ideal candidate will have hands-on experience working with Databricks, big data technologies, and AWS-native services, ensuring efficient data ingestion, transformation, and analytics to support business-critical decisions.

Key Responsibilities:

  • Data Pipeline Development: Design, implement, and manage scalable ETL/ELT pipelines using AWS services and Databricks.
  • Data Integration: Ingest and process structured, semi-structured, and unstructured data from multiple sources into AWS Data Lake or Databricks.
  • Data Transformation: Develop advanced data processing workflows using PySpark, Databricks SQL, or Scala to enable analytics and reporting.
  • Databricks Management: Configure and optimize Databricks clusters, notebooks, and jobs for performance and cost efficiency.
  • AWS Architecture: Design and implement solutions leveraging AWS-native services like S3, Glue, Redshift, EMR, Lambda, Kinesis, and Athena.
  • Collaboration: Work closely with Data Analysts, Data Scientists, and other Engineers to understand business requirements and deliver data-driven solutions.
  • Performance Tuning: Optimize data pipelines, storage, and queries for performance, scalability, and reliability.
  • Monitoring and Security: Ensure data pipelines are secure, robust, and monitored using CloudWatch, Datadog, or equivalent tools.
  • Documentation: Maintain clear and concise documentation for data pipelines, workflows, and architecture.

Required Skills & Qualifications:

  • Data Engineering Expertise: 4+ years of experience in data engineering with at least 2+ years working on Databricks.
  • AWS Cloud Services: Hands-on experience with AWS ecosystem, including S3, Glue, Redshift, DynamoDB, Lambda, and other AWS data services.
  • Programming Languages: Proficiency in Python (PySpark), Scala, or SQL for data processing and transformation.
  • Databricks: Extensive experience with Databricks Workspace, Delta Lake, and managing databricks jobs and pipelines.
  • Big Data Frameworks: Strong knowledge of Apache Spark for distributed data processing.
  • Data Warehousing: Experience with modern data warehouse solutions, including Redshift, Snowflake, or Databricks SQL.
  • Version Control & CI/CD: Familiarity with Git, Terraform, and CI/CD pipelines for deploying data solutions.
  • Monitoring & Debugging: Experience with tools like CloudWatch, Datadog, or equivalent for pipeline monitoring and troubleshooting.

Preferred Qualifications:

  • Certification in AWS Data Analytics or Databricks.
  • Experience with real-time data streaming tools like Kafka, Kinesis, or AWS MSK.
  • Knowledge of data governance and data security best practices.
  • Exposure to machine learning workflows and integration with Databricks


Requirements


  • Chennai, Tamil Nadu, India Amazon Web Services (AWS) Full time

    Job DescriptionDescriptionAWS Utility Computing (UC) provides product innovations from foundational services such as Amazons Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWSs services and features apart in the industry. As a member of the UC organization, youll...

  • AWS Data Engineer

    6 days ago


    Chennai, Tamil Nadu, India Risk Resources LLP Full time

    Role : AWS Data EngineerRequired Technical Skill Set : ETL, SQL & AWSDesired Competencies (Technical/Behavioral Competency) :Must-Have :- Strong Knowledge of Aws Glue/AWS REDSHIFt/SQL/ETL.Good knowledge and experience in Pyspark for forming complex Transformation logic.Good-to-Have : PythonResponsibility of / Expectations from the Role :1. Good knowledge of...

  • AWS - Data Engineer

    4 weeks ago


    Chennai, Tamil Nadu, India Hexaware Technologies Full time

    We are looking for AWS Data Engineer for Chennai , Mumbai , Pune & Bangalore.Exp - 6-12 yearsPlease share your details at Saudaminic@hexaware.com.Name:Contact No:Email id :Current Company:Total Exp:Rel Exp in AWS:Notice period:Current CTC:Expected CTC:Current Location:Preferred Location:AWS Data EngineerPrimary Skillset:Proven experience as a Data Engineer...

  • AWS Data Engineer

    3 days ago


    Chennai, Tamil Nadu, India MNR Solutions Pvt Full time

    Role : AWS Data Engineer Experience Required : 6 to 11 Years Location : Bangalore/Chennai Skills Required : - Design, develop, and optimize scalable data pipelines on AWS. - Collaborate with cross-functional teams to collect, process, and analyze large datasets. - Implement and manage ETL processes to ensure data integrity and availability. - Monitor and...

  • AWS - Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India Kavi Software Technologies Private Limited Full time

    Data EngineerWe are seeking an experienced AWS Data Engineer with expertise in Databricks to design, build, and optimize large-scale data pipelines and data processing workflows on the AWS Cloud. The ideal candidate will have hands-on experience working with Databricks, big data technologies, and AWS-native services, ensuring efficient data ingestion,...

  • AWS Data Engineer

    2 weeks ago


    Chennai, Tamil Nadu, India MNR Solutions Pvt Full time

    Role : AWS Data EngineerExperience Required : 6 to 11 YearsLocation : Bangalore/ChennaiSkills Required : - Design, develop, and optimize scalable data pipelines on AWS.- Collaborate with cross-functional teams to collect, process, and analyze large datasets.- Implement and manage ETL processes to ensure data integrity and availability.- Monitor and improve...

  • AWS Data Engineer

    2 weeks ago


    Chennai, Tamil Nadu, India Roshini R (Proprietor of BluePrism Solutions) Full time

    Job DescriptionWe are looking for an experienced AWS Data Engineer with 6+ years of experience, including at least 2+ years of hands-on work with Databricks. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines using AWS services and Databricks. You will play a key role in processing large datasets, enabling...

  • AWS Data Engineer

    2 weeks ago


    Chennai, Tamil Nadu, India MNR Solutions Full time

    Currently looking for AWS Data Engineer hiring for a Singapore Based MNC on permanent payroll.Experience : 6+yearsNotice Period : Immediate/Feb serving maxMode : Hybrid ModeLocation : Bangalore/Chennai(No Relocation candidates)Qualification : B.Tech/B.E Or BCA/MCA ( No Civil or Mechanical)Job Description : - Design, develop, and optimize scalable data...

  • AWS Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    Job Title: AWS Data Engineer Experience: 4-8 years of experience in the field. Key Skills: Proficiency in Python programming and Pyspark Data Engineering expertise using AWS core services (Lambda, Glue, EMR, and RedShift) Familiarity with SQL, Airflow, Snowflake, Hive, and AWS S3 Responsibilities: Design, build, and test complex data pipelines (batch and...


  • Chennai, Tamil Nadu, India HCLTech Full time

    Job DescriptionWe are seeking an experienced Hadoop and AWS Data Engineer to join our team at HCLTech. As a key member of our team, you will be responsible for designing, building, and deploying scalable data architectures on AWS.Key Responsibilities:Design and implement data pipelines, storage, and security on AWSLead the migration of Hadoop workloads from...


  • Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    **Job Summary:**We are looking for an experienced AWS Python Data Engineer to join our team at Tata Consultancy Services. As a key member of our data engineering group, you will be responsible for designing and developing scalable data transformation pipelines using Snowflake, AWS, and Python.The ideal candidate will have strong experience in Snowflake, AWS,...


  • Chennai, Tamil Nadu, India Giggso Full time

    **Job Summary**We are seeking a seasoned Cloud Data Engineer to join our team at Giggso. As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing data models and solutions for various business use cases. Your expertise in AWS, Snowflake, and SQL will enable you to create efficient data analysis processes...


  • Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    Role: AWS Data EngineerJOB LOCATION : Chennai, Indore , PuneEXPERIENCE REQUIREMENT : 4+Required Technical Skill:Strong Knowledge of Aws Glue/AWS REDSHIFT/SQL/ETL. Good knowledge and experience in Pyspark for forming complex Transformation logic.AWS Data Engineer,SQL,ETL, DWH , Secondary : AWS Glue , AirflowMust-HaveGood Knowledge of SQL , ETLA minimum of 3 +...


  • Chennai, Tamil Nadu, India Amazon Web Services (AWS) Full time

    Job DescriptionDescriptionAWS Utility Computing (UC) provides product innovations from foundational services such as Amazons Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWSs services and features apart in the industry. As a member of the UC organization, youll...

  • AWS Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India CBTS Full time

    Job Description Skills : Technical: solid understanding of Data Management, AWS tech stack like Python, Glue, Athena, Redshift, Lambda, data security, data governance, and privacy best practices to your AWS data pipelines Non-Tech: Solid communication, data design skills

  • Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India NTT DATA Full time

    Job Description Req ID: 303790 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Job Duties: As a data...


  • Chennai, Tamil Nadu, India Millims Full time

    Experience: 6+ years in data engineering with a focus on AWS cloud technologies. Technical Skills: o Expertise in AWS Glue, Lambda, EMR, Redshift, Kinesis, and Step Functions. o Proficiency in SQL, Python, and PySpark for data transformations. o Strong understanding of ETL/ELT best practices and data warehousing concepts. o Experience with Apache Airflow...


  • Chennai, Tamil Nadu, India HCLTech Full time

    Job OverviewHCLTech is seeking highly skilled professionals to lead our efforts in Hadoop and AWS data engineering. We are looking for individuals with expertise in designing and implementing data architectures on AWS, including data pipelines, storage, and security.


  • Chennai, Tamil Nadu, India HCLTech Full time

    About the Role:We are seeking a seasoned Hadoop and AWS data engineer to lead the migration of our Hadoop workloads from on-premise to AWS-EMR stack. The ideal candidate will have 5+ years of experience in designing and implementing data architectures on AWS, including data pipelines, storage, and security.The selected individual will collaborate with...


  • Chennai, Tamil Nadu, India NTT DATA Full time

    About the Job: NTT Data is looking for an experienced Full Stack Engineer who can lead the development of complex software systems using Java 8, REST, AWS, and Kafka. The ideal candidate will have a strong background in full-stack development and experience working in an Agile environment.Responsibilities: The selected candidate will be responsible...