AWS - Data Engineer
3 weeks ago
Data Engineer
We are seeking an experienced AWS Data Engineer with expertise in Databricks to design, build, and optimize large-scale data pipelines and data processing workflows on the AWS Cloud. The ideal candidate will have hands-on experience working with Databricks, big data technologies, and AWS-native services, ensuring efficient data ingestion, transformation, and analytics to support business-critical decisions.
Key Responsibilities:
- Data Pipeline Development: Design, implement, and manage scalable ETL/ELT pipelines using AWS services and Databricks.
- Data Integration: Ingest and process structured, semi-structured, and unstructured data from multiple sources into AWS Data Lake or Databricks.
- Data Transformation: Develop advanced data processing workflows using PySpark, Databricks SQL, or Scala to enable analytics and reporting.
- Databricks Management: Configure and optimize Databricks clusters, notebooks, and jobs for performance and cost efficiency.
- AWS Architecture: Design and implement solutions leveraging AWS-native services like S3, Glue, Redshift, EMR, Lambda, Kinesis, and Athena.
- Collaboration: Work closely with Data Analysts, Data Scientists, and other Engineers to understand business requirements and deliver data-driven solutions.
- Performance Tuning: Optimize data pipelines, storage, and queries for performance, scalability, and reliability.
- Monitoring and Security: Ensure data pipelines are secure, robust, and monitored using CloudWatch, Datadog, or equivalent tools.
- Documentation: Maintain clear and concise documentation for data pipelines, workflows, and architecture.
Required Skills & Qualifications:
- Data Engineering Expertise: 4+ years of experience in data engineering with at least 2+ years working on Databricks.
- AWS Cloud Services: Hands-on experience with AWS ecosystem, including S3, Glue, Redshift, DynamoDB, Lambda, and other AWS data services.
- Programming Languages: Proficiency in Python (PySpark), Scala, or SQL for data processing and transformation.
- Databricks: Extensive experience with Databricks Workspace, Delta Lake, and managing databricks jobs and pipelines.
- Big Data Frameworks: Strong knowledge of Apache Spark for distributed data processing.
- Data Warehousing: Experience with modern data warehouse solutions, including Redshift, Snowflake, or Databricks SQL.
- Version Control & CI/CD: Familiarity with Git, Terraform, and CI/CD pipelines for deploying data solutions.
- Monitoring & Debugging: Experience with tools like CloudWatch, Datadog, or equivalent for pipeline monitoring and troubleshooting.
Preferred Qualifications:
- Certification in AWS Data Analytics or Databricks.
- Experience with real-time data streaming tools like Kafka, Kinesis, or AWS MSK.
- Knowledge of data governance and data security best practices.
- Exposure to machine learning workflows and integration with Databricks
Requirements
-
SysDev Engineer II, AWS Bedrock
4 weeks ago
Chennai, Tamil Nadu, India Amazon Web Services (AWS) Full timeJob DescriptionDescriptionAWS Utility Computing (UC) provides product innovations from foundational services such as Amazons Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWSs services and features apart in the industry. As a member of the UC organization, youll...
-
AWS Data Engineer
6 days ago
Chennai, Tamil Nadu, India Risk Resources LLP Full timeRole : AWS Data EngineerRequired Technical Skill Set : ETL, SQL & AWSDesired Competencies (Technical/Behavioral Competency) :Must-Have :- Strong Knowledge of Aws Glue/AWS REDSHIFt/SQL/ETL.Good knowledge and experience in Pyspark for forming complex Transformation logic.Good-to-Have : PythonResponsibility of / Expectations from the Role :1. Good knowledge of...
-
AWS - Data Engineer
4 weeks ago
Chennai, Tamil Nadu, India Hexaware Technologies Full timeWe are looking for AWS Data Engineer for Chennai , Mumbai , Pune & Bangalore.Exp - 6-12 yearsPlease share your details at Saudaminic@hexaware.com.Name:Contact No:Email id :Current Company:Total Exp:Rel Exp in AWS:Notice period:Current CTC:Expected CTC:Current Location:Preferred Location:AWS Data EngineerPrimary Skillset:Proven experience as a Data Engineer...
-
AWS Data Engineer
3 days ago
Chennai, Tamil Nadu, India MNR Solutions Pvt Full timeRole : AWS Data Engineer Experience Required : 6 to 11 Years Location : Bangalore/Chennai Skills Required : - Design, develop, and optimize scalable data pipelines on AWS. - Collaborate with cross-functional teams to collect, process, and analyze large datasets. - Implement and manage ETL processes to ensure data integrity and availability. - Monitor and...
-
AWS - Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India Kavi Software Technologies Private Limited Full timeData EngineerWe are seeking an experienced AWS Data Engineer with expertise in Databricks to design, build, and optimize large-scale data pipelines and data processing workflows on the AWS Cloud. The ideal candidate will have hands-on experience working with Databricks, big data technologies, and AWS-native services, ensuring efficient data ingestion,...
-
AWS Data Engineer
2 weeks ago
Chennai, Tamil Nadu, India MNR Solutions Pvt Full timeRole : AWS Data EngineerExperience Required : 6 to 11 YearsLocation : Bangalore/ChennaiSkills Required : - Design, develop, and optimize scalable data pipelines on AWS.- Collaborate with cross-functional teams to collect, process, and analyze large datasets.- Implement and manage ETL processes to ensure data integrity and availability.- Monitor and improve...
-
AWS Data Engineer
2 weeks ago
Chennai, Tamil Nadu, India Roshini R (Proprietor of BluePrism Solutions) Full timeJob DescriptionWe are looking for an experienced AWS Data Engineer with 6+ years of experience, including at least 2+ years of hands-on work with Databricks. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines using AWS services and Databricks. You will play a key role in processing large datasets, enabling...
-
AWS Data Engineer
2 weeks ago
Chennai, Tamil Nadu, India MNR Solutions Full timeCurrently looking for AWS Data Engineer hiring for a Singapore Based MNC on permanent payroll.Experience : 6+yearsNotice Period : Immediate/Feb serving maxMode : Hybrid ModeLocation : Bangalore/Chennai(No Relocation candidates)Qualification : B.Tech/B.E Or BCA/MCA ( No Civil or Mechanical)Job Description : - Design, develop, and optimize scalable data...
-
AWS Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India Tata Consultancy Services Full timeJob Title: AWS Data Engineer Experience: 4-8 years of experience in the field. Key Skills: Proficiency in Python programming and Pyspark Data Engineering expertise using AWS core services (Lambda, Glue, EMR, and RedShift) Familiarity with SQL, Airflow, Snowflake, Hive, and AWS S3 Responsibilities: Design, build, and test complex data pipelines (batch and...
-
Hadoop and AWS Data Engineer
7 days ago
Chennai, Tamil Nadu, India HCLTech Full timeJob DescriptionWe are seeking an experienced Hadoop and AWS Data Engineer to join our team at HCLTech. As a key member of our team, you will be responsible for designing, building, and deploying scalable data architectures on AWS.Key Responsibilities:Design and implement data pipelines, storage, and security on AWSLead the migration of Hadoop workloads from...
-
AWS Python Data Engineer
3 days ago
Chennai, Tamil Nadu, India Tata Consultancy Services Full time**Job Summary:**We are looking for an experienced AWS Python Data Engineer to join our team at Tata Consultancy Services. As a key member of our data engineering group, you will be responsible for designing and developing scalable data transformation pipelines using Snowflake, AWS, and Python.The ideal candidate will have strong experience in Snowflake, AWS,...
-
AWS Data Engineering Lead
2 days ago
Chennai, Tamil Nadu, India Giggso Full time**Job Summary**We are seeking a seasoned Cloud Data Engineer to join our team at Giggso. As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing data models and solutions for various business use cases. Your expertise in AWS, Snowflake, and SQL will enable you to create efficient data analysis processes...
-
AWS Data Engineer(Chennai, Indore
2 weeks ago
Chennai, Tamil Nadu, India Tata Consultancy Services Full timeRole: AWS Data EngineerJOB LOCATION : Chennai, Indore , PuneEXPERIENCE REQUIREMENT : 4+Required Technical Skill:Strong Knowledge of Aws Glue/AWS REDSHIFT/SQL/ETL. Good knowledge and experience in Pyspark for forming complex Transformation logic.AWS Data Engineer,SQL,ETL, DWH , Secondary : AWS Glue , AirflowMust-HaveGood Knowledge of SQL , ETLA minimum of 3 +...
-
System Development Manager, AWS NGDE
2 weeks ago
Chennai, Tamil Nadu, India Amazon Web Services (AWS) Full timeJob DescriptionDescriptionAWS Utility Computing (UC) provides product innovations from foundational services such as Amazons Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWSs services and features apart in the industry. As a member of the UC organization, youll...
-
AWS Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India CBTS Full timeJob Description Skills : Technical: solid understanding of Data Management, AWS tech stack like Python, Glue, Athena, Redshift, Lambda, data security, data governance, and privacy best practices to your AWS data pipelines Non-Tech: Solid communication, data design skills
-
Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India NTT DATA Full timeJob Description Req ID: 303790 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Job Duties: As a data...
-
Senior AWS Data Pipeline Engineer
2 weeks ago
Chennai, Tamil Nadu, India Millims Full timeExperience: 6+ years in data engineering with a focus on AWS cloud technologies. Technical Skills: o Expertise in AWS Glue, Lambda, EMR, Redshift, Kinesis, and Step Functions. o Proficiency in SQL, Python, and PySpark for data transformations. o Strong understanding of ETL/ELT best practices and data warehousing concepts. o Experience with Apache Airflow...
-
Senior Hadoop and AWS Data Engineer
6 days ago
Chennai, Tamil Nadu, India HCLTech Full timeJob OverviewHCLTech is seeking highly skilled professionals to lead our efforts in Hadoop and AWS data engineering. We are looking for individuals with expertise in designing and implementing data architectures on AWS, including data pipelines, storage, and security.
-
Senior Hadoop and AWS Data Engineer
1 day ago
Chennai, Tamil Nadu, India HCLTech Full timeAbout the Role:We are seeking a seasoned Hadoop and AWS data engineer to lead the migration of our Hadoop workloads from on-premise to AWS-EMR stack. The ideal candidate will have 5+ years of experience in designing and implementing data architectures on AWS, including data pipelines, storage, and security.The selected individual will collaborate with...
-
Chennai, Tamil Nadu, India NTT DATA Full timeAbout the Job: NTT Data is looking for an experienced Full Stack Engineer who can lead the development of complex software systems using Java 8, REST, AWS, and Kafka. The ideal candidate will have a strong background in full-stack development and experience working in an Agile environment.Responsibilities: The selected candidate will be responsible...