Aws De- Airflow
6 months ago
**Work mode: One week of onboarding at Chennai office and later Remote**
- Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure.
- Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance.
- Collaborate with data scientists, analysts, and other stakeholders to understand requirements and translate them into technical solutions.
- Implement best practices for data ingestion, processing, transformation, and storage.
- Monitor pipeline performance, troubleshoot issues, and implement enhancements to improve efficiency and reliability.
- Ensure data quality and integrity throughout the data lifecycle by implementing robust validation and error handling mechanisms.
- Stay current with emerging technologies and industry trends in data engineering and contribute to continuous improvement initiatives.
- Mentor junior team members and provide technical guidance and support as needed.
**Qualification**:
- 5-7 years of professional experience in data engineering, with a focus on building and managing data pipelines.
- Extensive hands-on experience with Apache Airflow, including DAG design, scheduling, and orchestration.
- Strong proficiency in AWS services such as S3, EC2, EMR, Glue, Lambda, and Redshift.
- Solid understanding of data warehousing concepts, ETL processes, and data modeling techniques.
- Proficiency in programming languages such as Python, SQL, and Shell scripting.
**Job Type**: Temporary
Contract length: 3-6 months
**Benefits**:
- Work from home
Schedule:
- Monday to Friday
**Experience**:
- total work: 5 years (required)
- Spark Airflow: 5 years (required)
Work Location: In person
**Speak with the employer**
+91 8667001718
-
AWS DE- Airflow
4 weeks ago
Chennai, India WILLWARE TECHNOLOGIES Full timeJob Description:Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure.Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance.Collaborate with data scientists, analysts, and other stakeholders to understand requirements and translate them into technical solutions.Implement...
-
Aws Dataengineer 6 to 8 Years Pan India
2 weeks ago
Chennai, Tamil Nadu, India Capgemini Full time**Job Description**: - Hands on industry experience in design and coding from scratch in AWS Glue-Pyspark with services like - S3, DynamoDB, StepFunctions etc. - Hands on industry experience in design and coding from scratch in Snowflake. - Experience in Pyspark/Snowflake 1 to 3 years with overall around 5 years of experience in building data/analytics...
-
Data Architect
4 weeks ago
Tamil Nadu, India Frost and Sullivan Private Limited Full timeAbout the JobAt Frost & Sullivan, we are seeking a highly skilled and experienced Data Architect to join our Digital Products team. The ideal candidate will be responsible for designing, building, and maintaining our organization's data architecture.Key ResponsibilitiesDesign and implement efficient data architectures, storage solutions, and data processing...
-
Data Pipeline Engineer
3 weeks ago
Chennai, Tamil Nadu, India WILLWARE TECHNOLOGIES Full timeJob Title: AWS Cloud Data EngineerJob SummaryWe are seeking a highly skilled AWS Cloud Data Engineer to join our team at WILLWARE TECHNOLOGIES. The successful candidate will design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure. This is an excellent opportunity for a data engineer to work on challenging projects and...
-
AWS Data Engineer
4 weeks ago
Chennai, Tamil Nadu, India WILLWARE TECHNOLOGIES Full timeJob Title: AWS Data Engineer - Pipeline SpecialistAbout the Role:As a key member of our data engineering team at WILLWARE TECHNOLOGIES, you will be responsible for designing, developing, and deploying complex data pipelines using Apache Airflow on AWS infrastructure. Your expertise in data engineering, combined with your passion for innovative solutions,...
-
AWS Cloud Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India Fulcrum Digital Full timeAbout the RoleFulcrum Digital is a next-generation digital accelerating company providing digital transformation and technology services across various industries.We are seeking an experienced professional with a strong background in AWS cloud technologies and data engineering.The ideal candidate will be proficient in a wide range of AWS services and tools,...
-
Senior Data Engineer AWS
4 weeks ago
Chennai, Tamil Nadu, India Fulcrum Digital Full timeAbout the RoleFulcrum Digital is a cutting-edge digital transformation company providing innovative technology services to various industries. We are seeking a skilled Senior Data Engineer with expertise in AWS cloud technologies and data engineering.The ideal candidate will be proficient in a wide range of AWS services and tools, including Databricks, MWAA,...
-
AWS Cloud Data Engineer
4 weeks ago
Chennai, Tamil Nadu, India Fulcrum Digital Full timeAbout Fulcrum DigitalWe are an agile and next-generation digital accelerating company providing digital transformation and technology services right from ideation to implementation. Our services have applicability across a variety of industries including banking & financial services, insurance, retail, higher education, food, healthcare, and...
-
Sr Data Engineer AWS
4 weeks ago
Chennai, India Fulcrum Digital Full timeWho are we Fulcrum Digital is an agile and nextgeneration digital accelerating company providing digital transformation and technology services right from ideation to implementation. These services have applicability across a variety of industries including banking & financial services insurance retail higher education food healthcare and manufacturing. The...
-
AWS Cloud Data Engineer
4 weeks ago
Chennai, Tamil Nadu, India Fulcrum Digital Full timeAbout the RoleFulcrum Digital is a leading digital transformation company that provides cutting-edge technology services. We are seeking an experienced professional with a strong background in AWS cloud technologies and data engineering.Key ResponsibilitiesDesign, develop, and maintain scalable data pipelines using AWS Databricks, AWS Lambda, AWS S3, Delta...
-
AWS Cloud Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India Fulcrum Digital Full timeAt Fulcrum Digital, we are seeking a highly skilled professional with a strong background in AWS cloud technologies and data engineering to join our team.The ideal candidate will be proficient in a wide range of AWS services and tools, including Databricks, MWAA, Lambda, and Snowflake, among others.This role involves developing, managing, and optimizing...
-
Sr Data Engineer AWS
3 months ago
Chennai, India Fulcrum Digital Full timeWho areweFulcrumDigital is an agile and nextgeneration digital accelerating companyproviding digital transformation and technology services right fromideation to implementation. These services have applicabilityacross a variety of industries including banking &financial services insurance retail higher education foodhealthcare andmanufacturing.TheRoleWeare...
-
Aws data engineer(chennai, pune)
4 hours ago
Chennai, India Tata Consultancy Services Full timeRole: AWS Data EngineerJOB LOCATION : Chennai, PuneEXPERIENCE REQUIREMENT : 4+Required Technical Skill:Strong Knowledge of Aws Glue/AWS REDSHIFT/SQL/ETL. Good knowledge and experience in Pyspark for forming complex Transformation logic.AWS Data Engineer,SQL, ETL, DWH , Secondary : AWS Glue , AirflowMust-HaveGood Knowledge of SQL , ETLA minimum of 3 +...
-
AWS Data Engineer(Chennai, Pune)
1 day ago
Chennai, India Tata Consultancy Services Full timeRole: AWS Data EngineerJOB LOCATION : Chennai, PuneEXPERIENCE REQUIREMENT : 4+Required Technical Skill:Strong Knowledge of Aws Glue/AWS REDSHIFT/SQL/ETL. Good knowledge and experience in Pyspark for forming complex Transformation logic.AWS Data Engineer,SQL,ETL, DWH , Secondary : AWS Glue , AirflowMust-HaveGood Knowledge of SQL , ETLA minimum of 3 +...
-
AWS Data Engineer(Chennai, Pune)
14 hours ago
Chennai, India Tata Consultancy Services Full timeRole: AWS Data EngineerJOB LOCATION : Chennai, PuneEXPERIENCE REQUIREMENT : 4+Required Technical Skill:Strong Knowledge ofAws Glue/AWS REDSHIFT/SQL/ETL.Good knowledge and experience in Pyspark for forming complex Transformation logic.AWS Data Engineer,SQL,ETL,DWH, Secondary : AWS Glue , AirflowMust-HaveGood Knowledge of SQL , ETLA minimum of 3 + years'...
-
Sr Data Engineer AWS
4 weeks ago
chennai, India Fulcrum Digital Full timeWho are weFulcrum Digital is an agile and nextgeneration digital accelerating company providing digital transformation and technology services right from ideation to implementation. These services have applicability across a variety of industries including banking & financial services insurance retail higher education food healthcare and manufacturing. The...
-
Tamil Nadu, India Frost and Sullivan (India) Private Limited Full timeAbout the job :At Frost & Sullivan, we are seeking a highly skilled and experienced Data Architect to join our Digital Products team! The ideal fit for the role will be a person experienced in designing, building, and maintaining an organization's data architecture. The key focus of the position is designing and implementing data models, data...
-
Tamil Nadu, India Frost and Sullivan Private Limited Full timeAbout the job :At Frost & Sullivan, we are seeking a highly skilled and experienced Data Architect to join our Digital Products team! The ideal fit for the role will be a person experienced in designing, building, and maintaining an organization's data architecture. The key focus of the position is designing and implementing data models, data integration...
-
Lead AWS Data Engineer
1 week ago
Chennai, India Overture Rede Full timeJob Title - AWS Data Engineer LeadLocation - Pan India Notice Period: Only Immediate Joiners.Mandatory Skills: AWS ,Glue,Athena ,Redshift, S3, EMR,Pyspark, Python,SQL.Roles & Responsibilities :Experience of AWS tools (e.g Athena, Redshift, Glue, EMR) ,Scala, Python, Spark Min 6- 10 yearsExperience of developing enterprise grade ETL/ELT data pipelines and...
-
Data Ops Engineer
3 weeks ago
tamil nadu, India Live Connections Full timeExp : 12yrs to 16yrsWe are looking for a strong IC candidate who has strong expertise in DevOps, DataOps, AWS, and Python.12+ years in data and software engineering.Deep expertise in Data Engineering, Software Engineering, Cloud Engineering, and AI (Knowledge is Mandatory).Strong understanding of DevOps, DataOps, and product management.Experience with Agile...