AWS DE- Airflow
1 month ago
Job Description:
- Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure.
- Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance.
- Collaborate with data scientists, analysts, and other stakeholders to understand requirements and translate them into technical solutions.
- Implement best practices for data ingestion, processing, transformation, and storage.
- Monitor pipeline performance, troubleshoot issues, and implement enhancements to improve efficiency and reliability.
- Ensure data quality and integrity throughout the data lifecycle by implementing robust validation and error handling mechanisms.
- Stay current with emerging technologies and industry trends in data engineering and contribute to continuous improvement initiatives.
- Mentor junior team members and provide technical guidance and support as needed.
Qualification:
- 5-7 years of professional experience in data engineering, with a focus on building and managing data pipelines.
- Extensive hands-on experience with Apache Airflow, including DAG design, scheduling, and orchestration.
- Strong proficiency in AWS services such as S3, EC2, EMR, Glue, Lambda, and Redshift.
- Solid understanding of data warehousing concepts, ETL processes, and data modeling techniques.
- Proficiency in programming languages such as Python, SQL, and Shell scripting.
-
Aws De- Airflow
4 weeks ago
Chennai, Tamil Nadu, India Willware Technologies Full time**Work mode: One week of onboarding at Chennai office and later Remote** - Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure. - Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance. - Collaborate with data scientists, analysts, and other stakeholders to understand...
-
Platform Engineer
4 weeks ago
chennai, India Mobile Programming LLC Full timeJob Description : Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development...
-
Platform Engineer
2 weeks ago
Chennai, India Mobile Programming LLC Full timeJob Description :Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development plans,...
-
AWS cloud Engineer
4 weeks ago
chennai, India Virtusa Full timeAWS cloud Engineer - CREQ187082 Description Minimum 6-9 years of experience in AWS CloudBe responsible for the planning, implementation, and growth of the AWS cloud infrastructure.Implement continuous integration/continuous delivery (CI/CD) pipelines when necessary.Work alongside architecture and engineering teams to design and implement any scalable...
-
AWS Data Engineer
2 weeks ago
Bangalore/Pune/Chennai, India Change leaders Full timeJob Description : AWS Glue + Python PysparkJob Overview :- Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform.- This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...
-
AWS Data Engineer
3 weeks ago
Bangalore/Pune/Chennai, IN Change leaders Full timeJob Description : AWS Glue + Python PysparkJob Overview :- Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform.- This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...
-
AWS Data Engineer
3 weeks ago
Chennai, India CliqHR Full timeJob Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...
-
AWS Data Engineer
3 weeks ago
Chennai, India CliqHR Full timeJob Description : - Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...
-
AWS Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India CliqHR Full timeJob Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...
-
Platform Engineer
2 weeks ago
Chennai/Anywhere in India/Multiple Locations Mobile Programming LLC Full timeJob Description : Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development...
-
Platform Engineer
3 weeks ago
Chennai/Anywhere in India/Multiple Locations, IN Mobile Programming LLC Full timeJob Description :Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development plans,...
-
AWS Data Engineer
2 weeks ago
Pune/Chennai/Mumbai/Bangalore/Delhi NCR/Hyderabad, India Change Leaders Full timeJob Description : AWS Glue + Python PysparkJob Overview :Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform. This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...
-
AWS Data Engineer
3 weeks ago
Pune/Chennai/Mumbai/Bangalore/Delhi NCR/Hyderabad, IN Change leaders Full timeJob Description : AWS Glue + Python PysparkJob Overview :Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform. This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...
-
Senior Data Engineer( AWS Redshift)
1 month ago
chennai, India Anicalls (Pty) Ltd Full timeCandidate should have:Clear, concise communication abilities - writing, verbal, presentation - to all levels of technical and non-technical audiences with the ability to collaborateStrong experience in CICD technologies including Python, SQL, Airflow, and Git in a cloud settingUnderstanding of ELK stack is a plus for logging and reporting purposesExperience...
-
DevOps Engineer
2 weeks ago
Bangalore/Hyderabad/Pune/Chennai, India Growel Softech Pvt. Ltd. Full timeJob Description:We are seeking two experienced Senior DevOps Engineers to join our team, with a strong focus on AWS DevOps, particularly with expertise in EKS and Dockers. The ideal candidates will have a deep understanding of AWS Cloud services, including but not limited to EC2 instances, ECS, and data-related applications. Familiarity with data tools such...
-
AWS Data Engineer
4 weeks ago
Chennai, India CliqHR Full timeMandatory Skills : AWS, Python, SQL, spark, Airflow, SnowflakeResponsibilities :- Create and manage cloud resources in AWS- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and...
-
AWS Data Engineer
1 month ago
Chennai, India CliqHR Full timeMandatory Skills : AWS, Python, SQL, spark, Airflow, SnowflakeResponsibilities :- Create and manage cloud resources in AWS- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and...
-
AWS Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India CliqHR Full timeMandatory Skills : AWS, Python, SQL, spark, Airflow, SnowflakeResponsibilities :- Create and manage cloud resources in AWS- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and...
-
AWS Data Engineer
6 days ago
Chennai, India LexisNexis Full timeSoftware Engineer IIIDo you enjoy designing analytic solutions at an enterprise scale for global customers?Do you have experience in working on complex problems?About our TeamOur product analytics team focuses on using data to understand and enhance the user experience of our customer facing products. We gather and analyze user interaction data from various...
-
Senior Cloud Data Engineer
1 month ago
Hyderabad,Bangalore,Chennai,Vishakhapatnam,Vizag, India Mouri Tech (P) Ltd Full timeSenior Cloud Data Engineer (AWS, Python, Pyspark) Job Title : Senior AWS Cloud Data EngineerWork Location : Hyderabad/Kolhapur/Bangalore/Chennai Experience : 6 to 9 yearsRequired Skills : Python, Pyspark, AWS services such as DMS, SAM, Glue, Redshift, Lamda S3 with Apache Airflow, CI/CD, : - Minimum of 6-9 Experience with the core AWS Services and setting...