Aws De- Airflow

3 days ago

Chennai Tamil Nadu, India Willware Technologies Full time

**Work mode: One week of onboarding at Chennai office and later Remote**
- Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure.
- Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance.
- Collaborate with data scientists, analysts, and other stakeholders to understand requirements and translate them into technical solutions.
- Implement best practices for data ingestion, processing, transformation, and storage.
- Monitor pipeline performance, troubleshoot issues, and implement enhancements to improve efficiency and reliability.
- Ensure data quality and integrity throughout the data lifecycle by implementing robust validation and error handling mechanisms.
- Stay current with emerging technologies and industry trends in data engineering and contribute to continuous improvement initiatives.
- Mentor junior team members and provide technical guidance and support as needed.

**Qualification**:

- 5-7 years of professional experience in data engineering, with a focus on building and managing data pipelines.
- Extensive hands-on experience with Apache Airflow, including DAG design, scheduling, and orchestration.
- Strong proficiency in AWS services such as S3, EC2, EMR, Glue, Lambda, and Redshift.
- Solid understanding of data warehousing concepts, ETL processes, and data modeling techniques.
- Proficiency in programming languages such as Python, SQL, and Shell scripting.

**Job Type**: Temporary
Contract length: 3-6 months

**Benefits**:

- Work from home

Schedule:

- Monday to Friday

**Experience**:

- total work: 5 years (required)
- Spark Airflow: 5 years (required)

Work Location: In person

**Speak with the employer**

+91 8667001718

  • python Airflow

    4 weeks ago

    Hyderabad, Bengaluru, Chennai, India Coders Brain Technology Private Limited Full time

    Job DescriptionDescriptionWe are looking for a Python Airflow Developer to join our team in India. The ideal candidate will have a strong background in Python and experience in designing and implementing workflows using Apache Airflow to streamline our data processing and ETL tasks.Responsibilities- Design, develop, and maintain workflows using Apache...

  • Aws Glue

    4 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and maintain scalable data pipelines using Airflow to ensure efficient data processing and integration. - Utilize AWS services to design and implement cloud-based solutions that enhance data storage and retrieval. - Write and optimize SQL queries to extract transform and load data from various sources. -...

  • Data Architect- Aws-big Data

    5 days ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect - AWS-Big Data **Purpose of the Position**: As a Data Architect, you will be responsible for designing and optimizing complex Big Data solutions using AWS services, ensuring a balance between cost and performance. Responsibilities include assessing existing technology, performing impact assessments, optimizing designs, and...

  • AWS Data Engineer with AWS,SQL,Pyspark,Glue,SCD

    4 weeks ago

    Hyderabad, Chennai, Pune, India INNOVA SOLUTIONS Full time

    Job Description- 7-10 yrs exp with Python , Pyspark , SQL, AWS (S3,EMR,EC2,glue,redshift,lamba), Snowflake- S3 life cycle ,DPU in glue ,database connection from glue job, lambda , 100 files in Athena, Spark entry point ,Data Frame Dynamic Frame, Spark Join strategies ,Broadcast and Accumulator , DAG And Lineage, Start snowflake schema, SCD type 2...

  • Aws - Cloud Administrator (Chennai)

    3 days ago

    Chennai, India Forbes Advisor Full time

    Company Description **Forbes Advisor** is a new initiative for consumers under the Forbes Marketplace umbrella that provides journalist - and expert-written insights, news and reviews on all things personal finance, health, business, and everyday life decisions. We do this by providing consumers with the knowledge and research they need to make informed...

  • Aws Data Engineer- Chennai

    6 days ago

    Chennai, Tamil Nadu, India Diverse Lynx Full time

    HIRING FOR AWS DATA ENGINEER- CHENNAI Job Summary We are looking for a highly skilled AWS Data Engineer to join our growing data team In this role you will be responsible for designing building and maintaining scalable and reliable data pipelines and architectures on AWS to support analytics machine learning and business intelligence initiatives ...

  • Aws Cloud Solutions Architect

    6 days ago

    Chennai, Tamil Nadu, India beBeeCloud Full time ₹ 15,00,000 - ₹ 20,00,000

    Aws Cloud Architect JobAs a seasoned professional, you will lead the design and development of cloud-based data systems. We are seeking an experienced Aws Cloud Architect to join our team.Key Responsibilities:Cloud Architecture: Design and implement scalable, secure, and efficient cloud-based architectures using AWS services.Data Engineering: Develop and...

  • AmpsTek - AWS Data Engineer

    9 hours ago

    Chennai, Tamil Nadu, India Ampstek Full time

    Job Role : AWS Data Location : Chennai/Bangalore/Hyderabad/Pune (Hybrid)Key Skills : AWS Glue, RedShift, S3, Lambda Athena.We are seeking a highly skilled AWS Data Engineer with 5 - 8 years of experience in designing, developing, and deploying cloud-based data solutions. The role involves building scalable ETL pipelines, data warehouses, and data...

  • Am (5-7 Years) - Blockchain

    5 days ago

    Chennai, India Renault Group Full time

    Société RNTBCI PL Descriptif du poste AM (5-7 years) - Blockchain Blockchain concepts, AWS, Devops Good knowledge on Blockchain concepts like structure, categories, features, and mining concepts etc Knowledge on Cryptocurrency, Cryptography, Security & Encryption Algorithms and Hash functions Familiarity with IFPS, P2P networks & DLT Extensive...

  • Systems Integration Senior Specialist

    1 week ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Bachelor s degree Computer Science master s degree or Technical Diploma or equivalent At least 5 years of experience in a similar role At least 3 years of experience on AWS and or Azure At least 5 years of experience as Rstudio admin Posit At least 2 years of experience on multiples Azure and AWS PaaS solutions Azure Data Factory MSSQL Azure...