Data Engineer with AWS

1 day ago

Bengaluru, Karnataka, India Luxoft Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Project description

We are looking for a skilled Data Engineer with strong experience in AWS and Apache Airflow to join a dynamic data team. You will be responsible for building and maintaining scalable data pipelines, orchestrating workflows, and ensuring data quality and availability across platforms.

- Responsibilities
- ?Data Pipeline DevelopmentDesign, build, and maintain robust, scalable, and efficient ETL/ELT pipelines for ingesting, transforming, and loading data from various sources into our data lake and data warehouse.
- ?AWS ExpertiseDevelop and manage data solutions using a wide range of AWS services, including but not limited to
- ?StorageS3 (for data lakes), RDS (PostgreSQL, MySQL), Redshift (data warehousing), DynamoDB.
- ?Compute/ProcessingGlue, EMR (Spark), Lambda, Athena.
- ?Orchestration/WorkflowAWS Step Functions, AWS Data Pipeline.
- ?Networking & SecurityVPC, IAM, Security Groups.
- ?Apache Airflow OrchestrationDevelop, deploy, and manage complex Directed Acyclic Graphs (DAGs) in Apache Airflow to schedule, monitor, and automate data workflows. Implement best practices for Airflow DAG design, error handling, logging, and monitoring. Optimize Airflow performance and manage Airflow environments (e.g., using Amazon MWAA or self-hosted).
- Skills

Must have
- Overall, 8+ years of experience as a Data Engineer and over 5 years of relevant experience in AWS.
- ?ProgrammingStrong proficiency in Python for data manipulation, scripting, and automation.
- ?SQLExpert-level SQL skills for querying, manipulating, and optimizing data in relational and analytical databases.
- ?AWS ServicesProven hands-on experience designing, building, and operating data solutions on AWS (S3, Glue, Redshift, EMR, Lambda, RDS, Step Functions, etc.).
- ?Apache AirflowIn-depth experience with Apache Airflow for data pipeline orchestration, including DAG development, custom operators, sensors, and managing Airflow environments.
- ?ETL/ELTSolid understanding and practical experience with ETL/ELT methodologies and tools.

Nice to have
- N/A

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Migrate ETL workflows from SAP BODS to AWS Glue/dbt/Talend. Develop and maintain scalable ETL pipelines in AWS. Write PySpark scripts for large-scale data processing. Optimize SQL queries and transformations for AWS PostgreSQL. Work with Cloud Engineers to ensure smooth deployment and performance tuning. Integrate data pipelines with existing Unix systems....

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India HireWand Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    The AWS Data Engineer will design, develop, and maintain scalable data pipelines and architectures using AWS Glue, Amazon Redshift, AWS Lambda, and Amazon S3. This role requires expertise in AWS Kinesis, AWS CloudFormation, Apache Spark, Python, and AWS IAM to ensure secure and efficient data processing. The candidate will leverage AWS Data Pipeline, AWS...

  • Data Engineer with Snowflake, AWS Glue, Kafka, API

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 8,00,000 - ₹ 25,00,000 per year

    6+ years of experience in Data Engineering or related role.Hands-on experience with Snowflake (data modelling, performance tuning, query optimization, Snowpipe, Time Travel, Streams & Tasks).Strong expertise in AWS Glue for ETL job development, orchestration, and optimization.Proficiency with AWS services such as S3, Lambda, Redshift, Athena, Step Functions,...

  • AWS Architect

    6 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role SummaryAs a AWS Architect, you will lead the design and implementation of scalable, secure, and high-performance data solutions using AWS Platform. You will define architectural standards, guide engineering teams, and collaborate with stakeholders to align data strategies with business goals. Your role will focus on leveraging AWS capabilities to build...

  • AWS Data Engineer

    18 hours ago

    Bengaluru, Karnataka, India Info Origin Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Note: Virtual Drive on 14th and 15th OctRole OverviewWe are looking for an experienced AWS Engineer with around 5-7 years of hands-on experience in designing, deploying, and managing applications on the AWS cloud platform. The ideal candidate should have strong expertise in AWS services, automation, and cloud security, along with the ability to collaborate...

  • AWS Data Engineer

    3 days ago

    Bengaluru, Karnataka, India eSkillport Management Services Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Description Job Title: AWS Data Engineer (IoT/Streaming Data) Location: Chennai or Bangalore. Experience: 3 Years Employment Type: Full-time About the Role We are seeking a highly skilled AWS Data Engineer to design, build, and optimize large-scale data pipelines and analytics platforms. The ideal candidate will have strong...

  • AWS Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Zenotis Group Full time ₹ 1,00,00,000 - ₹ 3,00,00,000 per year

    Position Title:AWS Data EngineerYear of Experience Needed:6+ YearsLocation:Hyderabad | Bangalore | ChennaiWorking Time:2 PM – 11 PM ISTWorking Mode:Hybrid (3 Days WFO)Note: This opportunity is for one of our MNC Clients.Primary Skills:Python, Pyspark, AWS, Data EngineerJob Description / Skills Required:Primary Skills:• Python, Pyspark, Glue, Redshift,...

  • Software Development Manager, AWS OpenSearch Service, AWS OpenSearch Core

    1 day ago

    Bengaluru, Karnataka, India Amazon Web Services (AWS) Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    DescriptionAWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon's Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS's services and features apart in the industry. As a member of the UC organization, you'll support the...

  • AWS Data Engineer

    6 days ago

    Bengaluru, Karnataka, India Ubique Systems Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    ** NOTICE - Immediate to 10 days MaXX** Exp yrsAWS DATA ENGINEERResponsibility of / Expectations from the Role ::AWS data engineer to build ETL data pipelines (Glue, python based EKS ingestion pipelines) and with DevOps experience to build IaC terraform, CICD gitlab pipelines.Support FTP and ALM modules implementation.Provide hands-on support for the...

  • Data Engineer

    6 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...