Data Engineer with AWS

17 hours ago

Bengaluru, Karnataka, India Luxoft Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Project description

We are looking for a skilled Data Engineer with strong experience in AWS and Apache Airflow to join a dynamic data team. You will be responsible for building and maintaining scalable data pipelines, orchestrating workflows, and ensuring data quality and availability across platforms.

- Responsibilities
- ?Data Pipeline DevelopmentDesign, build, and maintain robust, scalable, and efficient ETL/ELT pipelines for ingesting, transforming, and loading data from various sources into our data lake and data warehouse.
- ?AWS ExpertiseDevelop and manage data solutions using a wide range of AWS services, including but not limited to
- ?StorageS3 (for data lakes), RDS (PostgreSQL, MySQL), Redshift (data warehousing), DynamoDB.
- ?Compute/ProcessingGlue, EMR (Spark), Lambda, Athena.
- ?Orchestration/WorkflowAWS Step Functions, AWS Data Pipeline.
- ?Networking & SecurityVPC, IAM, Security Groups.
- ?Apache Airflow OrchestrationDevelop, deploy, and manage complex Directed Acyclic Graphs (DAGs) in Apache Airflow to schedule, monitor, and automate data workflows. Implement best practices for Airflow DAG design, error handling, logging, and monitoring. Optimize Airflow performance and manage Airflow environments (e.g., using Amazon MWAA or self-hosted).
- Skills

Must have
- Overall, 8+ years of experience as a Data Engineer and over 5 years of relevant experience in AWS.
- ?ProgrammingStrong proficiency in Python for data manipulation, scripting, and automation.
- ?SQLExpert-level SQL skills for querying, manipulating, and optimizing data in relational and analytical databases.
- ?AWS ServicesProven hands-on experience designing, building, and operating data solutions on AWS (S3, Glue, Redshift, EMR, Lambda, RDS, Step Functions, etc.).
- ?Apache AirflowIn-depth experience with Apache Airflow for data pipeline orchestration, including DAG development, custom operators, sensors, and managing Airflow environments.
- ?ETL/ELTSolid understanding and practical experience with ETL/ELT methodologies and tools.

Nice to have
- N/A

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Migrate ETL workflows from SAP BODS to AWS Glue/dbt/Talend. Develop and maintain scalable ETL pipelines in AWS. Write PySpark scripts for large-scale data processing. Optimize SQL queries and transformations for AWS PostgreSQL. Work with Cloud Engineers to ensure smooth deployment and performance tuning. Integrate data pipelines with existing Unix systems....

  • Aws data engineer

    4 weeks ago

    Bengaluru, Karnataka, India Acme Services Full time

    We are seeking an AWS Data Engineer skilled in AWS, Python, Py Spark, Glue, S3, Lambda, and other AWS data services. The role involves designing and implementing scalable data pipelines, performing ETL processes, and optimizing data workflows on AWS. Candidates must have hands-on experience with cloud-based data solutions, strong coding skills in Python, and...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India HireWand Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    The AWS Data Engineer will design, develop, and maintain scalable data pipelines and architectures using AWS Glue, Amazon Redshift, AWS Lambda, and Amazon S3. This role requires expertise in AWS Kinesis, AWS CloudFormation, Apache Spark, Python, and AWS IAM to ensure secure and efficient data processing. The candidate will leverage AWS Data Pipeline, AWS...

  • System Development Engineer, AWS NGDE

    2 weeks ago

    Bengaluru, Karnataka, India Amazon Web Services (AWS) Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    DescriptionAWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon's Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS's services and features apart in the industry. As a member of the UC organization, you'll support the...

  • AWS Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India GLOBALFOUNDRIES Full time

    Job DescriptionKey Responsibilities:- Design, Migrate, build, and maintain data pipelines for performance, scalability, and reliability- Develop and maintain data models and schemas- Integrate data from diverse sources.- Ensure data quality and implement data governance policies.- Able to create reports using PowerBI.- Collaborate with data analysts, product...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Dakrish Solutions Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Role: AWS Glue Data EngineerLocation: Chennai/Bangalore/HyderabadMode Of Hire: PermanentWork Mode: Work from office / HybridNotice Period: Immediate joiners preferred.No of positions: 3Required Skills:5–10 years of overall data engineering experienceMinimum 3+ years hands-on with AWS Glue and PySparkProficiency in SQL, Python, and AWS data toolsStrong...

  • AWS Architect

    6 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role SummaryAs a AWS Architect, you will lead the design and implementation of scalable, secure, and high-performance data solutions using AWS Platform. You will define architectural standards, guide engineering teams, and collaborate with stakeholders to align data strategies with business goals. Your role will focus on leveraging AWS capabilities to build...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Sampoorna Consultants Pvt. Ltd Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job RequirementsMandatory SkillsBachelors degree in computer science, Data Science, engineering, mathematics, information systems, or a related technical discipline7+ years of relevant experience in data engineering rolesDetailed knowledge of data warehouse technical architectures, data modelling, infrastructure components, ETL/ ELT and reporting/analytic...

  • Data Engineer with Snowflake, AWS Glue, Kafka, API

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 8,00,000 - ₹ 25,00,000 per year

    6+ years of experience in Data Engineering or related role.Hands-on experience with Snowflake (data modelling, performance tuning, query optimization, Snowpipe, Time Travel, Streams & Tasks).Strong expertise in AWS Glue for ETL job development, orchestration, and optimization.Proficiency with AWS services such as S3, Lambda, Redshift, Athena, Step Functions,...

  • AWS Data Engineer

    10 hours ago

    Bengaluru, Karnataka, India Info Origin Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Note: Virtual Drive on 14th and 15th OctRole OverviewWe are looking for an experienced AWS Engineer with around 5-7 years of hands-on experience in designing, deploying, and managing applications on the AWS cloud platform. The ideal candidate should have strong expertise in AWS services, automation, and cloud security, along with the ability to collaborate...