Senior Data Engineer-AWS

2 days ago

Kolkata, India Tredence Inc. Full time

Technical Skills Required: - At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift - AWS Redshift: - Worked on creating provisioned clusters - Expert knowledge of Workload management - Materialized View, DDL Optimizations, Json handling - AWS Glue to Redshift Loads - Row & Column Level Security - AWS Glue (ETL, Jobs, Crawlers, Catalogue including Iceberg, Workflows, Dynamic Data Frames) - Worked on Batch & Stream workloads - Performance Tuning & Cost Optimization - IAM, KMS, Secret Manager and Fine-Grained Access Control (Encryption good to have) - PySpark, Python - Lambda, Amazon S3, Athena, RDS - Apache Parquet, JSON, CSV - Data Lake Design & Implementation Good to have: - CI/CD (Terraform or AWS CDK or CloudFormation) - Data Lineage & Governance - Kinesis/Kafka/Glue Streaming/AWS Batch Key Responsibilities: - Collaborate with business stakeholders to analyze data requirements and define ETL & API architecture aligned with business goals. - Design and implement scalable ETL workflows using AWS Glue and PySpark for ingesting structured and semi-structured data into the AWS data lake. - Develop reusable Glue jobs and crawlers for automated metadata cataloging and data transformations. - Optimize Glue job performance using dynamic frame partitioning, job bookmarking, and parallelism tuning. - Integrate Glue with other AWS services such as S3, Athena, Redshift, Lambda, and CloudWatch for end-to-end orchestration and monitoring. - Lead data lakehouse implementation leveraging Glue with Iceberg for versioned, transactional data storage. - Ensure secure access to datasets using fine-grained IAM policies and Lake Formation(Good to Have). - Mentor junior engineers, enforce coding best practices, and participate in code reviews and architectural discussions.

  • Aws data engineer

    2 weeks ago

    kolkata, India A Mid-size IT Co - Client Of Axim Digitech Full time

    Role: AWS Data Engineer –Location – KolkataJob Type: 6 months contract roleJob description• We're on-boarding an AWS Data Engineer who can own the data backbone—building scalable pipelines,automating integrations, and enabling analytics teams with clean, reliable datasets.• You'll be deployed onsite with a leading enterprise client, driving...

  • Senior Data Engineer-AWS

    3 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift AWS Redshift: Worked on creating provisioned clusters Expert knowledge of Workload management Materialized View, DDL Optimizations, Json handling AWS Glue to Redshift Loads Row & Column Level Security AWS Glue (ETL,...

  • Senior Data Engineer-AWS

    2 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift AWS Redshift: Worked on creating provisioned clusters Expert knowledge of Workload management Materialized View, DDL Optimizations, Json handling AWS Glue to Redshift Loads Row & Column Level Security AWS Glue (ETL,...

  • Senior Data Engineer-AWS

    2 days ago

    kolkata, India Tredence Inc. Full time

    Technical Skills Required:- At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift- AWS Redshift:- Worked on creating provisioned clusters- Expert knowledge of Workload management- Materialized View, DDL Optimizations, Json handling- AWS Glue to Redshift Loads- Row & Column Level Security- AWS...

  • Senior Data Engineer-AWS

    1 day ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required:- At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift- AWS Redshift:- Worked on creating provisioned clusters- Expert knowledge of Workload management- Materialized View, DDL Optimizations, Json handling- AWS Glue to Redshift Loads- Row & Column Level Security- AWS...

  • Senior Data Engineer-AWS

    1 day ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift AWS Redshift: Worked on creating provisioned clusters Expert knowledge of Workload management Materialized View, DDL Optimizations, Json handling AWS Glue to Redshift Loads Row & Column Level Security AWS Glue (ETL,...

  • Senior Data Engineer-AWS

    3 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), RedshiftAWS Redshift:Worked on creating provisioned clustersExpert knowledge of Workload managementMaterialized View, DDL Optimizations, Json handlingAWS Glue to Redshift LoadsRow & Column Level SecurityAWS Glue (ETL, Jobs,...

  • Senior Data Engineer-AWS

    3 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), RedshiftAWS Redshift:Worked on creating provisioned clustersExpert knowledge of Workload managementMaterialized View, DDL Optimizations, Json handlingAWS Glue to Redshift LoadsRow & Column Level SecurityAWS Glue (ETL, Jobs,...

  • Senior Data Engineer-Aws

    3 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), RedshiftAWS Redshift:Worked on creating provisioned clustersExpert knowledge of Workload managementMaterialized View, DDL Optimizations, Json handlingAWS Glue to Redshift LoadsRow & Column Level SecurityAWS Glue (ETL, Jobs,...

  • Senior Data Engineer-AWS

    1 day ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required:At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), RedshiftAWS Redshift:Worked on creating provisioned clustersExpert knowledge of Workload managementMaterialized View, DDL Optimizations, Json handlingAWS Glue to Redshift LoadsRow & Column Level SecurityAWS Glue (ETL, Jobs,...