Senior Data Engineer-AWS

4 days ago

Kolkata, India Tredence Inc. Full time

Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), RedshiftAWS Redshift:Worked on creating provisioned clustersExpert knowledge of Workload managementMaterialized View, DDL Optimizations, Json handlingAWS Glue to Redshift LoadsRow & Column Level SecurityAWS Glue (ETL, Jobs, Crawlers, Catalogue including Iceberg, Workflows, Dynamic Data Frames)Worked on Batch & Stream workloadsPerformance Tuning & Cost OptimizationIAM, KMS, Secret Manager and Fine-Grained Access Control (Encryption good to have)PySpark, PythonLambda, Amazon S3, Athena, RDSApache Parquet, JSON, CSVData Lake Design & ImplementationGood to have:CI/CD (Terraform or AWS CDK or CloudFormation)Data Lineage & GovernanceKinesis/Kafka/Glue Streaming/AWS BatchKey Responsibilities:Collaborate with business stakeholders to analyze data requirements and define ETL & API architecture aligned with business goals.Design and implement scalable ETL workflows using AWS Glue and PySpark for ingesting structured and semi-structured data into the AWS data lake.Develop reusable Glue jobs and crawlers for automated metadata cataloging and data transformations.Optimize Glue job performance using dynamic frame partitioning, job bookmarking, and parallelism tuning.Integrate Glue with other AWS services such as S3, Athena, Redshift, Lambda, and CloudWatch for end-to-end orchestration and monitoring.Lead data lakehouse implementation leveraging Glue with Iceberg for versioned, transactional data storage.Ensure secure access to datasets using fine-grained IAM policies and Lake Formation(Good to Have).Mentor junior engineers, enforce coding best practices, and participate in code reviews and architectural discussions.

  • Senior Data Engineer-AWS

    3 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift AWS Redshift: Worked on creating provisioned clusters Expert knowledge of Workload management Materialized View, DDL Optimizations, Json handling AWS Glue to Redshift Loads Row & Column Level Security AWS Glue (ETL,...

  • Senior Data Engineer-AWS

    4 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift AWS Redshift: Worked on creating provisioned clusters Expert knowledge of Workload management Materialized View, DDL Optimizations, Json handling AWS Glue to Redshift Loads Row & Column Level Security AWS Glue (ETL,...

  • Senior Data Engineer-AWS

    4 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift AWS Redshift: Worked on creating provisioned clusters Expert knowledge of Workload management Materialized View, DDL Optimizations, Json handling AWS Glue to Redshift Loads Row & Column Level Security AWS Glue (ETL,...

  • Senior Data Engineer-AWS

    4 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: - At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift - AWS Redshift: - Worked on creating provisioned clusters - Expert knowledge of Workload management - Materialized View, DDL Optimizations, Json handling - AWS Glue to Redshift Loads - Row & Column Level Security...

  • Senior Data Engineer-AWS

    4 days ago

    kolkata, India Tredence Inc. Full time

    Technical Skills Required:- At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift- AWS Redshift:- Worked on creating provisioned clusters- Expert knowledge of Workload management- Materialized View, DDL Optimizations, Json handling- AWS Glue to Redshift Loads- Row & Column Level Security- AWS...

  • Senior Data Engineer-AWS

    3 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required:- At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift- AWS Redshift:- Worked on creating provisioned clusters- Expert knowledge of Workload management- Materialized View, DDL Optimizations, Json handling- AWS Glue to Redshift Loads- Row & Column Level Security- AWS...

  • Senior Data Engineer-AWS

    3 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required:At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), RedshiftAWS Redshift:Worked on creating provisioned clustersExpert knowledge of Workload managementMaterialized View, DDL Optimizations, Json handlingAWS Glue to Redshift LoadsRow & Column Level SecurityAWS Glue (ETL, Jobs,...

  • Senior Data Engineer-AWS

    4 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), RedshiftAWS Redshift:Worked on creating provisioned clustersExpert knowledge of Workload managementMaterialized View, DDL Optimizations, Json handlingAWS Glue to Redshift LoadsRow & Column Level SecurityAWS Glue (ETL, Jobs,...

  • Senior Data Engineer-Aws

    4 days ago

    Kolkata, India Tredence Inc. Full time

    Technical Skills Required: At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), RedshiftAWS Redshift:Worked on creating provisioned clustersExpert knowledge of Workload managementMaterialized View, DDL Optimizations, Json handlingAWS Glue to Redshift LoadsRow & Column Level SecurityAWS Glue (ETL, Jobs,...

  • AWS Data Engineer

    3 weeks ago

    Kolkata, India Tata Consultancy Services Full time

    Greetings!!! TCS is hiring... Role: AWS Data Engineer EXP: 5- 8 YEARS LOCATION: Hyderabad/ Kolkata **** Notice Period- 30 Days *** Desired Skills -Must-Have - AWS Python , AWS Glue, Lambda, Athena, AWS Pyspark - Good hands-on experience in Python programming and Pyspark/Scala - Data Engineering experience using AWS core services (Lambda, Glue, EMR, S3 and...