Senior AWS Data Services Engineer

1 day ago

Bengaluru Bangalore Hyderabad Secunderabad Telangana, India beBeeDataLakeAdministrator Full time US$ 1,50,000 - US$ 2,00,000
Job Title

Highly Skilled AWS Data Lake Administrator Required

Job Description

We are seeking a highly skilled professional to manage and optimize our data lake architecture on AWS. The ideal candidate will have strong experience with AWS data services such as S3, Lake Formation, Glue Catalog, Redshift Spectrum, and extensive SQL knowledge.

The role involves managing data ingestion, security, and performance of the data lake, ensuring efficient and secure access to data for analytics and reporting.

  • Administer and manage the AWS Data Lake infrastructure, ensuring high availability, security, and performance.
  • Configure and manage AWS S3, Lake Formation, and Glue Data Catalog to organize, secure, and catalog data within the data lake.
  • Set up and manage Redshift Spectrum for querying and analyzing data stored in S3 using SQL and Redshift.
  • Implement and manage data ingestion pipelines to ingest structured and unstructured data from various sources into the data lake using AWS services like Glue ETL, Lambda, and other orchestration tools.
  • Define and enforce data governance policies, access control, and security measures using AWS Lake Formation, ensuring compliance with organizational and regulatory requirements.
  • Optimize data storage in S3 through partitioning, compression, and appropriate data formats like Parquet, Avro, or ORC to improve query performance.
  • Monitor and manage the Glue Catalog for maintaining metadata, table definitions, and data lineage within the data lake.
  • Collaborate with data engineers, architects, and analysts to ensure seamless access to data for analytics and machine learning.
  • Troubleshoot and optimize query performance in Redshift Spectrum, and manage workload performance in the data lake environment.
  • Implement backup, recovery, and disaster recovery strategies for data lake assets, ensuring data integrity and availability.
  • Maintain detailed documentation of data lake architecture, security policies, and data governance procedures.

Experience with data formats (Parquet, ORC, Avro, JSON) and optimizing data storage for cost and performance is also desired.

Required Skills and Qualifications

4-8 years of experience in managing data lake environments, with a focus on AWS data services.

Strong expertise in AWS services like S3, Lake Formation, Glue, Glue Data Catalog, Redshift Spectrum, and Lambda.

Good knowledge of SQL and query optimization techniques, particularly in querying data from S3 using Redshift Spectrum and Athena.

Experience in data ingestion pipelines using AWS Glue ETL, Lambda, and other AWS orchestration tools.

Strong understanding of data governance, access control, and security best practices in AWS environments.

Excellent communication and collaboration skills to work across teams and support business requirements.

Benefits

AWS certifications, such as AWS Certified Big Data – Specialty or AWS Certified Solutions Architect, are a plus.

  • AWS Data Engineer

    6 days ago

    Mumbai, Bengaluru / Bangalore, Hyderabad / Secunderabad, Telangana, India beBeeArchitect Full time US$ 90,000 - US$ 1,20,000

    Job DescriptionWe seek experienced professionals with 6 to 12 years of expertise in data engineering and architecture to drive our projects forward.Key Responsibilities:• Identify opportunities to enhance application performance by recognizing the current infrastructure and suggesting new concepts.• Document best practices and strategies associated with...

  • AWS Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from TCSTCS is Hiring For Principal AWS Data EngineerExperience: 10 to 14 yearsLocation: BangaloreJob DescriptionRole: AWS Data EngineerExperience: 10-12 yearsLocation: BengaluruMust Have Tech Skills:Prior Principal Engineer experience, leading team best practices in design, development, and implementation, mentoring team members, and fostering a...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Acme Services Full time

    We are seeking an AWS Data Engineer skilled in AWS, Python, PySpark, Glue, S3, Lambda, and other AWS data services. The role involves designing and implementing scalable data pipelines, performing ETL processes, and optimizing data workflows on AWS. Candidates must have hands-on experience with cloud-based data solutions, strong coding skills in Python, and...

  • AWS Data Engineer

    5 days ago

    Bengaluru, Karnataka, India Acme Services Full time

    We are seeking an AWS Data Engineer skilled in AWS, Python, PySpark, Glue, S3, Lambda, and other AWS data services. The role involves designing and implementing scalable data pipelines, performing ETL processes, and optimizing data workflows on AWS. Candidates must have hands-on experience with cloud-based data solutions, strong coding skills in Python, and...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Acme Services Full time

    We are seeking an AWS Data Engineer skilled in AWS, Python, PySpark, Glue, S3, Lambda, and other AWS data services. The role involves designing and implementing scalable data pipelines, performing ETL processes, and optimizing data workflows on AWS. Candidates must have hands-on experience with cloud-based data solutions, strong coding skills in Python, and...

  • AWS Data Engineer

    18 hours ago

    Bengaluru, Karnataka, India Acme Services Full time

    We are seeking an AWS Data Engineer skilled in AWS, Python, PySpark, Glue, S3, Lambda, and other AWS data services. The role involves designing and implementing scalable data pipelines, performing ETL processes, and optimizing data workflows on AWS. Candidates must have hands-on experience with cloud-based data solutions, strong coding skills in Python, and...

  • AWS Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from Tata Consultancy ServicesWe are hiring AWS Data Engineer Position: AWS Data EngineerJob Location: BangaloreExperience : 10-14 yearsInterested professionals kindly apply through the link.Must Have:Prior Principal Engineer experience, leading team best practices in design, development, and implementation, mentoring team members, and fostering...

  • AWS Data Engineer

    3 days ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from Tata Consultancy ServicesWe are hiring AWS Data Engineer Position: AWS Data EngineerJob Location: BangaloreExperience : 10-14 yearsInterested professionals kindly apply through the link.Must Have:- Prior Principal Engineer experience, leading team best practices in design, development, and implementation, mentoring team members, and fostering...

  • AWS Data Engineer

    18 hours ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from Tata Consultancy Services We are hiring AWS Data Engineer Position: AWS Data Engineer Job Location: Bangalore Experience : 10-14 years Interested professionals kindly apply through the link. Must Have: Prior Principal Engineer experience, leading team best practices in design, development, and implementation, mentoring team members, and...

  • AWS Data Engineer

    1 day ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from Tata Consultancy ServicesWe are hiring AWS Data Engineer Position: AWS Data EngineerJob Location: BangaloreExperience : 10-14 yearsInterested professionals kindly apply through the link.Must Have:Prior Principal Engineer experience, leading team best practices in design, development, and implementation, mentoring team members, and fostering a...