AWS Data Lake Administrator

4 weeks ago

Bengaluru, Karnataka, India Tata Consultancy Services Full time

Role: AWS Data Lake Administrator

Location: Bangalore and Hyderabad

Years of Experience: 10-14 years (Mandatory)

Notice Period: 30 days or lesser.

Job Description:

We are seeking a highly skilled AWS Data Lake Administrator to manage and optimize our data lake architecture on AWS. The ideal candidate will have strong experience with AWS data services such as S3, Lake Formation, Glue Catalog, Redshift Spectrum, and extensive SQL knowledge. The role involves managing data ingestion, security, and performance of the data lake, ensuring efficient and secure access to data for analytics and reporting.

Key Responsibilities:

  • Administer and manage the AWS Data Lake infrastructure, ensuring high availability, security, and performance.
  • Configure and manage AWS S3, Lake Formation, and Glue Data Catalog to organize, secure, and catalog data within the data lake.
  • Set up and manage Redshift Spectrum for querying and analyzing data stored in S3 using SQL and Redshift.
  • Implement and manage data ingestion pipelines to ingest structured and unstructured data from various sources into the data lake using AWS services like Glue ETL, Lambda, and other orchestration tools.
  • Define and enforce data governance policies, access control, and security measures using AWS Lake Formation, ensuring compliance with organizational and regulatory requirements.
  • Optimize data storage in S3 through partitioning, compression, and appropriate data formats like Parquet, Avro, or ORC to improve query performance.
  • Monitor and manage the Glue Catalog for maintaining metadata, table definitions, and data lineage within the data lake.
  • Collaborate with data engineers, architects, and analysts to ensure seamless access to data for analytics and machine learning.
  • Troubleshoot and optimize query performance in Redshift Spectrum, and manage workload performance in the data lake environment.
  • Implement backup, recovery, and disaster recovery strategies for data lake assets, ensuring data integrity and availability.
  • Maintain detailed documentation of data lake architecture, security policies, and data governance procedures.

Qualifications:

  • 4-8 years of experience in managing data lake environments, with a focus on AWS data services.
  • Strong expertise in AWS services like S3, Lake Formation, Glue, Glue Data Catalog, Redshift Spectrum, and Lambda.
  • Good knowledge of SQL and query optimization techniques, particularly in querying data from S3 using Redshift Spectrum and Athena.
  • Experience in data ingestion pipelines using AWS Glue ETL, Lambda, and other AWS orchestration tools.
  • Strong understanding of data governance, access control, and security best practices in AWS environments.
  • Experience with data formats (Parquet, ORC, Avro, JSON) and optimizing data storage for cost and performance.
  • Knowledge of data partitioning, compression, and indexing strategies in S3.
  • Familiarity with monitoring, troubleshooting, and optimizing AWS Data Lake performance.
  • Excellent communication and collaboration skills to work across teams and support business requirements.

AWS certifications, such as AWS Certified Big Data – Specialty or AWS Certified Solutions Architect, are a plus.

  • AWS Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Talent Worx Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Senior AWS Data EngineerOverview:We are seeking experienced AWS Data Engineers to design, implement, and maintain robust data pipelines and analytics solutions using AWS services. The ideal candidate will have a strong background in AWS data services, big data technologies, and programming languages.Exp- 3 to 7 years Location- Bangalore, Pune, Hyderabad,...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,25,000 - US$ 1,75,000

    Job OverviewWe are seeking a highly skilled AWS Data Engineer to join our team. As a key member of our engineering department, you will be responsible for designing and implementing scalable, high-performance data pipelines using AWS services.About the RoleThe successful candidate will have experience with AWS data services, including Glue, EMR, Lambda,...

  • AWS Sr Data Engineer

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a AWS Sr Data Engineer to join our team in Bangalore or Remote, Karnātaka (IN-KA), India (IN). Position Location:OUS with...

  • AWS Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India CareerXperts Consulting Full time

    About the Role: We are seeking a highly skilled AWS Data Engineer to design, build, and optimize large-scale data pipelines and analytics platforms. The ideal candidate will have strong hands-on expertise in AWS Glue, Athena, Redshift, PySpark, and SQL, with a solid understanding of data engineering and data warehousing concepts. You will work on batch and...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India CareerXperts Consulting Full time

    About the Role:We are seeking a highly skilled AWS Data Engineer to design, build, and optimizelarge-scale data pipelines and analytics platforms. The ideal candidate will havestrong hands-on expertise in AWS Glue, Athena, Redshift, PySpark, and SQL,with a solid understanding of data engineering and data warehousing concepts.You will work on batch and...

  • AWS Big data Architect

    3 weeks ago

    Bengaluru, Karnataka, India Birlasoft Limited Full time

    Job DescriptionKey Responsibilities- Design and architect end to end solutions on AWS and create the HLD and LLD documents- Experience in real-time Data Ingestion and Processing- Hands-on experience in AWS S3, Kafka, Glue, PySpark, Python, Redshift, AWS Cloud best practices and optimization- Experience with integration of different data sources with Data...

  • Expert - Aws Redshift Operations

    3 weeks ago

    Bengaluru, Karnataka, India Schneider Electric Full time

    Schneider Electric is an Equal Opportunity Employer It is our policy to provide equal employment and advancement opportunities in the areas of recruiting hiring training transferring and promoting all qualified individuals regardless of race religion color gender disability national origin ancestry age military status sexual orientation ...

  • AWS Data Engineer

    4 days ago

    Bengaluru, Karnataka, India TECHRACERS PRIVATE LIMITED Full time ₹ 1,20,000 - ₹ 1,80,000 per year

    Profile : AWS Data Engineer Mandate Skills : AWS + Databricks + Pyspark + SQL roleLocation : Bangalore /Pune /Hyderabad /Chennai /GurgaonNotice Period : ImmediateKey Responsibilities : - Design, build, and maintain scalable data pipelines to collect, process, and store data from multiple datasets - Optimize data storage solutions for performance,...

  • Aws Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Devon Software Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for an experienced Data Engineer with 4+ years of expertise in building scalable data pipelines on AWS. The ideal candidate should have strong knowledge of ETL processes, data modeling, and modern data platforms like Snowflake and Redshift.Key Skills Required:Strong hands-on experience with AWS services: EC2, S3, IAM, Glue, DMSExperience with...

  • AWS Big Data Engineer

    2 hours ago

    Bengaluru, Karnataka, India The IT Mind services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design, build, and maintain data pipelines on the AWS platform.Work with AWS services like S3, Glue, EMR, and Redshift.Process and analyze large datasets to support business insights.Ensure data quality, integrity, and security in the data lake.Location - Pan India.