AWS Data Engineer with Dynamo DB Experience

Job Title: Cloud Data Engineer with Database Expertise

Location: Bengaluru, Karnataka

About the Company:

We are a global organisation targeting various international markets. Our focus is on lifecycle management of end-to-end net-zero transitions while leveraging an automation-first approach. We offer collaborative services and counsel our clients on transition solutions and decarbonisation strategies.

Role Summary:

We are seeking a highly skilled and experienced data engineer to join our team. The ideal candidate should have a strong background in cloud services, including Glue, EMR, Lambda, and expertise in data modelling, Python, SQL, PartiQL and integrating third-party APIs from scratch.

Key Responsibilities:

• Design, develop, and implement end-to-end data engineering solutions from ground up on the AWS platform using services such as Glue, EMR, Lambda, and more.
• Design and implement scalable databases, optimizing performance and ensuring data consistency, throughput, and storage efficiency.
• Collaborate with cross-functional teams and stakeholders to gather data requirements and understand business objectives.
• Architect and implement data models and schemas that align with best practices and optimise query performance.
• Build robust and scalable data pipelines to extract, transform, and load (ETL) data from diverse sources into target systems.
• Integrate third-party APIs to retrieve and ingest data from external systems, ensuring data accuracy and reliability.
• Perform data profiling, data cleansing, and data quality checks to ensure accuracy and consistency.
• Optimise SQL queries and database performance, leveraging SQL tuning techniques and indexing strategies.
• Implement data governance and security measures to protect sensitive data throughout the pipeline.
• Develop and maintain documentation for data pipelines, data flows, and data transformations.
• Stay up-to-date with industry trends and emerging technologies in data engineering, cloud services, and third-party API integrations.
• Mentor junior team members and provide technical guidance and support as needed.
• Manage and analyse geospatial data, implementing solutions for storing, retrieving, and processing location-based information while integrating geospatial indexing and query optimisation techniques.

Education and Experience:

• Bachelor's degree in Computer Science or a related field.
• Minimum of 2+ years of experience in data engineering, specifically in building and maintaining data pipelines from scratch
• Strong proficiency in cloud services such as Glue, EMR, Lambda, Dynamo DB, Postgres.
• Expertise in data modelling, schema design, and relational and non-relational database concepts.
• Solid working knowledge of SQL, SQL tuning, and data modelling best practices.

Skills:

• Experience in designing and implementing single-table architecture in DynamoDB, optimising data models for scalability, efficient query patterns, and minimal read/write costs.
• Proficiency in using PartiQL to query and manipulate data in DynamoDB, with a focus on optimising queries for performance and integrating complex business logic.
• Experience in developing and optimising SQL queries for performance.
• Proficiency in Python programming for data manipulation, transformation, and scripting tasks.
• Familiarity with data warehouse concepts, OLAP, and Big Data technologies.
• Experience in integrating third-party APIs to retrieve and ingest data from external systems.

Personal Attributes:

• A team player with good interpersonal skills and problem-solving abilities.
• A self-motivator and a collaborative team player.
• Excellent time management skills and the ability to assume leadership roles when necessary.
• Agile, action-oriented and a quick learner.
• Out-of-the-box thinker.

Benefits:

• A flat hierarchy.
• ESOPs.
• Group Medical Coverage.
• 30 days of Leave.
• A gender-neutral parental, marriage, and compassionate leave policy.

And above all, if you have a passion to work, we invite you to be part of our sustainability journey.