AWS Data Engineer

Role: Data Engineer II

Location Offshore Bengaluru-Hybrid 2-3 Days onsite a week

Duration 6 Months can be extended

Our focus is on SQL, AWS, and Python.

Data Engineer II

Our Data & Analytics team focuses on designing, building, and optimizing data warehousing and data lake solutions that power a range of business analytics activities—including sales, finance, operations, and more. By gathering and analyzing data from diverse enterprise systems and operational sources, we aim to identify trends, improve operational efficiency, and derive insights that drive strategic decision-making. Our goal is to equip stakeholders with timely, accurate, and actionable information to navigate complex market dynamics and maintain a competitive edge. Through ongoing monitoring and refinement of key business metrics, we play a critical role in shaping the company's data strategy, optimizing business outcomes, and driving overall success.

In this role, you will design and implement robust data pipelines using AWS, Python, and SQL to underpin comprehensive business analytics and data-driven decision-making processes. As a Data Engineer, you will ensure the integrity, security, and performance of data processing and storage systems, while also creating and managing data lakes and data warehouses that support various analytical needs. You will collaborate with cross-functional teams—spanning sales, finance, operations, and product groups—to gather requirements and address complex data challenges. Your responsibilities will include optimizing data flows within AWS, developing scalable ETL processes and APIs, troubleshooting data issues, and documenting solutions for compliance and future reference. You will also play a key role in assessing new technologies, continually improving existing systems, and supporting both daily operations and long-term strategic initiatives within the data management team.

Responsibilities

• Design and implement robust data pipelines using AWS, Python, and SQL to support business analytics and decision-making across multiple domains (e.g., sales, finance, operations).
• Ensure data integrity and performance of data processing and storage systems.
• Optimize data flows and architecture within AWS to enhance performance, scalability, and reliability.
• Develop and maintain scalable and efficient ETL processes and APIs for data interaction.
• Collaborate with cross-functional teams to gather and address technical and design requirements.
• Maintain and enhance data models and databases to support diverse enterprise applications.
• Monitor and troubleshoot operational or data issues in the data pipelines.
• Continuously improve systems and processes by integrating new tools and technologies.
• Document all data engineering procedures and solutions for future reference and compliance.
• Support the data management team in daily operations and long-term projects.

Requirements

• Expertise in AWS services such as S3, EC2, RDS, Redshift, and AWS Glue.
• Proficiency in Python, with a solid understanding of libraries and frameworks for data engineering (e.g., Pandas, PySpark).
• Strong command of SQL for creating, managing, and optimizing data structures and complex queries.
• Familiarity with orchestration tools (e.g., Airflow) and data integration techniques.
• Understanding of data warehousing solutions and ETL processes.
• Knowledge of data modeling and database design.
• Ability to write clean, maintainable code, adhering to best practices in code quality and security.
• Experience with CI/CD pipelines for data operations in an AWS environment is a plus.
• Good grasp of Agile methodologies and project management practices.

Qualifications

• Minimum of Two years of experience in data engineering or a related field.
• Strong proficiency in building and maintaining data pipelines in AWS environments.
• Experience with Python for scripting and data manipulation.
• Advanced SQL knowledge for complex querying and database interaction.
• Bachelor's degree in computer science, Engineering, or a related field is preferred; equivalent practical experience is also considered.
• Demonstrated ability to design and implement effective data warehousing and analytics solutions in cloud environments.