Lead Data Engineer

Job Name : Lead Data Engineer - AWS RedShift

Years of Experience : 8

Job Description :

We are seeking a highly skilled and motivated Lead Data Engineer to join our Data Integration and Analytics team. The ideal candidate will bring deep expertise in AWS Redshift, with strong knowledge of Redshift architecture, query performance tuning, and workload optimization. You will be responsible for leading the design, development, and optimization of scalable ETL/ELT pipelines that enable efficient data integration and power critical business intelligence and analytics initiatives.

This role requires a proven ability to troubleshoot complex data challenges, drive performance improvements, and ensure our data ecosystem is both scalable and cost-efficient. A passion for continuous learning, problem-solving, and enabling data-driven decision-making is highly valued.

Primary Skills : AWS, ETL Concepts,Data Integration Tool (Any)

Secondary Skills : Python, Datastage, Informatica IICS, Informatica, SSIS

Role Description :

The Lead Data Engineer will be responsible for overseeing the design, development, and maintenance of our data infrastructure and pipelines. This role requires strong technical expertise, leadership skills, and the ability to work collaboratively with cross-functional teams to ensure efficient and reliable data processes.

Role Responsibility :

- Lead the design and architecture of scalable and robust data integration solutions on-premise and in the cloud. Ensure solutions meet business requirements and industry standards.

- Engage with executive leadership and key stakeholders to understand business needs and translate them into technical solutions.

- Define data models, schemas, and structures to optimize data storage, retrieval, and processing for analytical workloads.

- Work with data architects and solution architects to ensure alignment with overall data strategy and architecture principles.

- Lead a team of data engineers by providing technical guidance, mentorship, and support.

- Plan, prioritize, and manage data engineering projects, tasks, and timelines.

- Design, develop, and maintain data integration pipelines and ETL/ELT workflows.

- Lead more than one project or manage a larger team that might have sub-tracks.

Role Requirement :

- Proficient in basic and advanced SQL.

- Proficient in using Python for data integration and data engineering tasks.

- Proficiency in ETL/ELT tools such as Informatica, Talend, DataStage, SSIS, DBT, Databricks or equivalent.

- Experience with relational databases (like SQL Server, Oracle, MySQL, PostgreSQL) and NoSQL databases (like MongoDB, Cassandra), cloud databases (RedShift, Snowflake, Azure SQL).

- Familiarity with big data technologies like Hadoop, Spark, Kafka, and cloud platforms such as AWS, Azure, or Google Cloud.

- Solid understanding of data modeling, data warehousing concepts, and practices.

- Good Knowledge and Understanding of Data warehouse concepts (Dimensional Modeling,

change data capture, slowly changing dimensions etc.).

- Knowledgeable in performance tuning and optimization

- Experience in Data Profiling and Data validation

- Experience in requirements gathering and documentation processes and performing unit testing.

- Understanding and Implementing QA and various testing process in the project.

- Knowledge in any BI tools will be an added advantage.

- Sound aptitude, outstanding logical reasoning, and analytical skills.

- Willingness to learn and take initiatives.

- Ability to adapt to fast-paced Agile environment.

- Relevant certifications in data engineering or cloud platforms are a plus.

Additional Requirement :

- Overall 8+ years in ETL/ELT (preferably Informatica IICS/PowerCenter) and 4+ years of hands-on experience with AWS RedShift.

- Strong proficiency in AWS Redshift and writing complex SQL queries.

- Solid programming experience in Python for scripting, data wrangling, and automation.

- Experience with version control tools like Git and CI/CD workflows.

- Knowledge of data modeling and data warehousing concepts.

- Prior experience with data lakes and big data technologies is a plus.

---