Senior Data Engineer

We are seeking an experienced and highly motivated Senior Data Engineer to join our dynamic team. The ideal candidate will play a critical role in designing, developing, and maintaining high-performance data infrastructure and pipelines.

As a Senior Data Engineer, you will be responsible for ensuring the scalability, performance, and reliability of data systems that support business intelligence, analytics, and data science initiatives across the organization.

You will work in a collaborative environment with cross-functional teams to translate business needs into technical solutions and help shape the organizations overall data strategy.

This role offers an excellent opportunity to work with modern data engineering tools and frameworks, while continuously learning and contributing to best practices in the data domain.

Key Responsibilities :

- Design, develop, and maintain scalable and reliable data pipelines for batch and real-time processing.

- Architect and implement robust Data Lake, Data Warehouse, and Lakehouse solutions using various architectural patterns.

- Ensure data quality, data integrity, and governance across the entire data lifecycle.

- Collaborate with Data Architects, Data Scientists, Analysts, and other engineering teams to translate business requirements into technical specifications.

- Monitor and fine-tune the performance of data pipelines and processing systems.

- Implement and maintain data versioning, lineage, and metadata management systems.

- Develop and document best practices, standards, and guidelines for data engineering activities.

- Mentor and support junior data engineers to foster professional development and a strong technical culture.

- Evaluate and integrate emerging technologies and frameworks to enhance data engineering capabilities.

- Provide technical leadership in designing and building robust, scalable data platforms.

Required Qualifications :

- Bachelors degree in Computer Science, Information Systems, or a related field. A Masters degree is preferred.

- Minimum of 8 years of professional experience in data engineering or a related field.

- Proven experience in building scalable data pipelines and architectures for batch and real-time processing.

- Hands-on experience with AWS Data Services such as S3, Glue, Athena, EMR, Kinesis etc.

- Strong command over SQL and experience working with file formats such as Apache Parquet, Delta Lake, Apache Iceberg, or Apache Hudi.

- Familiarity with Change Data Capture (CDC) mechanisms and related patterns.

- Experience with stream processing frameworks such as Apache Flink, Kafka Streams, or Apache Spark/pySpark.

- Expertise in workflow orchestration and scheduling tools such as Apache Airflow.

- Solid understanding of data warehousing concepts and both relational and NoSQL databases.

- Proficiency in data integration tools and techniques, with a strong grasp of ETL/ELT pipelines.

- Experience working in cloud-native environments, with emphasis on AWS.

- Demonstrated analytical and problem-solving skills with the ability to work independently and handle multiple tasks effectively.

- Excellent verbal and written communication skills, with the ability to communicate complex technical concepts to non-technical stakeholders.

- Passion for staying updated with the latest trends and advancements in the data engineering space.

Preferred Skills:

- Experience with data cataloging, data governance, and security frameworks.

- Familiarity with modern data architecture principles, including data mesh or data fabric.

- Working knowledge of CI/CD practices for data workflows.

- Contributions to open-source data engineering tools or frameworks is a plus.

Why Join Us:

- Work on challenging, high-impact data engineering projects with modern technologies.

- Opportunity to shape the future of the companys data strategy.

- Collaborate with a diverse and talented team of engineers, analysts, and scientists.

- Continuous learning and career growth in a technology-driven environment.