Senior Data Engineer

Job Purpose

The Senior Data Engineer will be responsible for designing, building, and maintaining scalable and efficient data pipelines and architectures for the Enterprise Data Platform. This role will focus on enabling high-quality, reliable, and timely data access for analytics, reporting, and business decision-making. Working closely with business analysts, data scientists, and architects, the Senior Data Engineer will ensure data solutions meet business needs and adhere to best practices and governance standards

Duties and Responsibilities

• Design and implement robust, scalable, and high-performance data pipelines and ETL/ELT processes.
• Develop, optimize, and maintain data architectures including databases, data lakes, and data warehouses.
• Ensure the quality, integrity, and security of data through robust data validation and data quality frameworks.
• Collaborate with business analysts and stakeholders to understand business data requirements and translate them into technical designs.
• Work closely with data architects to align with enterprise architecture standards and strategies.
• Implement data integration solutions with various internal and external data sources.
• Monitor, troubleshoot, and optimize system performance and data workflows.
• Support the migration of on-premise data solutions to cloud-based environments (e.g., AWS, Azure, GCP).
• Stay up to date with the latest industry trends and technologies in data engineering and recommend innovative solutions.
• Create and maintain comprehensive documentation for all developed data pipelines and systems.
• Mentor junior data engineers and contribute to the development of best practices.

Key Decisions / Dimensions

• Selecting appropriate technologies, tools, and frameworks for data pipeline development.
• Designing data models and database schemas that optimize for both performance and scalability.
• Establishing standards for code quality, data validation, and monitoring processes.
• Identifying performance bottlenecks and recommending architectural improvements.

Major Challenges

• Managing and processing large volumes of structured and unstructured data with efficiency.
• Designing systems that can handle scaling needs as business requirements and data volumes grow.
• Balancing the need for quick delivery with the necessity for scalable and maintainable code.
• Ensuring data quality and compliance with data governance and security policies.
• Integrating disparate data sources with differing formats and standards into unified models.

Required Qualifications and Experiencea)Qualifications

• Bachelors Degree in Computer Engineering, Computer Science, Information Technology, or a related field.
• Professional certifications such as Google Professional Data Engineer, AWS Certified Data Analytics Specialty, or Microsoft Certified: Azure Data Engineer Associate are a plus.b)Work Experience
• Minimum of 4+ years of experience in data engineering or a related role.
• Strong expertise in building and optimizing ETL/ELT pipelines and data workflows.
• Proficient in programming languages such as Python, Java, or Scala.
• Hands-on experience with SQL and relational database systems (e.g., PostgreSQL, SQL Server, MySQL).
• Experience with big data technologies (e.g., Hadoop, Spark, Kafka).
• Familiarity with cloud platforms (AWS, Azure, GCP) and cloud-native data services (e.g., Redshift, BigQuery, Snowflake, Databricks).
• Solid understanding of data modeling, data warehousing concepts, and best practices.
• Knowledge of CI/CD pipelines and infrastructure-as-code (IaC) is a plus.
• Strong problem-solving s and the ability to work independently or in a team. c)s Keywords
• Data Architecture
• Delivery Management
• Project Management
• Cloud Data Platforms (e.g., Azure, AWS, GCP)
• Data Modeling
• Data Governance
• Stakeholder Management
• Quality Assurance
• Agile Methodology
• Team Leadership
• Budget Management
• Risk Management
• Data IntegrationScalable Data Solutions