Senior Data Engineer

Senior Data Engineer

Job Title: Senior Data Engineer

Location: Bengaluru

Job Type: Full-Time

Experience Required: 5–10 Years

Department: Data Engineering / Analytics

About the Role:

We are seeking a highly skilled and experienced Senior Data Engineer to lead the design, development, and maintenance of scalable and resilient data infrastructure and pipelines using Microsoft Azure technologies. This role involves building robust, automated data transformation systems and collaborating with cross-functional teams to support a wide range of analytical and business intelligence use cases.

Key Responsibilities:

• Architect and oversee the entire data infrastructure to ensure scalability , performance , and resiliency .
• Design, develop, and maintain automated data pipelines and ETL processes using Microsoft Azure services such as Azure Data Factory , Azure Synapse Analytics , Azure Databricks , and Azure Fabric .
• Organize and manage pipeline outputs using Azure Data Lake Storage Gen2 and Azure Blob Storage .
• Mentor and guide junior data engineers to foster skill development and ensure best practices.
• Work closely with data scientists , analysts , data architects , and business stakeholders to gather requirements and deliver high-quality data solutions.
• Optimize data processing workflows for cost efficiency , performance , and scalability in the Azure environment.
• Implement data validation frameworks to ensure the quality, accuracy, and consistency of data.
• Manage and document data pipeline architectures, data flow diagrams, configurations, and technical standards.
• Monitor production workflows and troubleshoot issues to maintain high availability and data reliability.
• Stay up to date with emerging tools , technologies , and industry best practices in data engineering and cloud computing.
• Lead the implementation of CI/CD practices for data solutions to streamline deployment and maintenance workflows.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field.
• 5–10 years of hands-on experience in data engineering , with a proven track record of building and managing scalable data pipelines and cloud-based data solutions.
• Strong proficiency with Microsoft Azure data tools, including:
• Azure Data Factory
• Azure Synapse Analytics
• Azure Databricks
• Azure Fabric
• Azure Data Lake Storage Gen2 & Blob Storage
• Solid understanding of data modeling , ETL frameworks , and big data architecture patterns.
• Proficiency in Python , SQL , and scripting for data pipeline development and automation.
• Experience with CI/CD tools and practices for deploying data infrastructure (e.g., Azure DevOps, Git).
• Strong problem-solving abilities and attention to detail.
• Excellent collaboration and communication skills to work effectively in cross-functional teams.
• Demonstrated ability to mentor and lead junior engineers.

Preferred Qualifications:

• Experience working with structured and unstructured data at scale.
• Familiarity with real-time streaming technologies (e.g., Azure Event Hubs, Kafka).
• Knowledge of data governance , data privacy , and compliance standards.
• Exposure to DevOps practices, including infrastructure as code (e.g., Terraform, Bicep).

Why Join Us?

• Work with cutting-edge technologies and enterprise-grade data infrastructure.
• Join a culture of innovation and continuous learning.
• Lead high-impact projects in a collaborative, agile team environment.
• Competitive compensation and career growth opportunities.