Azure Data Engineer

Job Summary:

We are looking for a talented and experienced Azure Data Engineer to join our data engineering team in Bangalore. As an Azure Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines, and working with Azure-based data services to ensure the seamless flow of data for analytical and operational purposes. The ideal candidate will have a strong understanding of Azure data platforms, data engineering practices, and be proficient in creating and optimizing data solutions using Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, and other related Azure services.

Key Responsibilities:

• Data Pipeline Development: Design, develop, and maintain efficient and scalable data pipelines in Azure Data Factory (ADF) to integrate data from multiple sources, process data, and load it into target systems.
• Data Integration: Work with various data sources like SQL, NoSQL, Blob Storage, and data lakes to ingest and process data in an efficient and scalable manner.
• Azure Data Services: Utilize Azure Synapse Analytics, Azure Data Lake, Azure SQL Database, Cosmos DB, and other Azure services to build and optimize data architecture and storage solutions.
• ETL/ELT Development: Implement both ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes using Azure tools, ensuring data is processed accurately and efficiently.
• Data Warehousing: Design and maintain cloud-based data warehousing solutions using Azure Synapse Analytics (formerly Azure SQL Data Warehouse) to support reporting, analytics, and data-driven decision-making.
• Performance Tuning: Optimize performance of data pipelines and processes, ensuring they run efficiently, and troubleshoot performance issues.
• Data Modeling & Management: Develop data models for large datasets, ensuring proper schema design, partitioning, indexing, and data governance.
• Automation & Monitoring: Set up automated processes for data pipeline monitoring, error handling, and logging to ensure high data quality and reliability.
• Security & Compliance: Ensure that the data solutions are secure and comply with data privacy regulations (e.g., GDPR, HIPAA) using Azure Security features and best practices.
• Collaboration: Collaborate with data scientists, business analysts, and stakeholders to understand data needs and ensure alignment of data engineering solutions with business objectives.
• Documentation: Maintain comprehensive documentation for data architecture, pipelines, data models, and best practices.

Required Skills & Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent experience).
• 5+ years of experience in data engineering, with at least 3 years of hands-on experience working with Azure data platforms.
• Strong proficiency in Azure Data Factory (ADF), Azure Synapse Analytics, Azure SQL Database, and Azure Data Lake.
• Solid understanding of ETL and ELT processes and experience with building data pipelines on the cloud.
• Proficient in SQL and experience with query optimization and performance tuning.
• Experience with Azure Databricks, HDInsight, Cosmos DB, and other Azure-based data technologies is a plus.
• Familiarity with Data Lake architectures and the ability to implement solutions for large-scale data processing.
• Strong knowledge of data modeling, data warehousing, and data governance practices.
• Experience working with cloud storage solutions, including Blob Storage and Data Lake Storage.
• Knowledge of Power BI, Azure Analysis Services, or similar tools for data visualization and reporting is a plus.
• Experience with Python, Spark, or Scala for data processing is highly desirable.
• Familiarity with DevOps practices and tools (e.g., Azure DevOps) for data engineering deployments is a bonus.
• Understanding of data security and compliance frameworks in cloud environments.

Preferred Skills (Nice to Have):

• Azure Certification such as Microsoft Certified: Azure Data Engineer Associate.
• Experience with machine learning models and AI workflows integrated with Azure data services.
• Exposure to Big Data technologies such as Hadoop, Spark, or Kafka.
• Familiarity with Agile development methodologies.
• Knowledge of CI/CD pipelines for automating deployments of data engineering solutions.
• Experience with DataOps for better management of data pipelines.

Job Type: Full-time

Pay: ₹900, ₹1,200,000.00 per year

Work Location: In person