Azure Data Engineer

Job Description Black And White Business Solutions is actively seeking a highly skilled Azure Data Engineer to join our data team. This pivotal role requires extensive experience in data modeling with a strong focus on cloud-based environments, particularly Azure. The ideal candidate will possess robust proficiency in SQL, Spark, and Python, with proven expertise in Azure Databricks, Azure Data Lake, and other Azure data services. You'll be instrumental in designing, implementing, and optimizing scalable data pipelines, specifically for migrating complex data from various source systems into the Azure cloud. Key Responsibilities - Design, implement, and optimize data models within Azure Databricks to support large-scale data migration projects from diverse source systems like Salesforce, Oracle, and SQL Server-based applications. - Collaborate effectively with business stakeholders, data architects, and data engineers to thoroughly understand data requirements and translate them into scalable, high-performance data models. - Develop conceptual, logical, and physical data models to ensure data consistency, integrity, and optimal performance within the Azure cloud environment. - Work closely with the data engineering team to ensure smooth integration of data from Salesforce, Oracle, and SQL Server into Azure Databricks using robust ETL pipelines. - Optimize and fine-tune data models for handling large datasets, focusing on performance, scalability, and cost efficiency within the Azure platform. - Ensure stringent data governance, security, and compliance standards are met when designing and implementing data models and pipelines. - Develop and maintain comprehensive documentation for data models, including entity relationships, attributes, transformations, and critical business rules. - Implement best practices for data modeling, including adherence to naming conventions, appropriate data types, and effective normalization/denormalization strategies. - Collaborate with data scientists and analysts to ensure the data models effectively support business intelligence, reporting, and machine learning initiatives. - Conduct thorough data quality assessments and validate the accuracy and completeness of migrated data. - Troubleshoot, resolve, and optimize data model-related performance and data quality issues across the entire data migration process. Required Skills and Qualifications - Data Modeling Experience: 5+ years of hands-on experience in data modeling, with a strong focus on cloud-based environments, preferably Azure. - Programming & Data Processing: Strong proficiency in SQL and extensive experience working with Spark, Python, and other relevant data processing languages. - Azure Data Services: Proven expertise with Azure Databricks, Azure Data Lake, and other critical Azure data services (e.g., Azure Synapse Analytics, Azure Data Factory). - Data Migration Expertise: Strong experience in migrating complex data from various source systems, including Salesforce, Oracle, and SQL Server-based applications, into cloud environments, specifically Azure. - Data Modeling Design: Expertise in designing and optimizing relational and dimensional data models (e.g., star/snowflake schemas, effective use of normalization/denormalization techniques). - ETL & Data Integration: In-depth understanding of ETL (Extract, Transform, Load) processes, robust data integration patterns, and pipeline optimization techniques. - Data Governance & Security: Familiarity with data governance principles, security policies, and compliance standards within cloud environments. - Communication & Collaboration: Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams and convey complex technical concepts clearly. - Requirement Translation: Ability to translate complex business requirements into scalable and optimized data models and technical specifications. - Problem-Solving: Strong problem-solving and analytical skills to troubleshoot and optimize complex data modeling issues and ensure data integrity. Qualification - Any Graduate or Above Relevant Experience - 6 to 15 Years CTC Range - 20 LPA to 35 LPA Notice Period - Any Mode of Interview - Virtual Contact: Vardhani IT Staffing Analyst Black and White Business Solutions Pvt Ltd Bangalore, Karnataka, INDIA. Mobile: 8686127477 | Email: [Confidential Information] | Website: www.blackwhite.in