Data engineer

The Development Lead will oversee the design, development, and delivery of advanced data solutions using Azure Databricks, SQL, and data visualization tools like Power BI. The role involves leading a team of developers, managing data pipelines, and creating insightful dashboards and reports to drive data-driven decision-making across the organization. The individual will ensure best practices are followed in data architecture, development, and reporting while maintaining alignment with business objectives.

Key Responsibilities:

Data Integration & ETL Processes: Design, build, and optimize ETL pipelines to manage the flow of data from various sources into data lakes, data warehouses, and reporting platforms.

Data Visualization & Reporting: Lead the development of interactive dashboards and reports using Power BI, ensuring that business users have access to actionable insights and performance metrics.

SQL Development & Optimization: Write, optimize, and review complex SQL queries for data extraction, transformation, and reporting, ensuring high performance and scalability across large datasets.

Azure Cloud Solutions: Implement and manage cloud-based solutions using Azure services (Azure Databricks, Azure SQL Database, Data Lake) to support business intelligence and reporting initiatives.

Collaboration with Stakeholders: Work closely with business leaders and cross-functional teams to understand reporting and analytics needs, translating them into technical requirements and actionable data solutions.

Quality Assurance & Best Practices: Implement and maintain best practices in development, ensuring code quality, version control, and adherence to data governance standards.

Performance Monitoring & Tuning: Continuously monitor the performance of data systems, reporting tools, and dashboards to ensure they meet SLAs and business requirements.

Documentation & Training: Create and maintain comprehensive documentation for all data solutions, including architecture diagrams, ETL workflows, and data models. Provide training and support to end-users on Power BI reports and dashboards.

Required Qualifications:

Bachelors or Masters degree in Computer Science, Information Systems, or a related field.

Proven experience as a Development Lead or Senior Data Engineer with expertise in Azure Databricks, SQL, Power BI, and data reporting/visualization.

Hands-on experience in Azure Databricks for large-scale data processing and analytics, including Delta Lake, Spark SQL, and integration with Azure Data Lake.

Strong expertise in SQL for querying, data transformation, and database management.

Proficiency in Power BI for developing advanced dashboards, data models, and reporting solutions.

Experience in ETL design and data integration across multiple systems, with a focus on performance optimization.

Knowledge of Azure cloud architecture, including Azure SQL Database, Data Lake, and other relevant services.

Experience leading agile development teams, with a strong focus on delivering high-quality, scalable solutions.

Strong problem-solving skills, with the ability to troubleshoot and resolve complex data and reporting issues.

Excellent communication skills, with the ability to interact with both technical and non-technical stakeholders.

Preferred Qualifications:

Knowledge of additional Azure services (e.g., Azure Synapse, Data Factory, Logic Apps) is a plus.

Experience in Power BI for data visualization and custom calculations.