Data Engineer

Overview of 66degrees

66degrees is a leading Google Cloud Premier Partner, though this specific role focuses on our Microsoft Azure projects. We believe that great engineering takes heart, and we are dedicated to building innovative data solutions that guide our clients on their digital transformation journey.

Role Description

As an Azure Data Engineer, you will be responsible for building and maintaining robust, scalable data platforms. You will work with a variety of Azure services to deliver raw data into analytics-ready assets and support a secure, efficient data environment. This role requires strong hands-on engineering skills to implement data pipelines, manage data integrity, and ensure compliance with security and governance standards.

Responsibilities

- Data Pipeline Development: Design, build, and maintain robust and scalable data pipelines to ingest, transform, and load data from various sources into the Azure data platform. This includes creating and managing jobs, dataflows, and triggers in Azure Data Factory and Azure Synapse Pipelines.
- Data Modeling and Warehousing: Develop and implement conceptual, logical, and physical data models for data warehouses and data lakes. Work with star schema, snowflake schema, and other data modeling techniques to optimize data for analytics and reporting.
- Data Lake Management: Manage data storage and organization within Azure Data Lake Storage Gen2. Ensure proper folder structures, security, and data lifecycle management for raw, processed, and curated data zones.
- Data Transformation: Utilize tools like dbt (data build tool) within the Azure ecosystem to transform raw data into analytics-ready datasets. Write complex SQL queries and stored procedures to cleanse, enrich, and aggregate data.
- Azure Service Integration: Integrate and work with a variety of Azure services, including Azure Synapse Analytics, Azure Databricks, Azure Functions, and Cosmos DB, to build a comprehensive data solution.
- ETL/ELT Processes: Implement and optimize Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes to ensure data is moved efficiently and accurately.
- Security and Governance: Implement and maintain data security best practices. Apply Role-Based Access Control (RBAC), managed identities, and Azure Key Vault to protect sensitive data. Work with Microsoft Purview for data cataloging and governance.
- Performance Optimization: Monitor and optimize the performance of data pipelines, queries, and data warehouse solutions. Fine-tune data structures and processing logic to improve efficiency and reduce costs.

Qualifications

- 4+ years of proven experience with key Azure data services, including Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, and Azure Databricks.
- Strong proficiency in SQL and at least one programming language such as Python or Spark (PySpark, Scala).
- Solid understanding of data warehousing principles, data modeling (dimensional modeling, etc.), and ETL/ELT concepts.
- CI/CD: Experience with version control systems like Git and Continuous Integration/Continuous Deployment (CI/CD) practices using Azure DevOps.
- Ability to work effectively with cross-functional teams, including data scientists, business analysts, and product owners, to understand requirements and deliver data solutions.
- Excellent analytical and problem-solving skills with a focus on delivering high-quality, reliable data solutions.
- Strong verbal and written communication skills to articulate technical concepts to both technical and non-technical stakeholders.

66degrees is an Equal Opportunity employer. All qualified applicants will receive consideration for employment without regard to actual or perceived race, color, religion, sex, gender, gender identity, national origin, age, weight, height, marital status, sexual orientation, veteran status, disability status, or other legally protected class.