Azure Data Engineer

About CleanHarbors
Clean Harbors Inc. ) is a NYSE listed US based $5.6 billion company. Clean Harbors was founded in 1980 near Boston by Mr. Alan S. McKim, who today remains the company's Chairman and CTO. Everywhere industry meets environment, Clean Harbors is on-site, providing premier environmental, energy and industrial services. Clean Harbors employs a dedicated workforce of 22,000 employees and has over 450 service locations across the U.S., Canada and Mexico. Beyond addressing the environmental needs of its more than 300,000 customers, the Company has been on the front lines of addressing some of the largest emergency response events in North America of the past several decades including the BP Oil Spill in the Gulf of Mexico, the anthrax attacks in New York, the 2015 avian flu outbreak that devastated the poultry industry, Hurricanes Katrina, Rita and Sandy, and the oil pipeline breaks that affected the Yellowstone and Kalamazoo rivers, along with other major chemical releases into the environment.

Clean Harbors India has been strategically established as the Global Capability Center (GCC) to serve our parent company. We take pride in serving some of the most tech savvy business stakeholders at Clean Harbors. The GCC employees form part of our globally integrated teams that provide cutting-edge IT solutions and production support across platforms. Our shared services teams work seamlessly with their global counterparts in providing critical 24x7 support in various functions including Finance, HR, Procurement, IT and Operations. Our aggressive growth plans open significant career advancement opportunities for our employees

Responsibilities
The Data Engineer will be responsible for designing, implementing, and managing data solutions on the Microsoft Azure cloud platform.This role involves working with various data storage options offered by Microsoft Azure, developing, and implementing data integration processes, and using big data technologies to handle and analyze large volumes of data for multiple purposes. In addition, the role requires supporting AI implementations and Power BI reporting.

• Ensure that Health and Safety is the number one goal by following policies, processes, and always acting in a safe manner.
• Design data storage systems that scale, perform, and are economical enough to satisfy the organization's requirements.
• Develop and implement data integration processes, frequently using Azure Data Factory, to extract, transform, and load (ETL) data from various sources into Azure data warehouses or data lakes.
• Use big data technologies like Apache Spark to handle and analyze enormous volumes of data.
• Collaborate with data scientists to create data sets and engineer features for AI Initiatives
• Implement and optimize data retrieval for Power BI solutions.

Qualifications

• A master's degree in computer science, Statistics, Mathematics, or a related field is required. A bachelor's degree with significant relevant experience would suffice.
• 5+ years of data engineering experience leveraging technologies such as Azure Data Factory, Azure Synapse Analytics, Azure Stream Analytics, Azure Event Hubs, Azure Data Lake Storage, Azure Databricks
• Experience with data pipelines and engineering infrastructure: You should have experience designing and implementing data pipelines and engineering infrastructure for machine learning systems.
• Solid knowledge of data processing languages, including SQL, Python, Scala.
• Experience with AI implementations and Power BI.
• Experience with Microsoft Azure's data services: You should have hands-on experience with various data services offered by Microsoft Azure, such as Azure Machine Learning, Azure Databricks, and Azure Data Factory. Familiarity with Azure Cognitive Services for integrating AI capabilities (like vision, language, and decision functionalities) directly into applications.
• Ability to work with large datasets and experience with data processing tools like SQL, Pandas, and big data technologies such as Apache Spark.
• Proficiency in Python, .NET C#, API development
• Experience with SQL, NoSQL databases and Data Fabric architectures.
• Strong communication skills to effectively collaborate with team members and stakeholders.
• Strong problem-solving skills and ability to think algorithmically.
• Ability to translate complex findings into a compelling narrative for non-technical stakeholders.
• Certifications like Microsoft Certified: Azure Data Engineer Associate will be preferred