Azure Databricks + Pyspark

**Job Summary**

**Responsibilities**
- Analyze complex data sets to identify trends and patterns that can inform business decisions.
- Develop and maintain data pipelines using Azure Data Factory to ensure seamless data integration.
- Utilize PySpark and Python to perform data transformations and analytics.
- Implement and manage workflows using BMC Control-M WA to ensure timely data processing.
- Collaborate with cross-functional teams to understand data requirements and deliver actionable insights.
- Create and maintain data models in Azure Databricks to support advanced analytics.
- Ensure data quality and integrity by performing regular audits and validations.
- Design and implement ETL processes to extract transform and load data into Azure Data Lake Store.
- Provide technical support and troubleshooting for data-related issues.
- Develop and maintain documentation for data processes and workflows.
- Monitor and optimize data pipelines for performance and efficiency.
- Stay updated with the latest industry trends and technologies to continuously improve data analytics capabilities.
- Communicate findings and insights to stakeholders through reports and presentations.

**Qualifications**
- Possess a Bachelors degree in Computer Science Information Technology or a related field.
- Demonstrate proficiency in BMC Control-M WA PySpark and Python.
- Have hands-on experience with Azure Databricks Azure Data Lake Store and Azure Data Factory ETL.
- Show strong analytical and problem-solving skills.
- Exhibit excellent communication and collaboration abilities.
- Display a proactive and self-motivated attitude.
- Have a solid understanding of data modeling and data warehousing concepts.
- Be familiar with data governance and data quality best practices.
- Show experience in developing and maintaining data pipelines.
- Demonstrate the ability to work in a hybrid work model.
- Have a minimum of 4 years and a maximum of 6 years of relevant experience.
- Be able to work effectively during the day shift.
- Show no requirement for travel.

**Certifications Required**
- Certified Data Management Professional (CDMP) Microsoft Certified: Azure Data Engineer Associate