Dynatrace Consultant

Responsibilities :

• Pipeline monitoring: Set up and maintain monitoring systems to track the health and performance of data pipelines. This may involve using custom monitoring solutions.
• Configure alerts and notifications to promptly identify and respond to issues or anomalies in data pipelines. Develop procedures and playbooks for incident response and resolution.
• Experience in working data, ETL, Data warehouse & BI Projects.
• Continuously monitor and analyze pipeline performance to identify bottlenecks and areas for improvement. Collaborate with data engineers to optimize data flows and processing.
• Implement logging mechanisms and error handling strategies to capture and analyze pipeline features, ensuring quick detection and troubleshooting.
• Work with data engineers and data analysts to monitor data quality metrics, delete data anomalies and develop processes to address data quality issues.
• Forecast resource requirements based on data growth and usage patterns, ensuring pipelines can handle increasing data volumes without performance degradation.
• Develop and maintain dashboards and reports to visualize key pipeline performance metrics, providing stakeholders with insights into systems health and data flow.
• Automate monitoring tasks and develop tools for streamlined management and observability of data pipelines.
• Ensure data pipeline observability aligns with security and compliance standards, such as data privacy regulations and industry best practices.
• Document monitoring processes, best practices, and system configurations. Share knowledge with team members to improve overall data pipeline reliability and efficiency.
• Collaborate with cross-functional teams, including data engineers, data scientists, and IT operations, to troubleshoot issues and implement improvements.
• Keep abreast of the latest developments in data pipeline monitoring and observability technologies and practices to recommend and implement advancements.
• Knowledge in AWS Glue, S3, Athena – Nice to have.
• Experience in JIRA – Nice to have
• Knowledge in anyone programming language Python, Java, Scala – Nice to have.

Requirements
Responsibilities :

• Pipeline monitoring: Set up and maintain monitoring systems to track the health and performance of data pipelines. This may involve using custom monitoring solutions.
• Configure alerts and notifications to promptly identify and respond to issues or anomalies in data pipelines. Develop procedures and playbooks for incident response and resolution.
• Experience in working data, ETL, Data warehouse & BI Projects.
• Continuously monitor and analyze pipeline performance to identify bottlenecks and areas for improvement. Collaborate with data engineers to optimize data flows and processing.
• Implement logging mechanisms and error handling strategies to capture and analyze pipeline features, ensuring quick detection and troubleshooting.
• Work with data engineers and data analysts to monitor data quality metrics, delete data anomalies and develop processes to address data quality issues.
• Forecast resource requirements based on data growth and usage patterns, ensuring pipelines can handle increasing data volumes without performance degradation.
• Develop and maintain dashboards and reports to visualize key pipeline performance metrics, providing stakeholders with insights into systems health and data flow.
• Automate monitoring tasks and develop tools for streamlined management and observability of data pipelines.
• Ensure data pipeline observability aligns with security and compliance standards, such as data privacy regulations and industry best practices.
• Document monitoring processes, best practices, and system configurations. Share knowledge with team members to improve overall data pipeline reliability and efficiency.
• Collaborate with cross-functional teams, including data engineers, data scientists, and IT operations, to troubleshoot issues and implement improvements.
• Keep abreast of the latest developments in data pipeline monitoring and observability technologies and practices to recommend and implement advancements.
• Knowledge in AWS Glue, S3, Athena – Nice to have.
• Experience in JIRA – Nice to have
• Knowledge in anyone programming language Python, Java, Scala – Nice to have.