Software Tester

Job Summary

As an ETL Test Engineer, you will be responsible for ensuring the quality and accuracy of data integration and transformation processes. You will work closely with developers, data engineers, and business analysts to validate that the ETL pipelines meet business requirements and technical specifications. Your expertise in Kafka, MongoDB (or SQL), and Azure Data Lakes will play a critical role in maintaining the reliability and scalability of the organization's data infrastructure.

Key Responsibilities

1. ETL Testing

• Design, develop, and execute test cases for ETL workflows to ensure data accuracy, completeness, and integrity.
• Validate data transformations, data mappings, and ETL logic against source-to-target requirements.
• Perform data validation and reconciliation between source and destination systems.

1. Data Pipeline Validation

• Test real-time data streaming using Kafka to ensure data ingestion meets performance and reliability criteria.
• Verify the integration of MongoDB or SQL databases with ETL pipelines.
• Ensure that data stored in Azure Data Lakes adheres to compliance and quality standards.

1. Defect Management

• Identify, report, and track defects during testing phases.
• Collaborate with developers to resolve issues and re-test fixes.

1. Automation

• Develop automated testing frameworks for ETL and data validation processes.
• Use scripting languages or tools to create reusable testing scripts.

1. Performance and Scalability Testing

• Test the performance of ETL pipelines under varying loads to ensure scalability.
• Monitor resource utilization and recommend optimizations.

1. Documentation

• Maintain comprehensive testing documentation, including test plans, test cases, and test reports.
• Provide detailed defect reports and testing summaries to stakeholders.

Required Skills

1. ETL Tools & Processes

• Strong understanding of ETL tools and data transformation processes.

1. Kafka

• Hands-on experience with Apache Kafka, including testing and validating streaming data pipelines.

1. Database Technologies

• Proficiency in working with MongoDB or relational databases like SQL Server, MySQL, or PostgreSQL.
• Experience in writing complex queries for data validation.

1. Azure Data Lake

• Familiarity with Azure Data Lake architecture, storage, and data integration processes.

1. Scripting & Automation

• Knowledge of scripting languages such as Python, Shell, or JavaScript for automation.
• Familiarity with testing tools like Selenium, JMeter, or similar tools is a plus.

1. Data Analysis Tools

• Proficiency in tools like Excel, Power BI, or Tableau for data validation and reporting.

1. Soft Skills

• Strong problem-solving abilities and attention to detail.
• Effective communication and collaboration skills to work with cross-functional teams.

Preferred Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• 4+ years of experience in ETL testing, with exposure to big data technologies.
• Certification in data engineering or cloud technologies (e.g., Microsoft Azure, Apache Kafka).

This role offers a great opportunity to work with cutting-edge technologies and contribute to building robust, high-quality data solutions.