Databricks QA Engineer

Job Title: Data QA Engineer – Databricks & Python Experience: 3 – 8 Years Immediate to 30 Days We are seeking a Data QA Engineer with strong experience in Databricks and Python to ensure the quality, reliability, and accuracy of data pipelines and analytical solutions. The ideal candidate will collaborate closely with Data Engineers, Analysts, and Business Teams to validate data integrity, perform automated testing, and drive continuous improvement in data quality processes. At ValueMomentum’s Technology Solution Centers, we are a team of passionate engineers who thrive on tackling complex business challenges with innovative solutions while transforming the P&C insurance value chain. We achieve this through a strong engineering foundation and by continuously refining our processes, methodologies, tools, agile delivery teams, and core engineering archetypes. Platforms, Infra/Cloud, Application, Data, Core, and Quality Assurance. You'll have the opportunity to showcase your talents by contributing to impactful projects. We are seeking a skilled Data QA Engineer with 3–8 years of experience in data testing and quality assurance, specializing in Databricks, Python, and SQL. The role involves designing and executing data validation frameworks, automating data quality checks, and ensuring the accuracy and integrity of large-scale data pipelines. Experience with PySpark, ETL/ELT testing, and cloud data platforms (AWS, Azure, or GCP) is essential. Design and execute test strategies for data pipelines and ETL processes built on Databricks. Develop and maintain automated data validation scripts using Python, PySpark, and SQL. Conduct data reconciliation, completeness, and consistency checks across source and target systems. Collaborate with data engineering teams to identify and resolve data quality issues. Develop reusable QA frameworks for data testing within Databricks and cloud data ecosystems. Integrate automated tests into CI/CD pipelines for continuous validation. Perform root cause analysis and provide recommendations for data issue remediation. 3–8 years of experience in Data QA / Data Testing roles. ~ Strong hands-on experience with Databricks and PySpark for data testing and validation. ~ Proficiency in Python for developing automated test frameworks and scripts. ~ Strong SQL skills for data profiling, validation, and transformation testing. ~ Experience in ETL/ELT testing and working with data warehouses or data lakes. ~ Exposure to cloud data platforms (AWS, Azure, or GCP) is preferred. ~ Familiarity with Agile methodologies and tools like JIRA, Git, and CI/CD pipelines. ~ Experience in the Property & Casualty (P&C) Insurance domain. Knowledge of Delta Lake, Apache Spark, and data governance frameworks. Exposure to DataOps practices or DevOps for data pipelines. Experience testing machine learning data pipelines or data APIs. ValueMomentum is a leading solutions provider for the global property and casualty insurance industry. The company helps insurers stay ahead with sustained growth and high performance, enhancing stakeholder value and fostering resilient societies. Having served over 100 insurers, ValueMomentum is one of the largest services providers exclusively focused on the insurance industry. We at ValueMomentum offer you a congenial environment to work and grow in the company of experienced professionals. Competitive compensation package. Career Advancement: Individual Career Development, coaching and mentoring programs for professional and leadership skill development. Comprehensive training and certification programs. Performance Management: Goal Setting, continuous feedback and year-end appraisal.