Databricks QA Engineer – Python, PySpark

Job Title: Data QA Engineer – Databricks & Python Experience: 3 – 8 Years Location: Hyderabad / Pune / Coimbatore Notice Period: Immediate to 30 Days About The Job We are seeking a Data QA Engineer with strong experience in Databricks and Python to ensure the quality, reliability, and accuracy of data pipelines and analytical solutions. The ideal candidate will collaborate closely with Data Engineers, Analysts, and Business Teams to validate data integrity, perform automated testing, and drive continuous improvement in data quality processes. Know your team (“Legacy Rewired, Engineering the Future”): At ValueMomentum’s Technology Solution Centers, we are a team of passionate engineers who thrive on tackling complex business challenges with innovative solutions while transforming the P&C insurance value chain. We achieve this through a strong engineering foundation and by continuously refining our processes, methodologies, tools, agile delivery teams, and core engineering archetypes. Our core expertise lies in six key areas: Platforms, Infra/Cloud, Application, Data, Core, and Quality Assurance. Join a team that invests in your growth. Our Infinity Program empowers you to build your career with role-specific skill development, leveraging immersive learning platforms. You'll have the opportunity to showcase your talents by contributing to impactful projects. Job Summary We are seeking a skilled Data QA Engineer with 3–8 years of experience in data testing and quality assurance, specializing in Databricks, Python, and SQL. The role involves designing and executing data validation frameworks, automating data quality checks, and ensuring the accuracy and integrity of large-scale data pipelines. Experience with PySpark, ETL/ELT testing, and cloud data platforms (AWS, Azure, or GCP) is essential. Exposure to the Property & Casualty (P&C) Insurance domain will be an added advantage Responsibilities Design and execute test strategies for data pipelines and ETL processes built on Databricks. Develop and maintain automated data validation scripts using Python, PySpark, and SQL. Conduct data reconciliation, completeness, and consistency checks across source and target systems. Collaborate with data engineering teams to identify and resolve data quality issues. Develop reusable QA frameworks for data testing within Databricks and cloud data ecosystems. Integrate automated tests into CI/CD pipelines for continuous validation. Prepare and maintain documentation for test plans, test cases, and validation reports. Perform root cause analysis and provide recommendations for data issue remediation. Requirements 3–8 years of experience in Data QA / Data Testing roles. Strong hands-on experience with Databricks and PySpark for data testing and validation. Proficiency in Python for developing automated test frameworks and scripts. Strong SQL skills for data profiling, validation, and transformation testing. Experience in ETL/ELT testing and working with data warehouses or data lakes. Exposure to cloud data platforms (AWS, Azure, or GCP) is preferred. Familiarity with Agile methodologies and tools like JIRA, Git, and CI/CD pipelines. Excellent analytical thinking, problem-solving, and communication skills. Good to Have Experience in the Property & Casualty (P&C) Insurance domain. Knowledge of Delta Lake, Apache Spark, and data governance frameworks. Exposure to DataOps practices or DevOps for data pipelines. Experience testing machine learning data pipelines or data APIs. About The Company ValueMomentum is a leading solutions provider for the global property and casualty insurance industry. The company helps insurers stay ahead with sustained growth and high performance, enhancing stakeholder value and fostering resilient societies. Having served over 100 insurers, ValueMomentum is one of the largest services providers exclusively focused on the insurance industry. Benefits We at ValueMomentum offer you a congenial environment to work and grow in the company of experienced professionals. Some benefits that are available to you are: Competitive compensation package. Career Advancement: Individual Career Development, coaching and mentoring programs for professional and leadership skill development. Comprehensive training and certification programs. Performance Management: Goal Setting, continuous feedback and year-end appraisal. Reward & recognition for the extraordinary performers.