MDM Data Architect

Job Description:

Master Data Management (MDM) Expert

We are seeking an experienced MDM expert to develop and implement distributed data pipelines using PySpark on Databricks. The successful candidate will design and deploy complex SQL for large-scale data processing, including match/merge algorithms and survivorship rules using Informatica MDM or Reltio APIs.

The ideal candidate will have hands-on experience with AWS services like S3, Glue, Lambda, and Step Functions for orchestrating MDM workflows. They will also be responsible for building and maintaining Delta Lake tables with schema evolution and versioning for master data domains.

In addition, the candidate will need to automate data quality checks using IDQ or custom PySpark validators with rule-based profiling. They will also integrate external enrichment sources via REST APIs and batch pipelines.

Key Responsibilities:

• Design and develop distributed data pipelines using PySpark on Databricks
• Implement complex SQL for large-scale data processing
• Develop match/merge algorithms and survivorship rules using Informatica MDM or Reltio APIs
• Orchestrate MDM workflows using AWS services
• Build and maintain Delta Lake tables with schema evolution and versioning
• Automate data quality checks using IDQ or custom PySpark validators
• Integrate external enrichment sources via REST APIs and batch pipelines

Must-Have Skills:

• Advanced proficiency in PySpark for distributed data processing and transformation
• Strong SQL skills for complex data modeling, cleansing, and aggregation logic
• Hands-on experience with Databricks including Delta Lake, notebooks, and job orchestration
• Deep understanding of MDM concepts including match/merge, survivorship, and golden record creation
• Experience with MDM platforms like Informatica MDM or Reltio, including REST API integration
• Proficiency in AWS services such as S3, Glue, Lambda, Step Functions, and IAM
• Familiarity with data quality frameworks and tools like Informatica IDQ or custom rule engines
• Experience building CI/CD pipelines for data workflows using GitHub Actions, Jenkins, or similar
• Knowledge of schema evolution, versioning, and metadata management in data lakes
• Ability to implement lineage and observability using Unity Catalog or third-party tools
• Comfort with Unix shell scripting or Python for orchestration and automation
• Hands on experience on RESTful APIs for ingesting external data sources and enrichment feeds

Good-to-Have Skills:

• Experience with Tableau or PowerBI for reporting MDM insights.
• Exposure to Agile practices and tools (JIRA, Confluence).
• Prior experience in Pharma/Life Sciences.
• Understanding of compliance and regulatory considerations in master data.

This is a challenging and rewarding role that offers opportunities for growth and development. If you are passionate about working with data and want to make a real impact, please apply.

What We Offer:

• A competitive salary package
• Ongoing training and professional development opportunities
• A collaborative and supportive team environment
• The opportunity to work with cutting-edge technologies and innovative solutions

Professional certifications such as MDM certification, Data Analysis certification, and cloud certification are highly valued but not essential.

As an MDM expert, you will be responsible for designing and implementing complex data pipelines using PySpark on Databricks. You will work closely with cross-functional teams to ensure seamless execution of MDM workflows.