Enterprise Data Platform Developer

The Universal Data Catalog is a comprehensive metadata management platform designed to provide a centralized repository for all data-related information across the organization. This platform enables efficient data discovery, governance, and utilization by offering detailed metadata definitions for tables, columns, and other data assets.

Key Responsibilities:

• Develop and maintain the Enterprise Data Catalog platform, ensuring it meets the organization's evolving needs.
• Implement and manage metadata ingestion processes to ensure the catalog is up-to-date with the latest data definitions and business context.
• Collaborate with data stewards, data owners, and other stakeholders to enrich metadata with business definitions, data lineage, and usage context.
• Enhance the catalog's search and discovery capabilities to provide users with intuitive and efficient access to data assets.
• Integrate the data catalog with other enterprise systems and tools to support data governance, data quality, and analytics initiatives.
• Monitor and optimize the performance of the data catalog to ensure scalability and reliability.
• Provide training and support to users on how to effectively leverage the data catalog for their data-related needs.
• Actively collaborate with the open-source community to contribute to and leverage the latest advancements in the Datahub project.
• Analyze industry best practices and keep the catalog functionality up-to-date with feature sets provided by the market, while focusing on Visa's scalability requirements.
• Champion the adoption of open infrastructure solutions that are fit for purpose while keeping technology relevant.
• Work collaboratively with cross-functional teams to identify and prioritize business needs, and develop effective solutions to meet those needs.

Basic Qualifications:

• 8+ years of relevant work experience with a Bachelor's Degree or at least 5 years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 2 years of work experience with a PhD, OR 11+ years of relevant work experience.

Preferred Qualifications:

• 9 or more years of relevant work experience with a Bachelor Degree or 7 or more relevant years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 3 or more years of experience with a PhD.
• 10 yrs+ of experience in architecture design and development of large-scale data management platforms and data application with simple solutions.
• Must have extensive hands-on coding and designing skills on Java/Python or MVC (model-view-controller) for end-to-end development, SQL/NoSQL technology, Web Services (REST/ SOAP/gRPC), React/Angular for front-end (UI front-end nice to have).
• Expertise in design and management of complex data structures and data processes.
• Deep knowledge and hands on experience on big data and cloud computing technologies.
• Strong service architecture and development experience with high performance and scalability.
• Technical background in data with deep understanding of issues in multiple areas such as data acquisition, ingestion and processing, data management, distributed processing, and high availability required.
• Strong on driving for results and self-motivated, strong learning mindset, with good understanding of related advanced/new technology.
• Bachelor's degree in computer science or related technical discipline required. Advanced degree is a plus.
• Strong leadership and team player.
• Strong skills on mentoring/growing junior people.
• Payment industry experience is a plus.

• Experience with ETL/ELT tools / applications.
• Experience with Apache NiFi and Apache Spark for processing large data sets.
• Experience on Elastic Search.
• Knowledge on Data Catalog tools.
• Experience in building Data Pipeline development tools.
• Experience with Data Governance and Data Quality tools.