Sr Consultant Data Engineer Personalization

Data Engineer, Sr Consultant, Personalization

Bangalore, India

Full-time

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

The Global Data Office

With data being the fuel that drives our future - our strategies, policies, and business successes around data will define our future growth prospects. Unlocking the value available through the innovative use of data on behalf of consumers, businesses, and communities is key to our future. With our ongoing commitment to Visa’s Data Values and the responsible use of data, we at Visa have a bold vision to continue to grow and accelerate our data-related businesses and capabilities.

Led by Visa’s first Chief Data Officer, the Global Data Office coordinates high-impact, complex, company-wide projects related to data business strategy, critical investments, policy development, and marketplace execution. Working closely with Visa’s Chief Privacy Officer, the Technology organization, the wider data community, and the full set of Visa’s global lines of business, the Global Data Office is evolving Visa’s data-related work with momentum and enthusiasm. Driven by its commitment to trusted use of data, Visa is seeking enthusiastic leaders who are change-agents and passionate about the future of data around the world.

The Global Personalization team comes under the Global Data Office and comprises of data practitioners with diverse data skillsets ranging from data science, data engineering, ML engineering and ML operations. The team is responsible for using Visanet data in creating near real time signals which are consumed by our clients for creating hyper personalized offers and decisions on their end customers.

We are looking for an exceptional technical leader in data engineering domain, who will be part of Global Personalization team and will report into the Head of Data Engineering within Global Data Team. In this role, you will be a Senior Manager/Architect, responsible in setting and managing the Data Engineering work stream. The ideal candidate will possess robust experience in data engineering and be adept at navigating the complexities associated with building, deploying and maintaining large Metric repositories. Your role will encompass the complete lifecycle of data repository management, from their development and testing to their deployment and data processing frameworks and applications using in-house tools and technologies.

You will partner closely with Global stakeholders across consulting, data science, data engineering, technology, and other teams. You will get chance to leverage your strategic planning, business analysis and technical knowledge of data engineering architectures, tools and solutions. You will also be a hands-on expert who can effectively partner with cross-functional teams to build effective data repositories.

Primary responsibilities

• Architecture & Design : Lead the architecture and design of the Metric repository across all regional requirements, using re-usable functions and frameworks ensuring scalability, reliability, and performance.
• Data Analysis: Lead the discussions with the requestors from across the regions, around understanding the Metric requirements and defining a process to standardize the Metrics and building the Metric repository
• Data Integration: Architect and oversee the integration of various data sources, including Transactional systems, Global and Regional data assets.
• Data Pipeline Development: Setup and lead a team Data Engineers, who will be responsible to build new scripts to development the metrics or re-engineer scripts created by the Data science teams, such that they can be easily plugged into existing ML Ops frameworks.
• AI/ML: Design and develop the integration of the AI/ML models outputs into the Metric repository, build the ML Ops components to support and manage the lifecycle of AI based models built by the regional and global data science teams.
• Data Quality and Governance: Establish and enforce data quality standards and governance policies around Data, Metrics management and all client deliveries.
• Code Review and Best Practices: Conduct code reviews and ensure adherence to best practices in development and deployment.
• Documentation: Maintain comprehensive documentation for all data assets and data engineering processes.
• Team Management & Collaboration: Lead the Personalization Data Engineering team, with end to end ownership of building the Metric repository. Collaborate with Data engineers, Data scientists and various groups within the organization to identify areas of improvement, bottlenecks and re-use existing frameworks/processes.

Technical skills

• Experience in building large globally applicable data platforms using different data modelling, data storage and data flow techniques to support the technical and business use cases and a solid understanding of best practices in data engineering
• Experience with machine learning model inference, validation, deployment and management of BAU operations.
• Knowledge of a variety of machine learning techniques (clustering, decision tree, etc.) and their real-world advantages/drawbacks.
• Strong programming skills in building data pipelines using PySpark, Hive, Airflow.
• Experience working with scheduling tools (Airflow, Oozie) or building data processing orchestration workflows.
• Hands-on experience working with large scale data ingestion, processing, and storage in the Hadoop ecosystem
• Experience in writing and optimizing SQL queries in Big data environment.
• Experience in creating data dictionaries, setup and monitor data validation alerts, and execute periodic jobs to maintain data pipelines for completed projects
• Experience working in Linux/Unix environment and exposure to command line utilities.
• Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.
• Experience working in building and integrating the code in the defined CI/CD framework using git.
• Experience in drafting solution architecture frameworks that rely on API’s and micro-services

Strategic and Functional Excellence

• Good business acumen to orient data analysis to business needs of clients, including experience in the payments space.
• Ability to translate data and technical concepts into requirements documents, business cases and user stories.
• Good understanding of agile working practices and related program management skills.
• Should have strong problem-solving capabilities and ability to quickly propose feasible solutions and effectively communicate strategy and risk mitigation approaches to leadership.
• Good communication and presentation skills with ability to interact with different cross-functional team members at varying levels
• Ability to learn new tools and paradigms as data science continues to evolve at Visa and elsewhere.

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

Qualifications

• 12+ years of relevant work experience with a Bachelor’s Degree or 10+ years of relevant work experience with a Master's or Advanced Degree with specialization in Computer science, Information science, Data Engineering and Analytics or relevant area.

• 5+ years of experience managing technical architecture, tooling and design systems for Data/ML products/solutions preferably in a financial services or consulting setting

• 5+ years of experience leading BI/ data visualization projects and related skills

• Familiarity with shared services, consulting, financial services is a strong plus

• Fundamental and broad knowledge of common data tools and methodologies

Additional information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.