Data Engineer

2 months ago


Kanpur, India Visa Full time

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.


Job Description

The Senior Data Engineer role will coordinate with other teams to manage and prioritize projects, and driving the strategy, roadmap, and execution of key data engineering initiatives. The role requires understanding and translating business needs into data models, creating robust data pipelines, and developing and maintaining data lakes. It also involves implementing and maintaining CI/CD pipelines for data solutions. The candidate should be able to define and manage data load procedures, implement data strategies, and ensure robust operational data management systems. Collaborating with stakeholders across the organization to understand their data needs and deliver solutions is also a key part of this role. The ideal candidate will be proficient in big data tools like Hadoop, Hive, and Spark, programming languages such as Scala, Python, SQL, and have strong analytical skills related to working with structured and unstructured datasets.

This position is based in Visa's offices in Bangalore, India, and presents an excellent opportunity for those looking to make a significant impact in the field of Data Engineering.


Essential Functions:

  • Requirement Analysis: Understand and translate business needs into data models supporting long-term solutions.
  • Data Modeling: Work with the Business team to implement data strategies, build data flows and develop conceptual data models.
  • Data Pipeline Design: Create robust and scalable data pipelines and data products in a variety of domains.
  • Data Integration: Develop and maintain data lakes by acquiring data from primary and secondary sources and build scripts that will make our data evaluation process more flexible or scalable across data sets.
  • Testing: Define and manage the data load procedures to reject or discard datasets or data points that do not meet the defined business rules and quality thresholds.
  • Deployment: Implement data strategies and develop physical data models, along with the development teams, data analyst teams and information system team to ensure robust operational data management systems.
  • Understanding of and ability to Implement Data Engineering principles and best practices.
  • Team Leadership: Coordinate with other teams to manage and prioritize projects.
  • Stakeholder Management: Collaborate with stakeholders across the organization to understand their data needs and deliver solutions.
  • Continuous Integration and Continuous Deployment (CI/CD): Implement and maintain CI/CD pipelines for data solutions, ensuring rapid, reliable, and streamlined updates to the data environment.


Technical Skills:

  • Extensive experience in big data tools: Hadoop, Hive, and Spark.
  • Proficiency in Scala, Python, SQL, and PySpark.
  • Experience with Unix/Linux systems with scripting experience in Bash.
  • Experience with data pipeline and workflow management tools like Airflow, etc.
  • Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
  • Experience with cloud services AWS or Azure.
  • Experience with stream-processing systems: Kafka, Spark-Streaming, etc.
  • Strong analytic skills related to working with structured & unstructured datasets.
  • Understanding of processes supporting data transformation, data structures, metadata, dependency, and workload management.
  • A successful history of manipulating, processing, and extracting value from large, disconnected datasets.


Preferred Qualifications:

• 5+ years of work experience with a bachelor's degree or at least 3+ years of work experience with an Advanced degree (e.g. Master’s, MBA, JD, MD) or 1+ years of work experience with a PhD degree

• Exposure to Financial Services/ Payments Industry

• Proven leadership skills with experience leading large data engineering initiatives.

Leadership Competencies

• Exhibits intellectual curiosity and a desire for continuous learning.

• Demonstrates integrity, maturity, and a constructive approach to business challenges.

• Respect for the Individuals at all levels in the workplace

• Strive for Excellence and extraordinary results.

• Use sound insights and judgments to make informed decisions in line with business strategy and

needs.

• Ability to successfully persuade/influence internal stakeholders for building best-in-class solutions.



  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    Job DescriptionMake an impact with NTT DATAWe are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Senior Associate Critical Facilities Operations and Maintenance...

  • Senior Engineer

    4 weeks ago


    Kanpur, Uttar Pradesh, India NTT DATA Full time

    Job DescriptionMake an impact with NTT DATANTT DATA is a $30+ billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success.Your day at NTT DATAThe Senior Associate Critical Facilities Operations and Maintenance Engineer...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    About the RoleThe Senior Associate Critical Facilities Operations and Maintenance Engineer is a developing subject matter expert, responsible for working with all major systems within a facility or building such as electrical and HVAC systems and performs regular maintenance on all systems.This role helps to design, plan, install, and/or maintain mechanical,...

  • Data Engineer

    3 weeks ago


    Kanpur, India MPACT TRAINING AND DEVELOPMENT Full time

    Data Engineer BangaloreExperience - 5 to 8 yrsJOB DESCRIPTION We are seeking experienced, motivated and enthusiastic Data Engineering Specialists to join our team. As part of the team, you will be involved in the implementation of the ongoing and new initiatives for our company and contribute to our mission of delivering high-quality data-driven solutions...

  • AWS - Data Engineer

    2 weeks ago


    Kanpur, India EliteRecruitments Full time

    Job opportunity for data engineers for Gurgaon locationExperience: 5+ yearsShould have good experience in AWS cloud platform.Role & Responsibilities Overview:A seasoned data team member with a good grasp of both business and technical aspects. Familiarity with data stackInteract with the other members of the data governance team,business teams to understand...


  • Kanpur, Uttar Pradesh, India DNA Technology Services Full time

    Job Title: Data/Platform Engineer (SNOWFLAKE)Location: RemoteExperience Required: 7-12 YearsResponsibilities:Design and develop robust data ingestion methods and patterns with tools like Fivetran, Snowpipe, and Secure Data Share.Implement best practices for full SDLC and CI/CD, using Python, Git, GitHub, and GitHub Actions.Automate infrastructure and...


  • kanpur, India Dhira Full time

    Role: Data Engineer for AI/MLExperience Required: 6+ yearsWe’re looking for a true data craftsman who can dexterously navigate the worlds of raw data engineering and refined machine learning pipelines.You’re not the average “ETL engineer” who relies on pre-packaged tools like Informatica or Talend. You are a solid programmer, fluent in Python, and an...


  • Kanpur, India Lear Corporation Full time

    We’re Lear For YouLear, a global automotive technology leader in Seating and E-Systems, is Making every drive better by delivering intelligent in-vehicle experiences for customers around the world. With over 100 years of experience, Lear has earned a legacy of operational excellence while building its future on innovation. Our talented team is committed to...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    About the Role:We are seeking an experienced Senior Network Engineer to join our team at NTT DATA. This is a senior-level technical professional responsible for leveraging their networking skills and experience to support, design, and optimize NTT's network infrastructure.This role works closely with network engineers, architects, and cross-functional teams...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    About the RoleJob DescriptionMake an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day...


  • kanpur, India AXA Full time

    Experience Required: 13 + years of experience in Cloud Data Engineering (Azure). Competencies Required: • Proficient in handling run and build projects. • Experience in leading a team of 15+ members. • Proficient in Cloud Services – Azure. • Proficient in building and supporting API-based data ingestion and distribution pipelines. •...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Senior Associate Specialist to join our team at NTT DATA. As a key member of our construction team, you will be responsible for overseeing the delivery of major capital construction projects of significant value.Key ResponsibilitiesLead a team of professionals to safely deliver major capital construction projects...

  • Data analyst

    5 days ago


    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training) Experience: 0-1 Years Location: Kanpur (New Kanpur) Job Type: Full Time Job Description: We are seeking a highly motivated Data Analyst with strong Python programming skills and a passion for working with data. The ideal candidate should have basic knowledge of machine learning and...

  • Data analyst

    7 days ago


    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience: 0-1 YearsLocation: Kanpur (New Kanpur)Job Type: Full TimeJob Description:We are seeking a highly motivated Data Analyst with strong Python programming skills and a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be familiar with...

  • Data Analyst

    1 week ago


    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience:0-1 YearsLocation:Kanpur (New Technopark @ IIT Kanpur)Job Type:Full TimeJob Description:We are seeking a highly motivatedData Analystwithstrong Python programming skillsand a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be familiar with...

  • Data Analyst

    1 week ago


    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience: 0-1 YearsLocation: Kanpur (New Technopark @ IIT Kanpur)Job Type: Full TimeJob Description:We are seeking a highly motivated Data Analyst with strong Python programming skills and a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be...


  • kanpur, India Greenway Health Full time

    Minimum Qualifications• 5-7 years’ experience in Data Engineering – Data Processing using various tools• Excellent understanding and hands on experience with T-SQL programming• Excellent understanding and hands on experience with Microsoft SQL Server, other RDBMS, and No-SQL systems• Excellent understanding and hands on experience with ETL, Data...

  • Big Data Developer

    1 month ago


    Kanpur, India Amossys Consulting Services Full time

    Position Title: Data EngineerExperience Required: 5+ years Employment Type: Full-time Employment Mode: HybridLocation: Bangalore, Pune, Trivandrum.Industry: Banking & Insurance sectorJob Summary:We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate will have strong...

  • Data Analyst

    1 week ago


    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience: 0-1 YearsLocation: Kanpur (New Technopark @ IIT Kanpur)Job Type: Full TimeJob Description:We are seeking a highly motivated Data Analyst with strong Python programming skills and a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be...

  • Data Analyst

    1 week ago


    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience: 0-1 YearsLocation: Kanpur (New Technopark @ IIT Kanpur)Job Type: Full TimeJob Description:We are seeking a highly motivated Data Analyst with strong Python programming skills and a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be...