Data Engineer
2 months ago
Company Description
Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.
Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.
Job Description
The Senior Data Engineer role will coordinate with other teams to manage and prioritize projects, and driving the strategy, roadmap, and execution of key data engineering initiatives. The role requires understanding and translating business needs into data models, creating robust data pipelines, and developing and maintaining data lakes. It also involves implementing and maintaining CI/CD pipelines for data solutions. The candidate should be able to define and manage data load procedures, implement data strategies, and ensure robust operational data management systems. Collaborating with stakeholders across the organization to understand their data needs and deliver solutions is also a key part of this role. The ideal candidate will be proficient in big data tools like Hadoop, Hive, and Spark, programming languages such as Scala, Python, SQL, and have strong analytical skills related to working with structured and unstructured datasets.
This position is based in Visa's offices in Bangalore, India, and presents an excellent opportunity for those looking to make a significant impact in the field of Data Engineering.
Essential Functions:
- Requirement Analysis: Understand and translate business needs into data models supporting long-term solutions.
- Data Modeling: Work with the Business team to implement data strategies, build data flows and develop conceptual data models.
- Data Pipeline Design: Create robust and scalable data pipelines and data products in a variety of domains.
- Data Integration: Develop and maintain data lakes by acquiring data from primary and secondary sources and build scripts that will make our data evaluation process more flexible or scalable across data sets.
- Testing: Define and manage the data load procedures to reject or discard datasets or data points that do not meet the defined business rules and quality thresholds.
- Deployment: Implement data strategies and develop physical data models, along with the development teams, data analyst teams and information system team to ensure robust operational data management systems.
- Understanding of and ability to Implement Data Engineering principles and best practices.
- Team Leadership: Coordinate with other teams to manage and prioritize projects.
- Stakeholder Management: Collaborate with stakeholders across the organization to understand their data needs and deliver solutions.
- Continuous Integration and Continuous Deployment (CI/CD): Implement and maintain CI/CD pipelines for data solutions, ensuring rapid, reliable, and streamlined updates to the data environment.
Technical Skills:
- Extensive experience in big data tools: Hadoop, Hive, and Spark.
- Proficiency in Scala, Python, SQL, and PySpark.
- Experience with Unix/Linux systems with scripting experience in Bash.
- Experience with data pipeline and workflow management tools like Airflow, etc.
- Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
- Experience with cloud services AWS or Azure.
- Experience with stream-processing systems: Kafka, Spark-Streaming, etc.
- Strong analytic skills related to working with structured & unstructured datasets.
- Understanding of processes supporting data transformation, data structures, metadata, dependency, and workload management.
- A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
Preferred Qualifications:
• 5+ years of work experience with a bachelor's degree or at least 3+ years of work experience with an Advanced degree (e.g. Master’s, MBA, JD, MD) or 1+ years of work experience with a PhD degree
• Exposure to Financial Services/ Payments Industry
• Proven leadership skills with experience leading large data engineering initiatives.
Leadership Competencies
• Exhibits intellectual curiosity and a desire for continuous learning.
• Demonstrates integrity, maturity, and a constructive approach to business challenges.
• Respect for the Individuals at all levels in the workplace
• Strive for Excellence and extraordinary results.
• Use sound insights and judgments to make informed decisions in line with business strategy and
needs.
• Ability to successfully persuade/influence internal stakeholders for building best-in-class solutions.
-
Lead data engineer
4 days ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Lead Data Engineer
1 week ago
India Wavicle Data Solutions Full timeJob Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...
-
Lead Data Engineer
1 week ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Aws data engineer
4 hours ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on Py Spark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 weeks ago
india ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...
-
AWS Data Engineer
2 weeks ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
Lead Data Engineer: Strategic Data Architect
1 week ago
India Wavicle Data Solutions Full timeWavicle Data Solutions is seeking an experienced Lead Data Engineer to play a pivotal role in architecting and implementing data solutions.About the RoleThis position offers a competitive salary of $180,000 - $220,000 per annum, commensurate with experience. Key ResponsibilitiesDesign and develop scalable and efficient data pipelines using Python and...
-
india Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...
-
india Wavicle Data Solutions Full timeJob Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...
-
india Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...
-
Technical Lead
2 months ago
india Data Axle India Full timeData Axle Inc. has been an industry leader in data, marketing solutions, sales, and research for over 50 years in the USA. Data Axle now has an established strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and by leveraging...
-
Lead Data Scientist
2 months ago
india Data Axle India Full timeAbout Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales, and research for over 50 years in the USA. Data Axle now has an established strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and...
-
Lead Data Scientist
2 months ago
india Data Axle India Full timeAbout Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales, and research for over 50 years in the USA. Data Axle now has an established strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and by...
-
Data engineer
1 month ago
India Ryden Global Full timeWe're #hiring #Data Engineers for our USA based client at offshore in India (must support afternoon shift in IST). Role Description This is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics...
-
Data Engineer
1 month ago
India Implere Technolgies Pvt Ltd Full timeRole Description This is a full-time remote role for a Data Engineer at Implere Technologies Pvt Ltd. The Data Engineer will be responsible for tasks such as data engineering, data modeling, ETL processes, data warehousing, and data analytics on a day-to-day basis. Qualifications Data Engineering and Data Modeling skills Experience in ETL (Extract...
-
Data Engineer
1 month ago
india Ryden Global Full timeWe're #hiring #DataEngineers for our USA based client at offshore in India (must support afternoon shift in IST). Role DescriptionThis is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics on a...
-
Data Engineer
1 month ago
India Ryden Global Full timeWe're #hiring #DataEngineers for our USA based client at offshore in India (must support afternoon shift in IST). Role Description This is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics on a...
