Data Engineer II

Overview:

Data is at the heart of our global financial network. In fact, the ability to consume, store, analyze and gain insight from data has become a key component of our competitive advantage. Our goal is to build and maintain a leading-edge data platform that provides highly available , consistent data of the highest quality for all users of the platform, including our customers, operations teams and data scientists. We focus on evolving our platform to deliver exponential scale to NCR Atleos , powering our future growth.

Data & AI Engineers at NCR Atleos experience working at one of the largest and most recognized financial companies in the world, while being part of a software development team responsible for next generation technologies and solutions. Our engineers design and build large scale data storage, computation and distribution systems. They partner with data and AI experts to deliver high quality AI solutions and derived data to our consumers.

We are looking for Data & AI Engineers who like to innovate and seek complex problems. We recognize that strength comes from diversity and will embrace your unique skills, curiosity, drive, and passion while giving you the opportunity to grow technically and as an individual. Engineers looking to work in the areas of orchestration, data modelling , data pipelines, APIs, storage, distribution, distributed computation, consumption and infrastructure are ideal candidates.

Responsibilities

As a Data Engineer, you will be joining a Data & AI team transforming our global financial network and improving the quality of our products and services we provide to our customers.

and you will be responsible for designing, implementing, and maintaining data pipelines and systems to support the organization's data needs. Your role will involve collaborating with data scientists, analysts, and other stakeholders to ensure data accuracy, reliability, and accessibility.

Key Responsibilities:

• Data Pipeline Development: Design, build, and maintain scalable and efficient data pipelines to collect, process, and store structured and unstructured data from various sources.
• Data Integration: Integrate data from multiple sources such as databases, APIs, flat files, and streaming platforms into centralized data repositories.
• Data Modeling: Develop and optimize data models and schemas to support analytical and operational requirements. Implement data transformation and aggregation processes as needed.
• Data Quality Assurance: Implement data validation and quality assurance processes to ensure the accuracy, completeness, and consistency of data throughout its lifecycle.
• Performance Optimization: Monitor and optimize data processing and storage systems for performance, reliability, and cost-effectiveness. Identify and resolve bottlenecks and inefficiencies in data pipelines and leverage Automation and AI to improve overall Operations.
• Infrastructure Management: Manage and configure cloud-based or on-premises infrastructure components such as databases, data warehouses, compute clusters, and data processing frameworks.
• Collaboration: Collaborate with cross-functional teams including data scientists, analysts, software engineers, and business stakeholders to understand data requirements and deliver solutions that meet business objectives .
• Documentation and Best Practices: Document data pipelines, systems architecture, and best practices for data engineering. Share knowledge and provide guidance to colleagues on data engineering principles and techniques.
• Continuous Improvement: Stay updated with the latest technologies, tools, and trends in data engineering and recommend improvements to existing processes and systems.

Qualifications and Skills:

• Bachelor's degree or higher in Computer Science, Engineering, or a related field.
• Proven experience in data engineering or related roles, with a strong understanding of data processing concepts and technologies.
• Mastery of programming languages such as Python, Java, or Scala.
• Knowledge of database systems such as SQL, NoSQL, and data warehousing solutions.
• Knowledge of stream processing technologies such as Kafka or Apache Beam.
• Experience with distributed computing frameworks such as Apache Spark, Hadoop, or Apache Flink .
• Experience deploying pipelines in cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Experience in implementing enterprise systems in production setting for AI, natural language processing. Exposure to self-supervised learning, transfer learning, and reinforcement learning is a plus .
• Have full stack experience to build the best fit solutions leveraging Large Language Models (LLMs) and Generative AI solutions with focus on privacy, security, fairness.
• Have good engineering skills to design the output from the AI with nodes and nested nodes in JSON or array, HTML formats for as-is consumption and display on the dashboards/portals.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork abilities.
• Experience with containerization and orchestration tools such as Docker and Kubernetes.
• Familiarity with data visualization tools such as Tableau or Power BI.