Senior Data Engineer

At Backbase, we're reimagining the future of banking through data and intelligence. Our Engagement Banking Platform enables financial institutions to deliver seamless, personalized customer experiences—and data is at the core of that transformation.
As a Senior Data Engineer – Consultant, you'll play a key role in building scalable, secure, and high-performance data systems that fuel AI and machine learning across our clients' ecosystems. You will work alongside AI/ML Engineers, solution architects, and client teams to unlock the full potential of data pipelines, analytics, and model-ready infrastructure.
If you thrive at the intersection of data engineering and intelligent system delivery, this role gives you the platform to drive measurable impact in financial services.

You will be part of our Consulting AI team, contributing to the design, development, and deployment of end-to-end data pipelines that support machine learning and generative AI use cases in production environments.
From real-time ingestion and feature stores to secure data access for LLM applications, you'll deliver foundational systems that enable scalable AI outcomes for banks across the globe.

Responsibilities:
● Proven experience building robust, high-throughput data pipelines in cloud and on-prem hybrid environments
● Hands-on with Databricks (Delta Lake, Spark, Workflows, Unity Catalog); able to optimize and debug pipelines at scale
● Strong Python and SQL skills; familiarity with APIs, data schemas, and version-controlled pipeline code
● Experience with stream processing platforms (Kafka, Kinesis, etc.) and working with semi-structured formats (Parquet, JSON, Avro)
● Familiarity with MLOps and DataOps practices: CI/CD, reproducibility, monitoring, and change tracking
● Exposure to GenAI-specific data needs: embedding generation, RAG orchestration, document ingestion
● Comfortable navigating privacy and compliance constraints in regulated domains like banking or insurance
● Consulting or client-facing project experience with agile methodologies
Strong communication and collaboration skills across distributed teams

● Solid foundation in building reliable, maintainable, and high-throughput data pipelines
● Hands-on experience with tools like Spark, Kafka, dbt, Airflow, Snowflake, BigQuery, or Delta Lake
● Experience with Databricks for data pipeline development, Delta Lake integration, and Spark-based processing.
● Hands-on experience with tools like Databricks (incl. Delta Lake and Spark), Kafka, dbt, Airflow, Snowflake, or BigQuery
● Strong programming skills in Python and SQL; familiarity with data APIs, connectors, and versioning systems
● Experience working in cloud-native environments (AWS, GCP, or Azure)
Comfortable working with unstructured and semi-structured data formats (JSON, XML, Parquet, etc.)
● Experience supporting data workflows for LLM and GenAI applications (vector stores, embedding pipelines, document loaders)
● Familiarity with MLOps and DataOps practices: pipeline CI/CD, monitoring, lineage, and governance
● Background working on consulting engagements, agile teams, or client-facing data delivery
● Strong documentation habits and communication skills to align with distributed teams

Requirements
● 5+ years in data engineering, backend systems, or data infrastructure
Strong hands-on experience with Databricks and Spark (Delta Lake, Workflows, MLflow optional)
● Familiarity with vector databases and embedding pipelines for AI/LLM systems
● Real-time data engineering experience with Kafka, Kinesis, or similar streaming platforms
● Experience building ML-ready datasets, feature stores, or analytical layers supporting intelligent applications
● CI/CD pipeline knowledge and working with DevOps teams to operationalize data workflows
● Understanding of secure data architecture: encryption, RBAC, anonymization, and auditability
● Prior work on consulting, system integration, or cross-functional delivery projects in complex enterprise settings
● Desired: experience integrating Databricks into secure enterprise cloud environments (e.g., Unity Catalog, private endpoints, secured clusters)