Data Engineer II

Job Description About Zeta Zeta is a Next-Gen Banking Tech company that empowers banks and fintechs to launch banking products for the future. It was founded by and Ramki Gaddipati in 2015. Our flagship processing platform - Zeta Tachyon - is the industry's first modern, cloud-native, and fully API-enabled stack that brings together issuance, processing, lending, core banking, fraud & risk, and many more capabilities as a single-vendor stack. 20M+ cards have been issued on our platform globally. Zeta is actively working with the largest Banks and Fintechs in multiple global markets transforming customer experience for multi-million card portfolios. Zeta has over 1700+ employees - with over 70% roles in R&D - across locations in the US, EMEA, and Asia. We raised $280 million at a $1.5 billion valuation from Softbank, Mastercard, and other investors in 2021. About the Role As a Data Engineer II, you will play a crucial role in developing, optimizing, and managing several large data lakes and data warehouses, comprising data from multiple disparate sources. Responsibilities - Data Pipeline Operations - Design, build, and maintain robust and scalable data pipelines to ingest, transform, and deliver structured and unstructured data from multiple sources. - Ensure high-quality data by implementing monitoring, validation, and error-handling processes. - Platform Engineering & Optimization - Create and update data models to represent the structure of the data. - Design, implement, and maintain database systems. Optimize database performance and ensure data integrity. Troubleshoot and resolve database issues. - Build and manage data warehouses for storage and analysis of large datasets. - Collaborate on data modeling, schema design, and performance optimization for large-scale datasets. - Data Quality and Governance: Implement and enforce data quality standards. Contribute to data governance processes and policies. - Scripting and Programming: Develop and automate data processes through programming languages (e.g., Python, Java, SQL). Implement data validation scripts and error handling mechanisms. - Version Control: Use version control systems (e.g., Git) to manage codebase changes for data pipelines. - Monitoring and Optimization: Implement monitoring solutions to track the performance and health of data systems. Optimize data processes for efficiency and scalability. - Cloud Platforms: Work with cloud platforms (e.g., AWS, Azure, GCP) to deploy and manage data infrastructure. Utilize cloud-based services for data storage, processing, and analytics. - Security: Implement and adhere to data security best practices. Ensure compliance with data protection regulations. - Troubleshooting and Support: Provide support for data-related issues and participate in root cause analysis. Skills - Expertise in data modeling, database design, and data warehousing. Proficient in SQL and programming languages such as Python, Java, or Scala. - Experience with big data technologies such as Hadoop, Spark, and Kafka. - Cloud-native architecture expertise (AWS, GCP, or Azure), including containerization (Docker, Kubernetes) and infrastructure-as-code (Terraform, CloudFormation). Experience and Qualification - Bachelor's/master's degree in engineering (computer science, information systems) with 3-5 years of experience in data engineering, BI engineering, and data warehouse development. - Excellent command on SQL and one or more programming languages, preferably Python or Java. - Knowledge of Flink, Airflow, Apache Spark, DBT, Athena / Presto - Experience working with Kubernetes.