PySpark Hive Data Engineer

The RoleThe Data Engineer is accountable for developing high quality data products to support the Bank's regulatory requirements and data driven decision making. A Data Engineer will serve as an example to other team members, work closely with customers, and remove or escalate roadblocks. By applying their knowledge of data architecture standards, data warehousing, data structures, and business intelligence they will contribute to business outcomes on an agile team.ResponsibilitiesDeveloping and supporting scalable, extensible, and highly available data solutionsDeliver on critical business priorities while ensuring alignment with the wider architectural visionIdentify and help address potential risks in the data supply chainFollow and contribute to technical standardsDesign and develop analytical data modelsRequired Qualifications & Work ExperienceFirst Class Degree in Engineering/Technology (4-year graduate course)8 to 12 years' experience implementing data-intensive solutions using agile methodologies, should be hands on on PySpark, Hive, HDFS, HadoopShould have strong understanding of AWS Glue Serverless Data Integration, Terraform, deploying Apache Spark on AWS, using Elastic Kubernetes Service (EKS), use of deployment tools LightSpeed TetkonExperience of relational databases and using SQL for data querying, transformation and manipulationExperience of modelling data for analytical consumersAbility to automate and streamline the build, test and deployment of data pipelinesExperience in cloud native technologies and patternsA passion for learning new technologies, and a desire for personal growth, through self-study, formal classes, or on-the-job trainingExcellent communication and problem-solving skillsAn inclination to mentor; an ability to lead and deliver medium sized components independentlyTechnical Skills (Must Have)ETL: Hands on experience of building data pipelines. Proficiency in two or more data integration platforms such as Ab Initio, Apache Spark, Talend and InformaticaBig Data: Experience of 'big data' platforms such as Hadoop, Hive or Snowflake for data storage and processingData Warehousing & Database Management: Expertise around Data Warehousing concepts, Relational (Oracle, MSSQL, MySQL) and NoSQL (MongoDB, DynamoDB) database designData Modeling & Design: Good exposure to data modeling techniques; design, optimization and maintenance of data models and data structuresLanguages: Proficient in one or more programming languages commonly used in data engineering such as Python, Java or ScalaDevOps: Exposure to concepts and enablers - CI/CD platforms, version control, automated quality control managementData Governance: A strong grasp of principles and practice including data quality, security, privacy and complianceTechnical Skills (Valuable)Ab Initio: Experience developing Co>Op graphs; ability to tune for performance. Demonstrable knowledge across full suite of Ab Initio toolsets e.g., GDE, Express>IT, Data Profiler and Conduct>IT, Control>Center, Continuous>FlowsCloud: Good exposure to public cloud data platforms such as S3, Snowflake, Redshift, Databricks, BigQuery, etc. Demonstratable understanding of underlying architectures and trade-offsData Quality & Controls: Exposure to data validation, cleansing, enrichment and data controlsContainerization: Fair understanding of containerization platforms like Docker, KubernetesFile Formats: Exposure in working on Event/File/Table Formats such as Avro, Parquet, Protobuf, Iceberg, DeltaOthers: Experience of using a Job scheduler e.g., Autosys. Exposure to Business Intelligence tools e.g., Tableau, Power BICertification on any one or more of the above topics would be an advantage.-Job Family Group:Technology-Job Family:Digital Software Engineering-Time Type:Full time-Most Relevant SkillsPlease see the requirements listed above.-Other Relevant SkillsFor complementary skills, please see above and/or contact the recruiter.-Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi. View Citi's EEO Policy Statement and the Know Your Rights poster.