Data Engineer

Company Description Experian is a global data and technology company powering opportunities for people and businesses around the world We help to redefine lending practices uncover and prevent fraud simplify healthcare create marketing solutions all using our unique combination of data analytics and software We also assist millions of people to realise their financial goals and help them save time and money We operate across a range of markets from financial services to healthcare automotive agribusiness insurance and many more industry segments We invest in people and new advanced technologies to unlock the power of data As a FTSE 100 Index company listed on the London Stock Exchange EXPN we have a team of 22 500 people across 32 countries Our corporate headquarters are in Dublin Ireland Learn more at We are looking for an enthusiastic Data Engineer - Scala to work to work at the forefront of our cloud modernization within our Credit Verification Services This is a hybrid role requiring travelling to Hyderabad office 40 per month Background This is an incredibly exciting time for the Experian UKI Region as we look to build our presence in the UK and Hyderabad and work on a technology transformation to meet our aspiration to significantly scale our business over the next five years This an opportunity to join Credit Verification Services on this journey and be part of a collaborative team that uses Agile DevSecOps principles to deliver business value Credit and Verification Services currently comprises nearly 100 engineering teams who deliver over 200 products achieving significant revenue per annum for our UK Business Our unique culture and agile ways of working offer a great opportunity to those looking to join an experienced diverse set of problem solvers to design build and maintain our products We pride ourselves in excellence adopting best practices and holding ourselves to the highest standards Main responsibilities You will be LI-hybrid based in Hyderabad and reporting to Director Engineering Develop and maintain batch data processing pipelines using Scala and Apache Spark Optimize Spark jobs for performance and scalability in a cloud-native environment AWS You will collaborate with data engineers and analysts to understand data requirements and deliver high-quality solutions Ensure data integrity reliability and consistency across systems Participate in code reviews design discussions and agile development practices Monitor and troubleshoot production jobs ensuring resolution of issues Qualifications 3-6 years of software development experience with at least 2 years in Scala Write clean scalable and re-usable code that implements SOLID principles common design patterns where applicable and adheres to published coding standards Strong understanding of functional programming and distributed computing Strong experience with DevOps IaC tooling - Terraform CI CD pipelines Git Jenkins Hands-on experience with Apache Spark for batch processing Familiarity with AWS services such as S3 AWS Glue Lambda Step Functions AWS Batch and CloudWatch Proficiency in working with structured and semi-structured data formats e g Parquet JSON Experience of the full development lifecycle application Lifecycle Management Tools - BitBucket Jira Experience with data lake architectures and ETL frameworks Exposure to containerization tools Docker Kubernetes Knowledge of data governance and security best practices Relevant certifications in AWS Spark or Scala Additional Information Our uniqueness is that we celebrate yours Experian s culture and people are important differentiators We take our people agenda very seriously and focus on what matters DEI work life balance development authenticity collaboration wellness reward recognition volunteering the list goes on Experian s people first approach is award-winning World s Best Workplaces TM 2024 Fortune Global Top 25 Great Place To Work TM in 24 countries and Glassdoor Best Places to Work 2024 to name a few Check out Experian Life on social or our Careers Site and Glassdoor to understand why Experian is proud to be an Equal Opportunity and Affirmative Action employer Innovation is a critical part of Experian s DNA and practices and our diverse workforce drives our success Everyone can succeed at Experian and bring their whole self to work irrespective of their gender ethnicity religion color sexuality physical ability or age If you have a disability or special need that requires accommodation please let us know at the earliest opportunity Benefits Experian care for employee s work life balance health safety and wellbeing In support of this endeavor we offer best-in-class family well-being benefits enhanced medical benefits and paid time off Experian Careers - Creating a better tomorrow together