Lead Data Engineer

Company Description Informa is a leading academic publishing business intelligence knowledge and events business creating unique content and connectivity for customers all over the world It is listed on the London Stock Exchange and is a member of the FTSE 100 Taylor Francis is one of the world s largest publishers of high-quality peer reviewed scholarly journals books e-books and reference works We empower learners thinkers and doers with trusted knowledge that advances research and enriches lives We bring together diverse people ideas and opinions to validate and publish the experts theories and truths that matter And we ve been doing this in an unbroken arc since 1798 Our purpose is to foster human progress through knowledge We strongly believe that this requires us to encourage and examine different ideas and voices so that any work that meets our exacting levels of quality deserves to be included in our publications This requires open minds the opportunity for robust debate and the courage to defend perspectives that stand up to scrutiny even if they conflict with our personal beliefs or values Because that s the only way to find the best obtainable version of the truth and ultimately foster human progress If you agree then we d love to hear from you Closing Date Applications will close on 11th November 2025 What you will be doing Function as part of an Agile engineering team in the role of Lead Data Engineer responsible for orchestrating and upholding the highest levels of Engineering and Enterprise Data Warehouse Concepts and practices throughout the Software Development Life Cycle collaborating closely with development operations and architecture teams Define delivery phases and perform project task and individual management duties for team setting activities sub-activities and milestones while ensuring that development and delivery standards are rigorously enforced Manage and enhance our Continuous Integration Delivery and Deployment processes teams and users to ensure high-quality and efficient delivery Collaborate closely with Engineering Managers and Business Analysts to produce accurate delivery estimates and oversee the transition from analysis through design to deliver Design and implement scalable data solutions using AWS services Design and implement data models for warehouse solutions Create and maintain Infrastructure as Code using CloudFormation Implement comprehensive unit testing strategies Work closely with cross-functional teams to gather requirements Mentor junior engineers on AWS best practices and data engineering concepts Participate in code reviews and technical discussions Document technical solutions and maintain knowledge base Deploy and troubleshoot solutions as needed adhering to best practices throughout the development lifecycle to identify and resolve issues prior to deployment Plan design and develop cloud-based applications analytical services and platforms Ensure adherence to company security guidelines and requirements Qualifications What we are looking for An enthusiastic team member who fosters collaborative relationships advocate for technical excellence and adapt to evolving requirements within a dynamic team environment Demonstrate excellent communication organizational skills and a proactive approach to meeting deadlines and achieving project goals Proven experience as a Lead Data Engineer with expertise in Python and Spark Experience in CI CD pipelines using GitHub and GitHub Actions to streamline development workflows Develop and maintain ETL pipelines using AWS Glue Lambda and EventBridge Manage data storage solutions across S3 and Redshift Implement security best practices using AWS IAM Experience with cloud computing services especially AWS ECS RDS EFS S3 Lambda to build scalable and resilient applications Experience with database management including SQL e g MySQL and NoSQL e g MongoDB ensuring efficient data storage and retrieval Ensure code quality through SonarCloud integration Create and maintain Infrastructure as Code using CloudFormation In-depth knowledge in Test Driven Development TDD for Data Pipelines and Python Excellent container orchestration skills using Docker ensuring efficient application deployment and management Strong understanding of Agile methodologies and implementations Kanban XP Scrum Excellent problem-solving and troubleshooting skills Process-oriented with great documentation skills Bachelor s degree in computer science or a related field or equivalent work experience preferred Additional Information What we offer in return 24 days annual leave 4 volunteering days annually Day off for your birthday Pension contributions Medical insurance for self and dependants life cover and personal accident cover for self Seasonal social and charitable events Training and development Blended style and flexi working time Right tools for remote working Additional Information You must have the right to work and live in Bengaluru India We believe that great things happen when people connect face-to-face That s why we work in-person with each other or with customers and partners three days a week or more When you re not spending time together in one of our offices or other workplaces - like at conference - you get the flexibility and support to work from home or remotely The successful candidate will be able to utilise our balanced working model We ask that candidates live within a commutable distance of our office in Bengaluru to ensure that as a balanced worker they can make it to our office location 3 days per week We want all our candidates to shine in our recruitment process Please let us know if there is anything we can do to ensure you are able to show us your best self This could include having the application form in a different format more time for questions or anything else - please ask us we are happy to be flexible Please contact Being Yourself at Taylor Francis If you re excited about working with Taylor Francis to foster human progress through knowledge we invite you to apply even if your existing skills and experience don t fit every item listed above At Taylor Francis we are at our best and most successful when colleagues can be themselves and contribute regardless of their identity or background As a colleague you will have the opportunity to further innovate and develop in areas that you are passionate about Our goal is to empower you with the resources incentives and flexibility you need to enjoy success at work and to live a healthy balanced life Taylor Francis is proud to be an Equal Opportunity Employer We believe in and value diversity of people and thought fostering a supportive and inclusive environment where all colleagues can learn and succeed as their true selves Taylor Francis recruits develops and retains colleagues without regard to any protected personal characteristics or other non-merit-based factor If you have a disability or face another barrier and would like to request an adjustment to assist your participation in the hiring process and or in the advertised position please contact Ivy Ma Your request will be reviewed and considered in the strictest confidence We genuinely care about our colleagues promoting work-life balance wellbeing and flexible working We believe that the skills and experience you bring to Taylor Francis are invaluable We want you to have the opportunity to develop your abilities and to innovate and develop in areas which you are passionate about To find out more about our business and the great career opportunities please go to our Careers Site