Data Engineer

Reporting to the A/NZ DSE Chapter Manager India PEC within Decision Sciences & Engineering, this role will own and be responsible for the data & analytic engineering chapter in India PEC.

The Data Engineer is an essential part of the business that enables the team to support the ongoing acquisition and internal purposing of data, through to the fulfilment of products, insights and systems. As a Data Engineer, you will be responsible for working with our internal customers to ensure that data and systems are being designed and built to move and manipulate data in a scalable, reusable and efficient manner to suit the environment, project, security and requirements.

What you'll do

• Design, architect, and implement scalable and secure data pipelines on GCP, utilizing services like Dataflow, Pub/Sub, and Cloud Storage.
• Develop and maintain data models, ensuring data quality, consistency, and accessibility for various internal stakeholders.
• Automate data processes and workflows using scripting languages like Python, leveraging technologies like Spark and Airflow.
• Monitor and troubleshoot data pipelines, identifying and resolving performance issues
• proactively.
• Stay up-to-date with the latest trends and advancements in GCP and related technologies, actively proposing and evaluating new solutions.
• Implement data governance best practices, including data security, access control, and lineage tracking.
• Lead security initiatives, design and implement security architecture.
• Lead data quality initiatives, design and implement monitoring dashboards.
• Mentor and guide junior data engineers, sharing knowledge and best practices to foster a high-performing team.
• Role requires a solid educational foundation and the ability to develop a strategic vision and roadmap for D&A's transition to the cloud while balancing delivery of near-term results that are aligned with execution.

What experience you need

• BS degree in a STEM major or equivalent discipline; Master's Degree strongly preferred
• 8+ years of experience as a data engineer or related role, with experience demonstrating leadership capabilities
• Cloud certification strongly preferred
• Expert level skills using programming languages such as Python or SQL (Big Query) and advanced level experience with scripting languages.
• Demonstrated proficiency in all Google Cloud Services
• Experience building and maintaining complex data pipelines, troubleshooting complex issues, transforming and entering data into a data pipeline in order for the content to be digested and usable for future projects; Proficiency in Airflow strongly desired
• Experience designing and implementing advanced to complex data models and experience enabling advanced optimization to improve performance
• Experience leading a team with Git expertise strongly preferred
• Hands on Experience on Agile Methodoligies
• Working Knowledge of CI/CD

What could set you apart:

• Self-starter that identifies/responds to priority shifts with minimal supervision.
• Strong communication and presentation skills
• Strong leadership qualities
• A well-balanced view of resource management, thinking creatively and effectively to deploy the team whilst building skills for the future
• Skilled in internal networking, negotiating and proactively developing individuals and teams to be the best they can be
• Strong communicator & presenter, bringing everyone on the journey.
• Knowledge of Big Data technology and tools with the ability to share ideas among a collaborative team and drive the team based on technical expertise and learning, sharing best practices
• Excellent communication skills to engage with senior management, internal customers and product management
• Sound understanding of regulations and security requirements governing access to data in Big Data systems
• Sound understanding of Insight delivery systems for batch and online
• Should be able to run Agile Scrum-based projects
• Demonstrated problem solving skills and the ability to resolve conflicts
• Experience creating and maintaining product and software roadmaps
• Experience overseeing yearly as well as product/project budgets
• Working in a highly regulated environment