Lead Data Engineer

Job OverviewWe are looking for a Data Engineer who will be part of our Analytics Practice and will be expected to actively work in a multi-disciplinary fast paced environment. This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project; its primary responsibility is the acquisition, transformation, loading and processing of data from a multitude of disparate data sources, including structured and unstructured data for advanced analytics and machine learning in a big data environment.

Responsibilities:

• Engineer a modern data pipeline to collect, organize, and process data from disparate sources.
• Performs data management tasks, such as conduct data profiling, assess data quality, and write SQL queries to extract and integrate data
• Develop efficient data collection systems and sound strategies for getting quality data from different sources
• Consume and analyze data from the data pool to support inference, prediction and recommendation of actionable insights to support business growth.
• Design and develop ETL processes using tools and scripting. Troubleshoot and debug ETL processes. Performance tuning and opitimization of the ETL processes.
• Provide support to new of existing applications while recommending best practices and leading projects to implement new functionality.
• Collaborate in design reviews and code reviews to ensure standards are met. Recommend new standards for visualizations.
• Learn and develop new ETL techniques as required to keep up with the contemporary technologies.
• Reviews the solution requirements and architecture to ensure selection of appropriate technology, efficient use of resources and integration of multiple systems and technology.
• Support presentations to Customers and Partners
• Advising on new technology trends and possible adoption to maintain competitive advantage

Experience Needed:

• 8+ years of related experience is required.
• A BS or Masters degree in Computer Science or related technical discipline is required
• ETL experience with data integration to support data marts, extracts and reporting
• Experience connecting to varied data sources
• Excellent SQL coding experience with performance optimization for data queries.
• Understands different data models like normalized, de-normalied, stars, and snowflake models. Worked with transactional, temporarl, time series, and structured and unstructured data.
• Experience on Azure Data Factory and Azure Synapse Analytics
• Worked in big data environments, cloud data stores, different RDBMS and OLAP solutions.
• Experience in cloud-based ETL development processes.
• Experience in deployment and maintenance of ETL Jobs.
• Is familiar with the principles and practices involved in development and maintenance of software solutions and architectures and in service delivery.
• Has strong technical background and remains evergreen with technology and industry developments.
• At least 3 years of demonstrated success in software engineering, release engineering, and/or configuration management.
• Highly skilled in scripting languages like PowerShell.
• Substantial experience in the implementation and exectuion fo CI/CD processes.

Additional Requirements

• Demonstrated ability to have successfully completed multiple, complex technical projects
• Prior experience with application delivery using an Onshore/Offshore model
• Experience with business processes across multiple Master data domains in a services based company
• Demonstrates a rational and organized approach to the tasks undertaken and an awareness of the need to achieve quality.
• Demonstrates high standards of professional behavior in dealings with clients, colleagues and staff.
• Is able to make sound and far reaching decisions alone on major issues and to take full responsibility for them on a technical basis.
• Strong written communication skills. Is effective and persuasive in both written and oral communication.
• Experience with gathering end user requirements and writing technical documentation
• Time management and multitasking skills to effectively meet deadlines under time-to-market pressure