Senior Data Engineer

Job Summary :

We're seeking a highly skilled and experienced Senior Data Engineer with 6-8 years of expertise to join our team in New Delhi.

This critical role focuses on building and optimizing our real-time data infrastructure, leveraging a powerful suite of TIBCO technologies and cutting-edge databases.

You'll be instrumental in developing high-performance data pipelines, managing complex ETL processes, ensuring data quality, and enabling unified data access to support real-time analytics and operational monitoring, particularly within demanding environments like refinery Responsibilities :

- Real-time Data Processing : Design, develop, and deploy real-time event processing pipelines using TIBCO StreamBase for critical applications, such as monitoring complex refinery operations.

- ETL Workflow Development : Design and implement robust ETL (Extraction, Transformation, Loading) workflows with TIBCO Jasper ETL to support advanced data analytics and reporting initiatives.

- Database Management & Optimization : Manage, optimize, and ensure the high performance of SingleStore Database instances for real-time data storage and retrieval, handling large volumes of transactional data.

- Master Data Management (MDM) : Support master data governance initiatives using TIBCO EBX, ensuring data consistency, accuracy, and quality across the enterprise.

- Data Virtualization : Integrate heterogeneous data sources through advanced Data Virtualization techniques to enable seamless and unified data access for various business users and applications.

- Cross-functional Collaboration : Work closely with cross-functional teams, including business stakeholders, data scientists, and operations personnel, to gather requirements, understand data needs, and deliver scalable data solutions.

- Production Support & Monitoring : Monitor and troubleshoot production data systems on both Linux and Windows environments, ensuring high availability, performance, and reliability of critical data pipelines.

- Documentation : Create and maintain comprehensive technical designs, detailed workflows, and clear operational procedures for knowledge sharing, compliance, and future & Tools :

- TIBCO StreamBase : Expertise in real-time event processing and complex event processing (CEP).

- TIBCO Jasper ETL : Proficient in data extraction, transformation, and loading processes.

- SingleStore Database : Experience with distributed SQL databases for real-time analytics and high-speed data ingestion.

- TIBCO EBX : Knowledge of Master Data Management (MDM) concepts and implementation.

- Data Virtualization : Experience with tools and concepts for unified data access across disparate systems.

- Operating Systems : Familiarity with Linux and Windows environments for system monitoring and Skills & Qualifications :

- Experience : Minimum of 6-8 years of hands-on experience as a Data Engineer, with a strong focus on real-time data processing and enterprise data solutions.

- Technical Proficiency : Proven expertise in the listed technologies, especially TIBCO StreamBase, TIBCO Jasper ETL, and SingleStore Database.

- Data Modeling & Architecture : Solid understanding of data modeling principles, database design, and data architecture patterns for high-performance and scalable systems.

- Problem-Solving : Excellent analytical and problem-solving skills, with the ability to troubleshoot complex data pipeline issues.

- Communication : Strong verbal and written communication skills to collaborate effectively with technical and non-technical stakeholders.

- Collaboration : Proven ability to work effectively within cross-functional teams and manage multiple priorities.

- Education : Bachelor's or Master's degree in Computer Science, Information Technology, or a related engineering discipline