Observability Platform Engineer

1 month ago

Bengaluru, India StoneX Full time
Overview

The Observability Engineer takes on the critical role of utilizing a ETL platform (extraction, transformation, and loading) to effectively manage and streamline log data flows. Their responsibilities encompass ensuring data integrity, security, and compliance. They collaborate closely with IT teams, engaging in tasks such as designing, implementing, and troubleshooting ETL platform configurations tailored to meet the distinct data processing needs of financial services organizations. The Observability Engineer plays a pivotal role in the design, construction, and ongoing maintenance of data pipelines and infrastructure. These elements are essential in facilitating data-driven decision-making throughout the organization. Your expertise in observability engineering technologies and best practices will prove invaluable in our quest to ensure data accuracy, availability, and security while also lending support to various data-driven initiatives.

Responsibilities

Design and deploy ETL solutions to optimize log data ingestion, transformation, and routing.Create custom pipelines and configurations tailored to the specific requirements of financial applications and systems.Develop and maintain log data transformation rules and functions to extract relevant information and enrich log events.Configure ETL platform to intelligently route log data to appropriate destinations, such as SIEM (Security Information and Event Management) systems, data lakes, or real-time monitoring tools.Implement filtering mechanisms to reduce noise and focus on critical events.Aid in the development of log metrics, including both KPI’s and KRI’s.Monitor and fine-tune ETL platform configurations to improve data processing efficiency and reduce latency.Monitor and optimize data pipelines for performance and scalability, identifying and resolving bottlenecks as needed.Implement security best practices to protect sensitive data within log streams.Ensure compliance with industry regulations, such as PCI DSS, GDPR, and SEC requirements, by managing and auditing log data appropriately.Create documentation and provide training for IT and operational teams on ETL platform best practices and usage.Collaborate with cross-functional teams, including IT, DevOps, and security, to integrate ETL platform seamlessly into the financial services infrastructure.Aid in the development of a log bill back model and proper reporting around that model.Work with vendors and support teams to resolve technical issues.

Qualifications

Proven experience in designing, implementing, and managing ETL solutions or with big data technologies (e.g., Hadoop, Spark) and data lake architectures.Strong understanding of log data processing, log management, and log analytics.Proficiency in scripting and programming languages (e.g., Python, JavaScript) for custom log data transformations.Knowledge of security industry standards (NIST, PCI, ISO), best practices (ITIL), regulations (SOX, FINRA), news, and other relevant cybersecurity information.Individual must be able to perform with minimal supervision of routine duties; must demonstrate ability to solve practical problems and deal with a variety of situations where only limited standardization exists.Communicate technical issues and concepts both verbally and in writing to audiences of varying technical backgrounds.Ability to help gather business requirements and align them to security solutions.Working a minimum of 3 days in the office.Willingness to work on US or London time zone hours.Expert organizational skills are a must as well as the ability to work successfully within a team.1-3+ years of overall technology experience. Prefer a diverse background including security, networking, compute, storage, and endpoint.1-3+ years of experience as a data engineer or log engineer.Education / Certification requirements:

in at least one of or working towards the following:Associate Degree or Bachelor’s degree in information security, Information Assurance, Information Systems, or equivalent experience.SANS related certifications (GSEC, GCIA, GCED, GCIH etc.).Other field appropriate certifications may be considered.Physical requirements/Working conditions:Climate controlled office environment.Minimal physical requirements other than occasional light lifting of boxed materials.Dynamic, time-sensitive environment.

  • Observability Engineer

    5 days ago

    Bengaluru, India IG Group Full time

    Job Description Observability Engineer Are you passionate about data-driven insights and leveraging them to optimize both the reliability of complex systems and delivery pipelines? Do you thrive in a collaborative environment working alongside developers and operations to achieve shared goals? Is so, this Observability Engineer role in our global...

  • Observability Engineer

    1 month ago

    Bengaluru, India Encora Inc. Full time

    Important InformationExperience: 3 to 5 yearsJob Location: Bangalore/Hyderabad/PunePosition Type: Full time.Work Mode- HybridRole: Observability EngineerJob...

  • Observability Engineer

    2 months ago

    Bengaluru, India Encora Inc. Full time

    Important Information Experience: 3 to 5 years Job Location: Bangalore/Hyderabad/Pune Position Type: Full time. Work Mode- Hybrid Role: Observability Engineer Job Summary: ...

  • Lead Observability Platform Engineer

    4 weeks ago

    Bengaluru, India BNC Global Services Pvt. Ltd. Full time

    Job Description :1. Lead team of Observability Engineers to design and implement and maintain scalable, reliable, performant, and efficient systems.2. Drive continuous improvements in our operating processes.3. Nurture engineer growth through mentorship4. Create site reliability best-practices to ensure platform resiliency, performance and...

  • Software Engineer, Observability

    5 days ago

    Bengaluru, India Dialpad, Inc. Full time

    About Dialpad Work Beautifully Dialpad is the leading Ai-Powered Customer Intelligence Platform that is transforming how the world works together. Based on 4 billion minutes of analyzed voice and meetings data and growing, we have designed one, beautiful workspace that seamlessly combines the most advanced Ai Contact Center, Ai Sales, Ai Voice, and Ai...

  • Lead Engineer

    3 weeks ago

    Bengaluru, India Kenvue Full time

    Lead EngineerWhat You will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...

  • Staff Engineer _ Observability

    3 weeks ago

    Bengaluru, India Kenvue Full time

    Staff EngineerWhat You Will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...

  • Staff Engineer _ Observability

    4 weeks ago

    Bengaluru, Karnataka, India Kenvue Full time

    Staff EngineerWhat You Will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...

  • Lead Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Kenvue Full time

    Lead EngineerWhat You will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...

  • Staff Engineer _ Observability

    4 weeks ago

    Bengaluru, India Kenvue Full time

    Staff Engineer What You Will Do ? As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will...

  • Lead Engineer

    4 weeks ago

    Bengaluru, India Kenvue Full time

    Lead Engineer What You will Do ? As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need...

  • Lead Engineer

    4 weeks ago

    Bengaluru, India Kenvue Full time

    Lead EngineerWhat You will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...

  • Staff Engineer _ Observability

    4 weeks ago

    Bengaluru, India Kenvue Full time

    Staff EngineerWhat You Will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...

  • Software Engineer

    1 week ago

    Bengaluru, India Cloudera Full time

    Job Description:At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for...

  • IT Architecture Associate-Observability Engineer

    2 months ago

    Bengaluru, India Teva Pharmaceuticals Full time

    The opportunityWe are seeking an experienced Observability Architect to join our dynamic team. As an Observability Architect, you will play a key role in designing and implementing observability solutions that enable comprehensive monitoring, logging, and tracing across our systems. You will work closely with cross-functional teams to ensure that our...

  • Software Engineer

    1 week ago

    Bengaluru, India Cloudera Full time

    Job Description :  At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation...

  • Senior Software Engineer, Observability DevOps

    1 month ago

    Bengaluru, India Roku Full time

    Teamwork makes the stream work.Roku is changing how the world watches TVRoku is the #1 TV streaming platform in the US, and we've set our sights on powering every television in the world. Roku pioneered streaming to the TV. Our mission is to be the TV streaming platform that connects the entire TV ecosystem. We connect consumers to the content they love,...

  • Lead Engineer

    3 weeks ago

    Bengaluru, India Kenvue Full time

    Lead EngineerWhat You will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...

  • Senior Software Engineer, Observability DevOps

    2 months ago

    Bengaluru, India Roku Full time

    Teamwork makes the stream work. Roku is changing how the world watches TV Roku is the #1 TV streaming platform in the US, and we've set our sights on powering every television in the world. Roku pioneered streaming to the TV. Our mission is to be the TV streaming platform that connects the entire TV ecosystem. We connect consumers to the content...

  • Lead Engineer

    4 weeks ago

    Bengaluru, India Kenvue Full time

    Lead EngineerWhat You will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...