Observability Platform Engineer

1 month ago

bangalore, India StoneX Full time

Overview

The Observability Engineer takes on the critical role of utilizing a ETL platform (extraction, transformation, and loading) to effectively manage and streamline log data flows. Their responsibilities encompass ensuring data integrity, security, and compliance. They collaborate closely with IT teams, engaging in tasks such as designing, implementing, and troubleshooting ETL platform configurations tailored to meet the distinct data processing needs of financial services organizations. The Observability Engineer plays a pivotal role in the design, construction, and ongoing maintenance of data pipelines and infrastructure. These elements are essential in facilitating data-driven decision-making throughout the organization. Your expertise in observability engineering technologies and best practices will prove invaluable in our quest to ensure data accuracy, availability, and security while also lending support to various data-driven initiatives.

Responsibilities

Design and deploy ETL solutions to optimize log data ingestion, transformation, and routing. Create custom pipelines and configurations tailored to the specific requirements of financial applications and systems. Develop and maintain log data transformation rules and functions to extract relevant information and enrich log events. Configure ETL platform to intelligently route log data to appropriate destinations, such as SIEM (Security Information and Event Management) systems, data lakes, or real-time monitoring tools. Implement filtering mechanisms to reduce noise and focus on critical events. Aid in the development of log metrics, including both KPI’s and KRI’s. Monitor and fine-tune ETL platform configurations to improve data processing efficiency and reduce latency. Monitor and optimize data pipelines for performance and scalability, identifying and resolving bottlenecks as needed. Implement security best practices to protect sensitive data within log streams. Ensure compliance with industry regulations, such as PCI DSS, GDPR, and SEC requirements, by managing and auditing log data appropriately. Create documentation and provide training for IT and operational teams on ETL platform best practices and usage. Collaborate with cross-functional teams, including IT, DevOps, and security, to integrate ETL platform seamlessly into the financial services infrastructure. Aid in the development of a log bill back model and proper reporting around that model. Work with vendors and support teams to resolve technical issues.

Qualifications

Proven experience in designing, implementing, and managing ETL solutions or with big data technologies (e.g., Hadoop, Spark) and data lake architectures. Strong understanding of log data processing, log management, and log analytics. Proficiency in scripting and programming languages (e.g., Python, JavaScript) for custom log data transformations. Knowledge of security industry standards (NIST, PCI, ISO), best practices (ITIL), regulations (SOX, FINRA), news, and other relevant cybersecurity information. Individual must be able to perform with minimal supervision of routine duties; must demonstrate ability to solve practical problems and deal with a variety of situations where only limited standardization exists. Communicate technical issues and concepts both verbally and in writing to audiences of varying technical backgrounds.  Ability to help gather business requirements and align them to security solutions. Working a minimum of 3 days in the office. Willingness to work on US or London time zone hours. Expert organizational skills are a must as well as the ability to work successfully within a team. 1-3+ years of overall technology experience. Prefer a diverse background including security, networking, compute, storage, and endpoint. 1-3+ years of experience as a data engineer or log engineer.

Education / Certification requirements: in at least one of or working towards the following:

Associate Degree or Bachelor’s degree in information security, Information Assurance, Information Systems, or equivalent experience. SANS related certifications (GSEC, GCIA, GCED, GCIH etc.). Other field appropriate certifications may be considered.

Physical requirements/Working conditions:

Climate controlled office environment. Minimal physical requirements other than occasional light lifting of boxed materials. Dynamic, time-sensitive environment.

  • Lead Engineer

    2 weeks ago

    Bangalore, India Seikor Full time

    Designation: Lead Engineer - Observability - Platform EngineeringSkill Set: Opentelemetry (No kubernets, working on distributed tracing and custom matrics), Java Scripts /type scripts knowledge, AWS RDS, ARORA, SQS, SNS, Kinesis, API Gateway, DynamoDB, AWS LAMBDAExperience: 3-10 YearsCTC: 18-25LpaLocation: BangaloreWork Mode: WFOCompany: Motherson Technology...

  • Lead Observability Platform Engineer

    1 month ago

    bangalore, India BNC Global Services Pvt. Ltd. Full time

    Job Description : 1. Lead team of Observability Engineers to design and implement and maintain scalable, reliable, performant, and efficient systems.2. Drive continuous improvements in our operating processes.3. Nurture engineer growth through mentorship4. Create site reliability best-practices to ensure platform resiliency, performance and...

  • Observability Engineer

    1 month ago

    bangalore, India Encora Inc. Full time

    Important Information Experience: 3 to 5 years Job Location: Bangalore/Hyderabad/Pune Position Type: Full time. Work Mode- Hybrid Role: Observability Engineer Job Summary: ...

  • Lead Observability Platform Engineer

    2 weeks ago

    Bangalore/Hyderabad, India BNC Global Services Pvt. Ltd. Full time

    Job Description : 1. Lead team of Observability Engineers to design and implement and maintain scalable, reliable, performant, and efficient systems.2. Drive continuous improvements in our operating processes.3. Nurture engineer growth through mentorship4. Create site reliability best-practices to ensure platform resiliency, performance and...

  • Lead Engineer

    2 weeks ago

    bangalore, India Kenvue Full time

    Lead EngineerWhat You will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...

  • Staff Engineer

    2 weeks ago

    bangalore, India Kenvue Full time

    Staff EngineerWhat You Will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will need to...

  • Staff Engineer

    2 weeks ago

    bangalore, India Kenvue Full time

    Staff Engineer What You Will Do ? As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you will...

  • Senior DevOps Engineer

    1 week ago

    bangalore, India Caizin Full time

    Job Description: We're seeking a seasoned Senior DevOps Engineer who excels in observability and telemetry, adept at crafting, implementing, and managing robust monitoring and analytics solutions. In this pivotal role, you'll be instrumental in architecting and fine-tuning our observability and telemetry infrastructure, delivering actionable insights into...

  • Senior Software Engineer, Observability DevOps

    1 month ago

    bangalore, India Roku Full time

    Teamwork makes the stream work. Roku is changing how the world watches TV Roku is the #1 TV streaming platform in the US, and we've set our sights on powering every television in the world. Roku pioneered streaming to the TV. Our mission is to be the TV streaming platform that connects the entire TV ecosystem. We connect consumers to...

  • Senior Software Engineer

    2 weeks ago

    bangalore, India Kenvue Full time

    Senior Software EngineerWhat You Will Do ?As Observability engineers you are passionate about using software-based approaches to solve complex data-driven challenges and automate those solutions. You will lead efforts aimed at setting up enterprise observability solutions at scale and establish the technical strategy to achieve this. To be successful, you...

  • Senior Software Engineer

    1 month ago

    bangalore, India Fortanix Full time

    Fortanix is a dynamic start-up solving some of the world’s most demanding data protection challenges for companies and governments around the world. Our disruptive technology maintains data privacy across its entire lifecycle -- at rest, in motion, and in use across any enterprise IT infrastructure -- public cloud, on-premise, hybrid cloud, and SaaS. With...

  • Cloud Observation Engineer

    2 weeks ago

    Bangalore, India ARR Recruitment Solutions Full time

    Key Responsibilities :Cloud Infrastructure Monitoring :Implement and manage cloud infrastructure monitoring solutions to ensure the availability, performance, and security of cloud environments.Observability Systems Design :- Design and develop observability systems, utilizing tools such as Prometheus, Grafana, and AWS CloudWatch to provide insights into...

  • Senior Machine Learning Engineer

    4 weeks ago

    bangalore, India Kimberly-Clark Full time

    Description Your Job You’re raising the stakes for your career to do more. Learn more. Impact more. Here, your innovation, ideas, and technical curiosity will help us deliver better care for billions of people worldwide. You’ll put your professional expertise, talent, and drive to work by building and managing the technology behind our...

  • Victrix Systems and Labs

    2 weeks ago

    Bangalore, India Victrix Systems & Labs Full time

    Title: Senior AWS DevOps Engineer - Observability and Telemetry ExpertLocation: Bangalore(Hybrid)Job Description: We are looking for an experienced Senior AWS DevOps Engineer with a deep understanding of observability and telemetry principles and a proven track record in designing, implementing, and managing robust monitoring and analytics solutions on the...

  • Senior Cloud Platform Engineer

    1 month ago

    bangalore, India Calix Full time

    We, the Cloud Platform Engineering team at Calix are responsible for the Platforms, Tools, and CI/CD pipelines at Calix. Our mission is to enable Calix engineers to accelerate the delivery of world-class products while ensuring the high availability, reliability, security, and performance expected by our customers.We are seeking Cloud Engineers with proven...

  • Senior Cloud Platform Engineer

    2 days ago

    bangalore, India Calix Full time

    We, the Cloud Platform Engineering team at Calix are responsible for the Platforms, Tools, and CI/CD pipelines at Calix. Our mission is to enable Calix engineers to accelerate the delivery of world-class products while ensuring the high availability, reliability, security, and performance expected by our customers.We are seeking Cloud Engineers with proven...

  • Senior Staff Platform Engineer

    1 month ago

    bangalore, India SolarWinds Full time

    Your Role : We are seeking a Senior Staff Engineer to join our Developer Experience team in the SolarWinds Observability engineering org. Our goal is to help our developers succeed by providing an exceptional developer experience. You will work alongside other engineers in an agile development environment and share solutions to advance the technologies...

  • Data Migration Specialist

    2 weeks ago

    bangalore, India Whizz HR Full time

    Job Summary :As a Data Migration Specialist, you will play a critical role in ensuring a smooth transition of customers monitoring and observability infrastructure. Your expertise in various other observability tools, coupled with a strong understanding of PromQL, will be essential in successfully migrating alerts and dashboards, optimizing system...

  • Senior/Principal Product Manager, Observability Data Collection

    1 month ago

    bangalore, India Sumo Logic Full time

    Sumo Logic is a cloud-native SaaS data analytics platform that solves complex observability and security problems. Customers choose our product because it allows them to easily monitor, optimize, and secure their applications, systems, and infrastructures.  Sumo Logic is seeking an experienced product management leader to help define and drive the...

  • Technical Director Individual Contributor

    1 month ago

    bangalore, India Fortanix Full time

    About Us:Fortanix is a dynamic start-up solving some of the world's most demanding data protection challenges for companies and governments around the world. Our disruptive technology maintains data privacy across its entire lifecycle -- at rest, in motion, and in use across any enterprise IT infrastructure -- public cloud, on-premise, hybrid cloud, and...