Logstash Engineer

We are seeking a skilled and experienced Logstash Engineer to join our team and help design, implement, and maintain log data pipelines. The successful candidate will have a deep understanding of Logstash and its role within the ELK stack, ensuring that data is efficiently ingested, transformed, and routed to support real-time analytics and insights. This position is critical in maintaining the performance, scalability, and reliability of our data processing systems.
Key Responsibilities:
·      Design, implement, and maintain Logstash pipelines to aggregate, parse, and forward logs from various sources (servers, applications, databases).
·      Develop Logstash configurations to handle data ingestion, transformation, and enrichment for real-time and batch data processing.
·      Collaborate with teams using Elasticsearch, Kibana, and other tools to provide effective data visualizations and reporting solutions.
·      Monitor and troubleshoot Logstash performance, optimizing throughput and minimizing latency.
·      Integrate Logstash with different input/output plugins (Kafka, RabbitMQ, HTTP, File, Beats, etc.).
·      Work closely with DevOps and Data Engineering teams to ensure the log management infrastructure is aligned with the broader platform architecture.
·      Stay updated with the latest features and updates in Logstash, recommending upgrades and enhancements to the system.
·      Document all Logstash pipelines and processes to ensure maintainability and scalability.
Qualifications:
·       5+ years of experience working with Logstash, preferably as part of an ELK stack implementation.
·      Bachelor’s degree in computer science, Information Systems, or a related field (or equivalent experience).
·      Strong knowledge of Logstash configuration files, filters, input/output plugins, and pipelines.
·      Familiarity with Elasticsearch and Kibana for data storage and visualization.
·      Experience with data parsing techniques (JSON, XML, CSV, etc.).

·      Familiarity with message brokers like Kafka or RabbitMQ and integrating them with Logstash.
·      Proficiency in Linux/Unix environments and scripting languages such as Bash, Python, or Ruby.
·      Experience with cloud platforms (AWS, GCP, Azure) and containerization tools (Docker, Kubernetes) is a plus.
·      Strong problem-solving skills, with the ability to diagnose and resolve complex issues related to data ingestion and processing.
Preferred Skills:
·      Experience in high-volume log ingestion environments.
·      Knowledge of log management tools such as Beats, Fluentd, or Graylog.
·      Familiarity with security best practices for log management and data processing.
·      Experience with distributed systems and maintaining high availability environments.