Data Engineer

Job Title: Data Engineer

Location: Hyderabad

About Blurgs AI

Blurgs AI is a deep-tech startup focused on maritime and defence data-intelligence solutions, specialising in multi-modal sensor fusion and data correlation.

Our flagship product, Trident, provides advanced domain awareness for maritime, defence, and commercial sectors by integrating data from various sensors like AIS, Radar, SAR, and EO/IR.

At Blurgs AI, we foster a collaborative, innovative, and growth-driven culture. Our team is passionate about solving real-world challenges, and we prioritise an open, inclusive work environment where creativity and problem-solving thrive.

We encourage new hires to bring their ideas to the table, offering opportunities for personal growth, skill development, and the chance to work on cutting-edge technology that impacts global defence and maritime operations.

Join us to be part of a team that's shaping the future of technology in a fast-paced, dynamic industry.

Job Summary:

We are looking for a Senior Data Engineer to design, build, and maintain a robust, scalable on-premise data infrastructure. You will focus on real-time and batch data processing using platforms such as Apache Pulsar and Apache Flink, work with NoSQL databases like MongoDB and ClickHouse, and deploy services using containerization technologies like Docker and Kubernetes. This role is ideal for engineers with strong systems knowledge, deep backend data experience, and a passion for building efficient, low-latency data pipelines in a non-cloud, on-prem environment.

Key Responsibilities:

Data Pipeline & Streaming Development

Design and implement real-time data pipelines using Apache Pulsar and Apache Flink to support mission-critical systems.

Develop high-throughput, low-latency data ingestion and processing workflows across streaming and batch workloads.

Integrate internal systems and external data sources into a unified on-prem data platform.

Data Storage & Modelling

Design efficient data models for MongoDB, ClickHouse, and other on-prem databases to support analytical and operational workloads.

Optimise storage formats, indexing strategies, and partitioning schemes for performance and scalability.

Infrastructure & Containerization

Deploy, manage, and monitor containerised data services using Docker and Kubernetes in on-prem environments.

Performance, Monitoring & Reliability

Monitor the performance of streaming jobs and database queries; fine-tune for efficiency and reliability.

Implement robust logging, metrics, and alerting solutions to ensure data system availability and uptime.

Identify bottlenecks in the pipeline and proactively implement optimisations.

Required Skills & Experience:

Strong experience in data engineering with a focus on on-premise infrastructure.

Strong expertise in streaming technologies like Apache Pulsar, Apache Flink, or similar.

Deep experience with MongoDB, ClickHouse, and other NoSQL or columnar storage databases.

Proficient in Python, Java, or Scala for data processing and backend development.

Hands-on experience deploying and managing systems using Docker and Kubernetes.

Familiarity with Linux-based systems, system tuning, and resource monitoring.

Preferred Qualifications:

Bachelor's or Master's degree in Computer Science, Engineering, or a related field, or an equivalent combination of education and experience.

Additional Responsibilities for Senior Data Engineers

For those hired as Senior Data Engineers, the role will come with added responsibilities, including:

Leadership & Mentorship: Guide and mentor junior engineers, sharing expertise and best practices.

System Architecture: Lead the design and optimization of complex real-time and batch data pipelines, ensuring scalability and performance.

Sensor Data Expertise: Focus on building and optimizing sensor-based data pipelines and stateful stream processing for mission-critical applications in domains like maritime and defense.

End-to-End Ownership: Take responsibility for the performance, reliability, and optimization of data systems.