Big Data Platform Engineer

2 weeks ago

Pune, Maharashtra, India Cognyte Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Big Data Platform Engineer / BD Site Reliability Engineer (SRE) - Job Requirements

Position Overview

We are seeking a Big Data Platform Engineer to support our enterprise customers' big data infrastructure deployments and operations. This role requires a hands-on technical professional who can work both remotely and on-site at customer locations, including weekend emergency support when critical issues arise.

Key Responsibilities

  • Customer Site Support: Provide ongoing technical support for customer big data clusters and installations
  • Installation & Upgrades: Lead and support installation/upgrade projects both remotely (back office support) and on-premises at customer sites
  • Travel Requirements: Willingness to travel to customer sites as needed for installations, upgrades, and critical issue resolution
  • Emergency Response: Available for weekend/after-hours support during customer site crises
  • Installation Code Development: Develop, maintain, and fix installation automation scripts and tools
  • CI/CD Pipeline Management: Build and maintain continuous integration/deployment pipelines
  • Monitoring Solutions: Develop and enhance monitoring tools for big data infrastructure

Required Technical Skills
Core Infrastructure & Automation

  • Ansible: Advanced proficiency in playbook development and infrastructure automation
  • Bash: Strong shell scripting capabilities for system administration and deployment
  • Python: Solid programming skills for automation tools and utilities development
  • RedHat Linux: Deep knowledge of RedHat Enterprise Linux distribution, system administration, and package management

CI/CD & Virtualization

  • Jenkins: Experience building and maintaining CI/CD pipelines
  • VMware vSphere: Virtual infrastructure management and deployment

Big Data Ecosystem (Required)

  • Apache Spark: Cluster configuration, tuning, and troubleshooting
  • YARN: Resource management and cluster administration
  • HDFS: Distributed file system management and optimization
  • Apache Kafka: Streaming platform deployment and maintenance
  • Apache ZooKeeper: Coordination service configuration and management

Preferred (Bonus) Skills

  • Kubernetes: Container orchestration and cloud-native deployments
  • Vanilla, Open Shift, RKE
  • Harbor: Container registry management
  • Longhorn: Distributed storage solutions

Requirements:

Work Environment & Requirements

  • Hybrid Role: Mix of remote work and customer site visits
  • Travel: Must be willing and able to travel to customer sites domestically and potentially internationally
  • Availability: Flexible schedule including weekend emergency support capability
  • Customer-Facing: Strong communication skills for direct customer interaction during installations and support

Ideal Candidate Profile

  • 3-5 years experience with big data infrastructure and automation
  • Proven track record in customer-facing technical roles
  • Strong problem-solving skills under pressure
  • Self-motivated with ability to work independently at customer sites

  • Big Data Platform Engineer

    2 weeks ago

    Pune, Maharashtra, India Cognyte Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Big Data Platform Engineer / BD Site Reliability Engineer (SRE) - Job RequirementsPosition OverviewWe are seeking a Big Data Platform Engineer to support our enterprise customers' big data infrastructure deployments and operations. This role requires a hands-on technical professional who can work both remotely and on-site at customer locations, including...

  • Big Data Engineer

    1 week ago

    Pune, Maharashtra, India Talent Sketchers Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Designation: Big Data EngineerExperience: 4+ YearsWork Mode: Remote OpportunityNotice Period: Immediate Joiners/ Serving Notice PeriodJob Description:Role: Big Data EngineerThis Data Engineer will be engaged in data science-related research and software application development and engineering duties related to our enterprise-grade Wi-Fi technology to provide...

  • Big Data Engineer

    6 days ago

    Pune, Maharashtra, India Vodafone Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

    Design, develop, and maintain scalable data pipelines for processing structured and unstructured data. Build and optimise data storage solutions including distributed file systems, NoSQL databases, and data warehouses. Utilise big data frameworks such as Apache Hadoop, Spark, Flink, and Kafka to develop data processing workflows. Ensure high performance and...

  • Big Data Engineer

    2 days ago

    Pune, Maharashtra, India Funic Tech Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Opening: Big Data EngineerLocation: Aundh, Pune (Work From Office)Job Type: Full-Time | General Shift | Monday – FridayKey ResponsibilitiesBuild and manage highly scalable Big Data pipelines for large structured datasets.Collaborate with Product Management & Engineering leadership to design and implement optimal solutions.Participate in design...

  • Big Data Lead

    7 days ago

    Pune, Maharashtra, India Populace World Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Designation: Big Data Lead/ArchitectLocation: PuneExperience: 8-10 yearsNP - immediate joiner/15-30 days' noticeJob OverviewWe are looking to hire a talented big data engineer to develop and manage our company's Big Data solutions. In this role, you will be required to design and implement Big Data tools and frameworks, implement ELT processes, collaborate...

  • Big Data Developer

    4 days ago

    Pune, Maharashtra, India airtel Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Job Title:Big Data Developer (Manager)Location: PuneExperience: 3-5 yearsJob Description:Looking for an experienced Big Data Developer to develop, maintain, and optimize our big data solutions. The ideal candidate will have strong expertise in Java, Spark, API development, and a deep understanding of data engineering and big data tools.Key...

  • Senior Java Big Data Engineer

    5 days ago

    Pune, Maharashtra, India 6b7ed692-3bb5-4e80-abde-f1d5ee352806 Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    We are seeking a Senior Java Big Data Engineer to design, develop, and optimize large-scale, high-performance data processing and backend systems. The ideal candidate will have a strong foundation in java, Soring Boot, RESTful APIs, and big data streaming platforms (especially Kafka),with additional experience in Azure Cloud developments and CI /CD...

  • Big Data

    2 days ago

    Pune, Maharashtra, India Atgeir Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Minimum Experience: 10 YearsWe are looking for a Big Data / Cloud Architect to become part of Atgeir's Advanced Data Analytics team. The desired candidate is a Professional with proven track record of working on Big Data and Cloud Platforms.Required Skills:Work closely with customers, understand customer requirements and render those as architectural...

  • Data Engineer-Data Engineering-Big Data Engineering

    2 weeks ago

    Pune, Maharashtra, India EXL Talent Acquisition Team Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Summary:We are seeking an experienced GCP Data Engineer with strong expertise in designing, building, and managing scalable data pipelines and analytical solutions on Google Cloud Platform (GCP). The ideal candidate will have a solid background in big data technologies, data warehousing, and cloud-based data architecture, with hands-on experience using...

  • Data Engineer-Data Engineering-Big Data Engineering

    2 weeks ago

    Pune, Maharashtra, India EXL Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    DescriptionJob Summary:We are seeking an experienced GCP Data Engineer with strong expertise in designing, building, and managing scalable data pipelines and analytical solutions on Google Cloud Platform (GCP). The ideal candidate will have a solid background in big data technologies, data warehousing, and cloud-based data architecture, with hands-on...