Big Data Infrastructure Specialist

1 week ago

Hyderabad, Telangana, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,10,00,000
Job Opportunity:

We are seeking a seasoned expert to spearhead the development and operation of our big data infrastructure projects. This role involves designing, building, and maintaining reliable and efficient data infrastructure that supports scalable data processing and analytics for both internal and external customers.

Your Key Responsibilities:
  • Develop Data Processing and Analytics Services: Utilize our big data stack, including Spark, Trino, Airflow, and Kafka, to create scalable data processing and analytics services supporting real-time and batch data workflows.
  • Architect Distributed Systems: Design, develop, and operate resilient distributed systems managing thousands of compute nodes across multiple data centers, ensuring scalability and high availability.
  • Troubleshoot and Drive Innovation: Resolve complex technical challenges and drive innovations enhancing system resilience, availability, and performance.
  • Service Ownership and Live-Site Management: Manage the full lifecycle of services, balancing live-site reliability, feature development, and technical debt retirement. Maintain a strong focus on service reliability and performance.
  • On-Call Support: Participate in the team's on-call rotation to address complex, real-time issues, keeping critical services operational and highly available.
  • Mentor and Guide Team Members: Provide mentorship and technical guidance to junior engineers, fostering growth, collaboration, and knowledge-sharing within the team.

Required Skills and Qualifications:

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field with 8+ years of experience in distributed systems, big data, or similar roles.
  • Cloud Expertise: Proficiency in cloud platforms (AWS, GCP, or Azure), containerization (Docker, Kubernetes), and infrastructure-as-code (Terraform, Ansible).
  • Big Data Expertise: Hands-on experience with Hadoop, Spark, Trino (or similar SQL query engines), Airflow, Kafka, and related ecosystems.
  • Programming Skills: Strong skills in Python, Java, Scala, or other programming languages relevant to distributed systems.
  • Distributed Systems Knowledge: Solid understanding of distributed computing principles, data partitioning, fault tolerance, and performance tuning.
  • Analytical and Problem-Solving Skills: Proven ability to troubleshoot complex system issues, optimizing for speed, efficiency, and scale.

  • Data Infrastructure Specialist

    1 week ago

    Hyderabad, Telangana, India beBeeDatabase Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Title: Data Infrastructure SpecialistJob Description: In this pivotal role, you will be responsible for designing and implementing data infrastructure solutions that ensure seamless data flow across our enterprise. This is a high-impact position that requires expertise in database management systems, data warehousing, and big data analytics. As a skilled...

  • Data Infrastructure Expert

    7 days ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data SpecialistAs a key contributor to our data infrastructure, you will play a pivotal role in designing, building, and maintaining large-scale data processing systems. You will collaborate with cross-functional teams to develop scalable and efficient solutions using Google Cloud Platform services.Key Responsibilities:Design and implement data...

  • Big Data DevOps Lead

    2 weeks ago

    Hyderabad, Telangana, India Zeta Global Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Summary: As a Senior or Lead Big Data DevOps Engineer, you will be working with a team responsible for setting up, scaling, and maintaining Big Data infrastructure and tools in private and public cloud environments.Main Responsibilities: Driving improvement of the efficiency of Big Data infrastructure.Coordinating cross-team infrastructure and Big Data...

  • Big Data Engineering Specialist

    7 days ago

    Hyderabad, Telangana, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking a seasoned data engineering professional to spearhead the development and implementation of cutting-edge big data analytics platforms.Key Responsibilities:Design and implement robust big data pipelines utilizing Spark, Kafka, Glue, EMR, Hudi, Schema Registry, and Data Lineage.Develop expertise in graph databases such as Neo4j,...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data ArchitectData architects play a crucial role in designing and implementing large-scale data systems that enable business growth. As a seasoned professional in the field of big data, you will be responsible for leading the development of data infrastructure that drives performance tuning of Spark jobs.Key Responsibilities:Expert-level...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Description:To design, develop and maintain large-scale data systems that meet business needs.You will create efficient pipelines against multiple data sources such as SQL, Big Data on cloud platforms. Large, complex datasets will be processed to meet both functional and non-functional business requirements.Existing data models, pipelines, related...

  • Big Data Leader Position

    2 weeks ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 1,42,61,900 - ₹ 2,51,23,100

    Job Overview:About the RoleThis is a high-level leadership position responsible for overseeing data engineering teams in a distributed environment.As Big Data Leader, you will drive business outcomes by leading cross-functional teams and collaborating with stakeholders to achieve strategic goals.The ideal candidate will have expertise in big data...

  • Senior Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Veltris Full time

    Position-Senior Big Data EngineerMust have Big Data analytics platform experience.• Key stacks: Spark, Druid, Drill, ClickHouse.• 8 years experience in Python/Java, CI/CD, infrastructure & cloud, Terraform, plus depth in:           o Big Data pipelines: Spark, Kafka, Glue, EMR, Hudi, Schema Registry, Data Lineage.           o Graph...

  • Big Data Solutions Architect

    2 weeks ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Big Data EngineerWe are seeking a seasoned Senior Big Data Engineer to spearhead the design, development, and implementation of cutting-edge Big Data analytics platforms.About the Role:Design, develop, deploy and maintain scalable Big Data pipelines using technologies such as Spark, Kafka, Glue, EMR, Hudi, Schema Registry, and Data Lineage.Leverage...

  • Big Data Solutions Architect

    2 weeks ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title:Big Data Solutions ArchitectAbout the Role:We are seeking a highly skilled and experienced Big Data Solutions Architect to join our organization. As a key member of our data engineering team, you will be responsible for designing, developing, and deploying scalable big data solutions using Scala, Apache Spark, and cloud-based platforms.Key...