Staff Cloud Platform Engineer

2 weeks ago


Bengaluru, India Calix Full time
This position is based in Bangalore, India.

We, the Cloud Platform Engineering team at Calix are responsible for the Platforms, Tools, and CI/CD pipelines at Calix. Our mission is to enable Calix engineers to accelerate the delivery of world-class products while ensuring the high availability, reliability, security, and performance expected by our customers.

We are seeking Platform Engineers with proven expertise in the design and building of large-scale Cloud Infrastructure Platforms for Datalake to join our team and help us deliver the foundation on which Calix products can thrive. You will help us achieve our mission through innovative solutions.

Responsibilities:

  • Design, build, and maintain a scalable and reliable cloud-native data lake infrastructure using Presto, Alluxio, S3, Hive and Spark technologies.
  • Implement integration with Zookeeper for coordination and leader election across Datalake components.
  • Configure Presto to access and query data stored in Alluxio and S3, optimizing performance and resource utilization.
  • Deploy and manage Spark clusters for data processing and analytics tasks, leveraging Alluxio and S3 as storage layers.
  • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and implement solutions that meet business objectives.
  • Develop and maintain automated workflows for data ingestion, processing, and transformation using Spark and other relevant tools.
  • Implement monitoring and alerting solutions to ensure the health and performance of the data lake infrastructure.
  • Provide technical guidance and support to junior team members, fostering a culture of continuous learning and improvement.

Requirements:

  • Bachelor's degree in Computer Science, Engineering, or related field; or equivalent work experience.
  • 10+ years of experience as a Data Engineer or similar role, with expertise in building and managing cloud-native data lake infrastructure.
  • Strong proficiency in Presto, Alluxio, S3, Spark, Hive and Zookeeper, with hands-on experience in designing and implementing solutions using these technologies.
  • Experience with Spark Administration, Hive administration, Presto administration and AWS resource management.
  • Experience with monitoring of distributed systems.
  • Experience with infrastructure as code tools such as Terraform/Ansible for provisioning and managing cloud resources.
  • Experience with container orchestration platforms such as Kubernetes for deploying and managing Datalake components.
  • Solid understanding of distributed computing principles, data modelling, and database technologies.
  • Experience with serverless architecture would be plus like Lambda and EMR
  • Experience in Observability and Monitoring in AWS.
  • Fast learner and able to troubleshoot complex scenarios while process large volumes of data (Terabytes and Petabytes)
  • Excellent problem-solving skills, with the ability to troubleshoot complex issues and implement effective solutions.
  • Strong communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.
  • Having Databricks experience will be an added advantage.

Optional Skills:

  • JVM tuning. 
  • Alluxio cluster management. 



  • Bengaluru, India Calix Full time

    This position is based in Bangalore, India.We, the Cloud Platform Engineering team at Calix are responsible for the Platforms, Tools, and CI/CD pipelines at Calix. Our mission is to enable Calix engineers to accelerate the delivery of world-class products while ensuring the high availability, reliability, security, and performance expected by our...


  • Bengaluru, India Pantheon Platform Full time

    Staff Software Engineer - File System TeamLocation: Bangalore, India About PantheonPantheon is the WebOps platform for websites that deliver extraordinary results. We believe in putting the magic of the internet in everyone’s hands. That’s why we’re so passionate about helping developers, IT and marketing develop, test, and release website changes...


  • Bengaluru, India Pantheon Platform Full time

    Staff Software Engineer - File System TeamLocation: Bangalore, India About PantheonPantheon is the WebOps platform for websites that deliver extraordinary results. We believe in putting the magic of the internet in everyone’s hands. That’s why we’re so passionate about helping developers, IT and marketing develop, test, and release website changes...


  • Bengaluru, India Reltio Full time

    Staff Platform Engineer - DevOps at Reltio (View all jobs) Bangalore At Reltio®, we believe data should fuel business success. Reltio’s AI-powered data unification and management capabilities—encompassing entity resolution, multi-domain master data management (MDM), and data products—transform siloed data from disparate sources into unified,...


  • Bengaluru, India Reltio Full time

    Staff Platform Engineer - DevOpsat Reltio (View all jobs) BangaloreAt Reltio®, we believe data should fuel business success. Reltio’s AI-powered data unification and management capabilities—encompassing entity resolution, multi-domain master data management (MDM), and data products—transform siloed data from disparate sources into unified, trusted,...


  • Bengaluru, India SolarWinds Full time

    Your Role:We are seeking a Senior Staff Engineer to join our Developer Experience team in the SolarWinds Observability engineering org. Our goal is to help our developers succeed by providing an exceptional developer experience. You will work alongside other engineers in an agile development environment and share solutions to advance the technologies running...


  • Bengaluru, India SolarWinds Full time

    Your Role : We are seeking a Senior Staff Engineer to join our Developer Experience team in the SolarWinds Observability engineering org. Our goal is to help our developers succeed by providing an exceptional developer experience. You will work alongside other engineers in an agile development environment and share solutions to advance the technologies...


  • Bengaluru, India Cloudera Full time

    Job Description:At Cloudera, we empower people to transform complex data into clear and actionable insights. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.Join our Cloudera’s Machine Learning Platform team as a Staff Backend Software Engineer. You'll...


  • Bengaluru, India NCR Corporation Full time

    About NCR VOYIX NCR VOYIX Corporation (NYSE: VYX) is a leading global provider of digital commerce solutions for the retail, restaurant and banking industries. NCR VOYIX is headquartered in Atlanta, Georgia, with approximately 16,000 employees in 35 countries across the globe. For nearly 140 years, we have been the global leader in consumer transaction...


  • Bengaluru, India Cloudera Full time

    Job Description :  At Cloudera, we empower people to transform complex data into clear and actionable insights. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.   Join our Cloudera’s Machine Learning Platform team as a Staff Backend Software Engineer....


  • Bengaluru, India Calix Full time

    We, the Cloud Platform Engineering team at Calix are responsible for the Platforms, Tools, and CI/CD pipelines at Calix. Our mission is to enable Calix engineers to accelerate the delivery of world-class products while ensuring the high availability, reliability, security, and performance expected by our customers.We are seeking Cloud Engineers with proven...


  • Bengaluru, India Harness Full time

    Harness is a high-growth company that is disrupting the software delivery market. Our mission is to enable the 30 million software developers in the world to deliver code to their users reliably, efficiently, securely and quickly, increasing customers’ pace of innovation while improving the developer experience. We offer solutions for every step of the...


  • Bengaluru, India Harness Full time

    Harness is a high-growth company that is disrupting the software delivery market. Our mission is to enable the 30 million software developers in the world to deliver code to their users reliably, efficiently, securely and quickly, increasing customers’ pace of innovation while improving the developer experience. We offer solutions for every step of the...


  • Bengaluru, India RapidBraiins Full time

    Experience Level:10-15 yearsNotice period: 0-30 DaysLocation: BengaluruPosition Type: Full-TimeJob DescriptionAs a Senior Cloud Security Engineer, you will play a crucial role in ensuring the security and compliance of our Google Cloud Platform (GCP) environments. You will be responsible for designing and implementing robust security solutions, conducting...


  • Bengaluru, India Harness Full time

    Harness is a high-growth company that is disrupting the software delivery market. Our mission is to enable the 30 million software developers in the world to deliver code to their users reliably, efficiently, securely and quickly, increasing customers’ pace of innovation while improving the developer experience. We offer solutions for every step of the...


  • Bengaluru, India Harness Full time

    Harness is a high-growth company that is disrupting the software delivery market. Our mission is to enable the 30 million software developers in the world to deliver code to their users reliably, efficiently, securely and quickly, increasing customers’ pace of innovation while improving the developer experience. We offer solutions for every step of the...


  • Bengaluru, India Harness Full time

    Harness is a high-growth company that is disrupting the software delivery market. Our mission is to enable the 30 million software developers in the world to deliver code to their users reliably, efficiently, securely and quickly, increasing customers’ pace of innovation while improving the developer experience. We offer solutions for every step of the...


  • Bengaluru, India NCR Corporation Full time

    About NCRVOYIXNCR VOYIX Corporation (NYSE: VYX) is a leading global provider of digital commerce solutions for the retail, restaurant and banking industries. NCR VOYIX is headquartered in Atlanta, Georgia, with approximately 16,000 employees in 35 countries across the globe. For nearly 140 years, we have been the global leader in consumer transaction...


  • Bengaluru, India Greenlight Full time

    Technology need:Backend: Kotlin, TypeScript/Cloud experience (AWS preferred)Kubernetes, Ambassador, HelmGithub Actions (or other CI experience i.e CircleCI, Jenkins, TravisCI)Continuous Delivery (ArgoCD or equivalent)Library development experienceWhat you will be doing:Technical Excellence: Take charge of Greenlight’s Developer Platform, enable high...


  • Bengaluru, India Squareroot Consulting Pvt Ltd Full time

    Job Description : - We are looking for a motivated senior technical leader who wants to have a big impact at a fast-growing Platform company.- You'll work on our cloud data platform and the connectors that bring in data from many sources we support.- We're looking for someone who can understand the key values that make our product great and...