Staff Cloud Platform Engineer

2 weeks ago

bangalore, India Calix Full time
This position is based in Bangalore, India.
We, the Cloud Platform Engineering team at Calix are responsible for the Platforms, Tools, and CI/CD pipelines at Calix. Our mission is to enable Calix engineers to accelerate the delivery of world-class products while ensuring the high availability, reliability, security, and performance expected by our customers.
We are seeking Platform Engineers with proven expertise in the design and building of large-scale Cloud Infrastructure Platforms for Datalake to join our team and help us deliver the foundation on which Calix products can thrive. You will help us achieve our mission through innovative solutions.

Responsibilities:

  • Design, build, and maintain a scalable and reliable cloud-native data lake infrastructure using Presto, Alluxio, S3, Hive and Spark technologies.
  • Implement integration with Zookeeper for coordination and leader election across Datalake components.
  • Configure Presto to access and query data stored in Alluxio and S3, optimizing performance and resource utilization.
  • Deploy and manage Spark clusters for data processing and analytics tasks, leveraging Alluxio and S3 as storage layers.
  • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and implement solutions that meet business objectives.
  • Develop and maintain automated workflows for data ingestion, processing, and transformation using Spark and other relevant tools.
  • Implement monitoring and alerting solutions to ensure the health and performance of the data lake infrastructure.
  • Provide technical guidance and support to junior team members, fostering a culture of continuous learning and improvement.

Requirements:

  • Bachelor's degree in Computer Science, Engineering, or related field; or equivalent work experience.
  • 10+ years of experience as a Data Engineer or similar role, with expertise in building and managing cloud-native data lake infrastructure.
  • Strong proficiency in Presto, Alluxio, S3, Spark, Hive and Zookeeper, with hands-on experience in designing and implementing solutions using these technologies.
  • Experience with Spark Administration, Hive administration, Presto administration and AWS resource management.
  • Experience with monitoring of distributed systems.
  • Experience with infrastructure as code tools such as Terraform/Ansible for provisioning and managing cloud resources.
  • Experience with container orchestration platforms such as Kubernetes for deploying and managing Datalake components.
  • Solid understanding of distributed computing principles, data modelling, and database technologies.
  • Experience with serverless architecture would be plus like Lambda and EMR
  • Experience in Observability and Monitoring in AWS.
  • Fast learner and able to troubleshoot complex scenarios while process large volumes of data (Terabytes and Petabytes)
  • Excellent problem-solving skills, with the ability to troubleshoot complex issues and implement effective solutions.
  • Strong communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.
  • Having Databricks experience will be an added advantage.

Optional Skills:

  • JVM tuning. 
  • Alluxio cluster management. 

  • Staff Software Engineer

    5 days ago

    bangalore, India Pantheon Platform Full time

    Staff Software Engineer - File System TeamLocation: Bangalore, IndiaAbout PantheonPantheon is the WebOps platform for websites that deliver extraordinary results. We believe in putting the magic of the internet in everyone’s hands. That’s why we’re so passionate about helping developers, IT and marketing develop, test, and release website changes...

  • Staff Software Engineer

    5 days ago

    bangalore, India Pantheon Platform Full time

    Staff Software Engineer - File System Team Location: Bangalore, India About Pantheon Pantheon is the WebOps platform for websites that deliver extraordinary results. We believe in putting the magic of the internet in everyone’s hands. That’s why we’re so passionate about helping developers, IT and marketing develop, test, and release website...

  • Staff Software Engineer

    5 days ago

    bangalore, India Pantheon Platform Full time

    Staff Software Engineer - File System Team Location: Bangalore, India About Pantheon Pantheon is the WebOps platform for websites that deliver extraordinary results. We believe in putting the magic of the internet in everyone’s hands. That’s why we’re so passionate about helping developers, IT and marketing develop, test, and release website...

  • Staff Platform Engineer

    5 days ago

    bangalore, India Reltio Full time

    Staff Platform Engineer - DevOps at Reltio (View all jobs) Bangalore At Reltio®, we believe data should fuel business success. Reltio’s AI-powered data unification and management capabilities—encompassing entity resolution, multi-domain master data management (MDM), and data products—transform siloed data from disparate sources into...

  • Senior Staff Platform Engineer

    2 months ago

    bangalore, India SolarWinds Full time

    Your Role : We are seeking a Senior Staff Engineer to join our Developer Experience team in the SolarWinds Observability engineering org. Our goal is to help our developers succeed by providing an exceptional developer experience. You will work alongside other engineers in an agile development environment and share solutions to advance the technologies...

  • Cloud Platform Engineer

    1 week ago

    bangalore, India NCR Corporation Full time

    About NCR VOYIX NCR VOYIX Corporation (NYSE: VYX) is a leading global provider of digital commerce solutions for the retail, restaurant and banking industries. NCR VOYIX is headquartered in Atlanta, Georgia, with approximately 16,000 employees in 35 countries across the globe. For nearly 140 years, we have been the global leader in consumer...

  • Staff Backend Software Engineer, Enterprise AI Platform

    2 months ago

    bangalore, India Cloudera Full time

    Job Description :  At Cloudera, we empower people to transform complex data into clear and actionable insights. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.  Join our Cloudera’s Machine Learning Platform team as a Staff Backend Software Engineer....

  • Senior Cloud Platform Engineer

    2 weeks ago

    bangalore, India Calix Full time

    We, the Cloud Platform Engineering team at Calix are responsible for the Platforms, Tools, and CI/CD pipelines at Calix. Our mission is to enable Calix engineers to accelerate the delivery of world-class products while ensuring the high availability, reliability, security, and performance expected by our customers.We are seeking Cloud Engineers with proven...

  • Cloud Security Engineer

    4 weeks ago

    Bangalore, India RapidBraiins Full time

    Experience Level:10-15 yearsNotice period: 0-30 DaysLocation: BengaluruPosition Type: Full-TimeJob DescriptionAs a Senior Cloud Security Engineer, you will play a crucial role in ensuring the security and compliance of our Google Cloud Platform (GCP) environments. You will be responsible for designing and implementing robust security solutions, conducting...

  • Cloud Security Engineer

    4 weeks ago

    bangalore, India RapidBraiins Full time

    Experience Level:10-15 yearsNotice period: 0-30 DaysLocation: BengaluruPosition Type: Full-TimeJob DescriptionAs a Senior Cloud Security Engineer, you will play a crucial role in ensuring the security and compliance of our Google Cloud Platform (GCP) environments. You will be responsible for designing and implementing robust security solutions, conducting...

  • Cloud Security Engineer

    1 month ago

    Bangalore, Karnataka, India RapidBraiins Full time

    Experience Level:10-15 yearsNotice period: 0-30 DaysLocation: BengaluruPosition Type: Full-TimeJob DescriptionAs a Senior Cloud Security Engineer, you will play a crucial role in ensuring the security and compliance of our Google Cloud Platform (GCP) environments. You will be responsible for designing and implementing robust security solutions, conducting...

  • Senior Engineering Manager

    2 weeks ago

    bangalore, India Postman Full time

    Who Are We? Postman is the world’s leading API platform, used by more than 25 million developers and 500,000 organizations, including 98% of the Fortune 500. Postman is helping developers and professionals across the globe build the API-first world by simplifying each step of the API lifecycle and streamlining collaboration—enabling users to create...

  • Senior/Staff Software Engineer

    2 months ago

    bangalore, India Squareroot Consulting Pvt Ltd Full time

    Job Description : - We are looking for a motivated senior technical leader who wants to have a big impact at a fast-growing Platform company. - You'll work on our cloud data platform and the connectors that bring in data from many sources we support. - We're looking for someone who can understand the key values that make our product great and implement...

  • Senior/Staff Software Engineer

    4 weeks ago

    Bangalore, India Squareroot Consulting Pvt Ltd Full time

    Job Description : - We are looking for a motivated senior technical leader who wants to have a big impact at a fast-growing Platform company. - You'll work on our cloud data platform and the connectors that bring in data from many sources we support. - We're looking for someone who can understand the key values that make our product great and...

  • Senior/Staff Software Engineer

    1 month ago

    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    Job Description : - We are looking for a motivated senior technical leader who wants to have a big impact at a fast-growing Platform company.- You'll work on our cloud data platform and the connectors that bring in data from many sources we support.- We're looking for someone who can understand the key values that make our product great and implement...

  • DevOps Engineer

    2 days ago

    bangalore, India Whizz HR Full time

    As a GCP (Google Cloud Platform) DevOps Engineer, you will be responsible for designing, implementing, and maintaining our cloud infrastructure on Google Cloud Platform. You will work closely with development teams, system administrators, and other stakeholders to ensure the reliability, scalability, and security of our cloud-based systems.Key...

  • DevOps Engineer

    2 days ago

    Bangalore, India Whizz HR Full time

    As a GCP (Google Cloud Platform) DevOps Engineer, you will be responsible for designing, implementing, and maintaining our cloud infrastructure on Google Cloud Platform. You will work closely with development teams, system administrators, and other stakeholders to ensure the reliability, scalability, and security of our cloud-based systems.Key...

  • DevOps Engineer

    3 hours ago

    Bangalore, Karnataka, India Whizz HR Full time

    As a GCP (Google Cloud Platform) DevOps Engineer, you will be responsible for designing, implementing, and maintaining our cloud infrastructure on Google Cloud Platform. You will work closely with development teams, system administrators, and other stakeholders to ensure the reliability, scalability, and security of our cloud-based systems.Key...

  • Cloud Platform Engineer

    4 weeks ago

    Bangalore, India Zyoin group Full time

    Job Description :We are seeking a talented and experienced Cloud Platform Engineer to join our innovative team.- The ideal candidate will have a proven track record in cloud technologies and a strong background in designing, implementing, and maintaining cloud-based solutions.- As a Cloud Platform Engineer, you will play a key role in shaping and optimizing...

  • Staff Engineer, DevX platform

    1 day ago

    bangalore, India Greenlight Full time

    Technology need: Backend: Kotlin, TypeScript/ Cloud experience (AWS preferred) Kubernetes, Ambassador, Helm Github Actions (or other CI experience i.e CircleCI, Jenkins, TravisCI) Continuous Delivery (ArgoCD or equivalent) Library development experience What you will be doing: Technical Excellence: Take charge of Greenlight’s Developer Platform,...