Kubernetes Spark Administrator
2 weeks ago
Location :
Mumbai
Job Description:
Spark Cluster Deployment:
Deploy, configure, and maintain Apache Spark clusters on Kubernetes, ensuring scalability, reliability, and performance.
Application Deployment:
Collaborate with data engineers and data scientists to deploy Spark applications and workloads, ensuring they run efficiently.
Monitoring and Optimization:
Implement monitoring solutions to track cluster performance, resource utilization, and application health. Proactively identify and resolve performance bottlenecks.
Resource Management:
Manage cluster resources, including CPU, memory, and storage allocation, to ensure optimal utilization and cost efficiency.
Security:
Implement and maintain security measures, including authentication, authorization, and encryption, to protect sensitive data and Spark clusters.
Backup and Recovery:
Develop and maintain backup and recovery strategies to ensure data integrity and availability in case of failures.
Documentation:
Maintain clear and comprehensive documentation of Spark cluster configurations, deployment procedures, and best practices.
Troubleshooting:
Quickly diagnose and resolve issues related to Spark clusters, applications, and Kubernetes infrastructure.
Collaboration:
Work closely with cross-functional teams, including data engineers, data scientists, and DevOps, to understand application requirements and optimize Spark clusters accordingly.
Qualifications:
Proven experience deploying and managing Apache Spark on Kubernetes in a production environment.
Proficiency in containerization technologies, particularly Docker and Kubernetes.
Strong knowledge of Spark architecture, including cluster, driver, and worker nodes.
Familiarity with Spark tuning, optimization, and performance monitoring.
Experience with resource management tools like Kubernetes Resource Quotas and LimitRanges.
Understanding of data processing and analytics workflows.
Excellent problem-solving and troubleshooting skills.
Strong communication and collaboration skills.
Preferred
Qualifications:
Experience with Spark cluster orchestration tools like Helm.
Knowledge of Spark ecosystem components such as Spark SQL, Spark Streaming, and MLlib.
Familiarity with cloud-based solutions (e.g., AWS, Azure, Google Cloud).
Certification in Kubernetes (e.g., Certified Kubernetes Administrator CKA).
Knowledge of CI/CD pipelines and infrastructure as code (IaC) tools (e.g., Terraform).
Scripting skills in languages like Python, Bash, or Shell.
Understanding of DevOps practices and automation.
Kindly share your updated resume in word format.
-
Cateina Technologies
4 weeks ago
Mumbai, Maharashtra, India Cateina Technologies Full timeJob Title : Red Hat OpenShift AdministratorJob Summary : As a Red Hat OpenShift Administrator, you will be responsible for designing, deploying, and maintaining Red Hat OpenShift Container Platform environments. Your primary focus will be on ensuring the availability, scalability, and security of containerized applications while collaborating with...
-
Linux Administrator
4 weeks ago
Mumbai, Maharashtra, India Careerfit.ai Full timeResponsibilities :- Install, configure, and maintain Linux operating systems and server infrastructure to ensure high availability and performance of critical systems.- Monitor system performance, identify and troubleshoot issues, and optimize Linux servers to enhance efficiency and reliability.- Manage user accounts, permissions, and access controls on...
-
Senior Data Engineer
1 week ago
Mumbai, Maharashtra, India Sampoorna Consultants Pvt. Ltd Full timeKey Responsibilities : - Design, build, and maintain scalable and efficient data pipelines.- Implement the complete ELT process to enhance our data platform's performance and reliability.- Collaborate with cross-functional teams to gather requirements and deliver high-quality data solutions.- Optimize data flow and collection to improve data accuracy and...
-
Lead Data Engineer
4 weeks ago
Mumbai, Maharashtra, India Sampoorna Consultants Pvt. Ltd Full timeClient is looking out for a technical delivery leader on platform and project builds for building and maintaining optimised and highly available data pipelines that facilitate data management and data analytics solutions.The role will drive the development of data processing frameworks and architectural approaches that scale to handle the business's...
-
Mumbai, Maharashtra, India SAMTA INFOTECH PRIVATE LIMITED Full timeJob Description :We are seeking a DevOps Engineer with an experience of 2 to 4 years, to join our iNDX team. The individual will be responsible for processes related to application programming, automation, and deployments.This regular, full-time position will be office-based at our office in Powai.Responsibilities :- Strong background in Linux/Unix...
-
Nielsen - Platform DevOps Engineer
2 weeks ago
Mumbai, Maharashtra, India timesjobs Full timeJob Title: Platform DevOps EngineerLocation: Mumbai, IndiaEmployment Type: Full TimeWork Arrangement: HybridAt Nielsen, we believe that career growth is a partnership. You ultimately own, fuel and set the journey. By joining our team of nearly 14,000 associates, you will become part of a community that will help you to succeed. We champion you because when...
-
DevOps Engineer
4 weeks ago
Mumbai, Maharashtra, India Armour Corporate Services Full timeAssignment Brief:Job Title : DevOps / Onsite EngineerLocation : Mumbai Reporting to : Enterprise ArchitectNature of Work : Monday to Friday, regular work from office Job Overview:The successful candidate will be responsible for designing, implementing, and maintaining the infrastructure and software systems required to support our development and production...
-
Senior Data Mesh Engineer
4 weeks ago
Mumbai, Maharashtra, India Olive Trees Consulting Full timeJob Description :- Our IT client based in Malad, Mumbai is looking for a Data Mesh Engineer. The main purpose of the role is to design, build, and maintain data pipelines and infrastructure using Microsoft Fabric components. You'll play a crucial role in enabling data mesh architecture within the organization, ensuring scalability, reliability, and...
-
Intern
3 weeks ago
Mumbai, Maharashtra, India Cornerstone OnDemand Full timeIn this role you will be...Perform daily system monitoring, verifying the integrity and availability of all hardware, server resources, systems and key processes, reviewing system and application logs, and verifying completion of scheduled jobs such as backups.Research and recommend innovative and automated approaches for system administration tasks.Identify...
-
Software Engineer
2 weeks ago
Mumbai, Maharashtra, India Nielsen Full timeAt Nielsen, we believe that career growth is a partnership. You ultimately own, fuel and set the journey. By joining our team of nearly 14,000 associates, you will become part of a community that will help you to succeed. We champion you because when you succeed, we do too. Embark on a new initiative, explore a fresh approach, and take license to think big,...
-
Ergode - DevOps Engineer - IT Automation Tools
3 weeks ago
Mumbai, Maharashtra, India Ergode IT Services pvt. ltd. Full timeKey Responsibilities :- Collaborate with development, operations, and other cross-functional teams to understand infrastructure requirements and implement solutions to meet business needs- Assist in the design, implementation, and maintenance of CI/CD pipelines for software delivery and deployment automation- Configure and manage cloud infrastructure and...
-
Senior DevSecOps Engineer
1 week ago
Mumbai, Maharashtra, India Mithra Full timeAs Senior DevSecOps Engineer, you will be responsible for DevSecOps platform and solutions for AWS and on-prem environments. You will lead design, estimate, develop and deploy cutting-edge DevSecOps solutions and services to support infrastructure/platform as a service.Your role will also be to ensure that design adequately represents and support the needs...
-
Site Reliability Engineer
4 weeks ago
Mumbai, Maharashtra, India Talent Leads HR Solutions Pvt. Ltd. Full timeJob Description :Skill, Knowledge &Trainings : - Site Reliability Engineer will be responsible to develop and implement services that improve Software development Life Cycle. - Build automations which will help optimize software delivery. - Improve reliability, quality, and time - to - market of our suite of software solutions. - Will be responsible for...
-
DevOps Engineer III
1 week ago
Bangalore/Mumbai, Maharashtra, India Xaltworks Consulting Full timeJob Requirement :What you'll do :- Bridging the gaps b/w core infra, security, QA and development team.- Owning the end-to-end Availability, Performance, Capacity of applications and their infrastructure and creating/maintaining the respective observability with Prometheus/New Relic/ELK/Loki.- Providing 24X7 infra & app support, building processes and...
-
Digital Software Architect
1 month ago
Mumbai, Maharashtra, India ABB Full timeDigital Software ArchitectTake your next career step at ABB with a global team that is energizing the transformation of society and industry to achieve a more productive, sustainable future.At ABB, we have the clear goal of driving diversity and inclusion across all dimensions: gender, LGBTQ+, abilities, ethnicity and generations. Together, we are embarking...
-
Cloud Data Architect
4 weeks ago
Mumbai, Maharashtra, India EDGESOFT Full timeJob Description : Cloud Data Architect - Data Warehousing/Data Lake for Enterprise Position Overview : We are seeking a talented and experienced Cloud Data Architect to lead the design and implementation of a robust, scalable, and efficient cloud-based data warehousing and data lake solution for our enterprise. As a Cloud Data Architect, you will play a...
-
Akasa Air
22 hours ago
Mumbai, Maharashtra, India SNV AVIATION PRIVATE LIMITED Full timeRole Description :This is a full-time on-site role for a Site Reliability Engineer at Akasa Air located in Mumbai. As a Site Reliability Engineer, you will be responsible for ensuring the reliability and performance of our systems and infrastructure. This includes troubleshooting issues, developing and maintaining software, managing system administration...
-
Mumbai, Maharashtra, India NDX P2P Private Limited Full timeJob Role - AWS DevOps EngineerJob Description :As an AWS DevOps Engineer, you will work with development teams daily to address their infra concerns and deploy appropriate AWS infra solutions. You will also continuously monitor services and perform necessary infra upgrades. You will work as an independent contributor ready to take on new challenges and...
-
Adobe Analytics Developer
4 weeks ago
Bangalore/Mumbai, Maharashtra, India Growel Softech Pvt. Ltd. Full timeJD : Roles and responsibilities :- Maintain and configure development tools such as CI/CD, environment automation, code quality, and testing tools- Design, engineer, and support Adobe Target products and necessary infrastructure.- Install/configure/maintenance of Adobe Target Infrastructure- Installation/configuration/maintenance of Apache web server and...
-
Staff/Principal Engineer
2 weeks ago
Mumbai, Maharashtra, India Capabiliq IT Services Private Limited Full timePosition Overview : We are seeking a talented and experienced Technical Architect (Staff/Principal Engineer- Application) to join our dynamic team. The ideal candidate will work closely with an Engineering Manager and be responsible for one or more software products. The Technical Architect will provide leadership, architectural guidance, and technical...