Spark Admin

2 weeks ago

Pune Maharashtra, India Virtusa Full time

Spark Administration:
Install, configure, and manage Apache Spark 3.x. clusters.
Monitor and optimize Spark cluster performance.
Troubleshoot and resolve Spark-related issues.
Manage Spark job scheduling and resource allocation.
Ensure high availability and reliability of Spark clusters.
DevOps Practices:
Implement and manage CI/CD pipelines using tools like Jenkins, Nexus, GitHUB CI.
Automate infrastructure provisioning and configuration management using tools like Ansible, Ansible Tower Monitor system performance and ensure system availability and reliability.
Implement and manage containerization using Docker i.e. build Jupyter notebook.
Security and Compliance:
Code scanning tools SAST, SonarCube, SonatypeIQ
Implement security best practices for Spark clusters and DevOps pipelines.
Ensure compliance with industry standards and regulations.
Collaboration and Communication:
Required Skills and Qualifications:
Bachelors degree in computer science, Information Technology, or a related field.
Proven experience as a Spark Administrator, with hands-on experience in Spark 3.x.
Strong understanding of Spark architecture and internals.
Experience with DevOps tools and practices, including CI/CD, configuration management, and containerization.
Proficiency in scripting languages such as Python, Bash, or PowerShell.
Experience with cloud platforms such as AWS, Azure, or Google Cloud.
Strong problem-solving skills and attention to detail.
Excellent communication and collaboration skills. Work closely with data engineers, other stakeholders to understand requirements and provide solutions.
Document processes, configurations, and procedures.
Preferred Skills:
Experience with big data technologies such as Hadoop, Hive, and Kafka.
Knowledge of monitoring tools like Prometheus, Grafana, Apdynomics & Splunk.
Experience with version control systems like Git.
Familiarity with Agile methodologies.

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • Hadoop Admin

    1 week ago

    Pune, Maharashtra, India crutz leela Full time

    **Hadoop Admin/Sr Hadoop Admin** - **4+ years of hands-on experience - Hadoop, System administration with sound knowledge in Unix based Operating System internals**. - Working experience on Cloudera CDP and CDH and Hortonworks HDP Distribution. - Linux experience (RedHat, CentOS, Ubuntu). - Experience in setting up and supporting Hadoop environment (Cloud...

  • Hadoop Admin

    4 days ago

    Kothrud, Pune, Maharashtra, India crutz leela Full time

    **Hadoop Admin/Sr Hadoop Admin** - **4+ years of hands-on experience - Hadoop, System administration with sound knowledge in Unix based Operating System internals**. - Working experience on Cloudera CDP and CDH and Hortonworks HDP Distribution. - Linux experience (RedHat, CentOS, Ubuntu). - Experience in setting up and supporting Hadoop environment (Cloud...

  • Admin Executive

    7 days ago

    Pune, India JLL Full time

    About JLL - We’re JLL—a leading professional services and investment management firm specializing in real estate. We have operations in over 80 countries and a workforce of over 91,000 individuals around the world who help real estate owners, occupiers and investors achieve their business ambitions. As a global Fortune 500 company, we also have an...

  • Hadoop Admin

    4 days ago

    Pune, Maharashtra, India CRUTZ LEELA ENTERPRISES Full time ₹ 2,00,000 - ₹ 10,00,000 per year

    Position - Hadoop AdminExperience: 4 to 7 YearsJob Description: -4+ years of hands-on experience - Hadoop, System administration with sound knowledgeIn Unix based Operating System internals. Working experience on Cloudera CDP and CDH and Hortonworks HDP Distribution. Linux experience (RedHat, CentOS, Ubuntu). Experience in setting up and supporting...

  • Hadoop Admin

    2 days ago

    Pune, Maharashtra, India Emergys Full time

    **Experience**: 4 - 7 years **Location**: Pune, India (Work from Office) - 4+ years of hands-on experience - Hadoop, System administration with sound knowledge in Unix based Operating System internals. - Working experience on Cloudera CDP and CDH and Hortonworks HDP Distribution. - Linux experience (RedHat, CentOS, Ubuntu). - Experience in setting up and...

  • Python

    7 days ago

    Pune, Maharashtra, India Cognizant Full time

    **Job summary** **Qualifications** - Possess strong technical skills in Python Amazon Redshift Amazon S3 PySpark Apache Airflow Databricks SQL and Spark in Scala. - Experience with Risk Management Databricks Workflows Structured Streaming Delta Live Pipelines Databricks CLI Databricks Unity Catalog Admin Databricks Delta Lake and Delta Sharing is a plus. -...

  • Hadoop Admin

    5 days ago

    Pune, Maharashtra, India HSBC Full time

    Job description Some careers shine brighter than others If you re looking for a career that will help you stand out join HSBC and fulfil your potential Whether you want a career that could take you to the top or simply take you in an exciting new direction HSBC offers opportunities support and rewards that will take you further HSBC is one of the largest...

  • Big Data admin-Hadoop, Spark, Nifi, Impala

    4 days ago

    Pune, Maharashtra, India Cortex Consultants Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Work Experience: 4 to 6 years Official Notice Period – Immediate to 15 days Excellent communication skills Job Location – PuneShift Timing: Regular shift & UK shift (Should be flexible) Job Description The RolePlan, manage, and oversee all aspects of a Production Environment for Big Data Platforms.Define strategies for Application Performance...

  • Admin Assistant

    2 weeks ago

    Pune, India JLL Full time

    JLL supports the Whole You, personally and professionally. Admin Assistant Work Dynamics **What this job involves**: Being on top of facilities and operations costs Balancing the planned budget and actual costs requires extensive financial knowledge—that’s why we’re on the lookout for a billing expert who can get the job done flawlessly. Guided by...

  • Hadoop Admin

    9 hours ago

    Kothrud, Pune, Maharashtra, India CRUTZ LEELA ENTERPRISES Full time

    Experience: 2 to 6 Years Location: Pune (work from office) Job Description: - 4+ years of hands-on experience - Hadoop, System administration with sound knowledge In Unix based Operating System internals. - Working experience on Cloudera CDP and CDH and Hortonworks HDP Distribution. - Linux experience (RedHat, CentOS, Ubuntu). - Experience in setting up...