IBM - Spark Data Engineer/Developer - Python/Scala

2 days ago


Bengaluru, Karnataka, India Kyndryl Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Description :

Spark Data Engineer / Developer (DevOps & OpenShift)

About the Role :

We are seeking a highly skilled and experienced Spark Data Engineer / Developer to join our dynamic team. This role is critical for building, optimizing, and supporting our cutting-edge data platform, leveraging Apache Spark, Apache Iceberg, and a robust DevOps approach within an OpenShift environment. The ideal candidate will be adept at both developing high-performance data solutions and ensuring their stability and reliability in a production setting.

Key Responsibilities :

- Design, develop, and optimize scalable and resilient data processing applications using Apache Spark (batch, streaming, and real-time).

- Implement and manage data pipelines, ensuring data quality, consistency, and performance.

- Perform Spark job performance tuning and optimization to handle large-scale datasets efficiently.

- Manage and automate the deployment of Spark applications within OpenShift clusters, utilizing Docker and Kubernetes.

- Establish and maintain CI/CD pipelines for automated testing, deployment, and release management of Spark workloads.

Required Skills & Experience :

years of hands-on experience in data engineering or software development roles.

- Expert-level proficiency in Apache Spark (Spark Core, Spark SQL, Spark Streaming).

- Strong programming skills in Scala, Python (PySpark), or Java.

- Significant experience with OpenShift, including deploying, managing, and automating containerized applications within the platform.

- Solid understanding of Docker and Kubernetes for containerization and orchestration.

- Proven experience implementing and maintaining CI/CD pipelines using tools like Jenkins,


  • spark scala

    3 weeks ago


    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Hi,Greetings from TCS Team!Role Spark Scala DeveloperRequired Technical Skill Set Spark Scala - Coding knowledge expectedDesired Experience Range 6-9 YearsLocation of Requirement TCS - Chennai, Bangalore, Kochi, Pune, BhubaneswarIf you’re interested in the role if the experience, location and skillset match your profile then please share your updated...


  • Bengaluru, Karnataka, India IQVIA Full time

    Senior Spark + Scala Developer Proficiency in both Scala and Python Profile:- Scala with Functional Programming, python [ Java is second option] Exp.:- 4+ Years of Experience Scala [ 2+ years experienced in data engineering, python, scala] Skills :- Mandatory:- Scala, Play Framework, Slick, PostgreSQL, python, Spark (Basic), Akka (Basic),...


  • Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Job Role : Spark, Scala Developer Experience : 5+ yearsLocation : Bengaluru/ HyderabadMust Have :Must have 5+ years of IT experienceMust have good experience in Spark and ScalaGood to have experience instreaming systems like Spark streaming and StormExpereicne with Spark Data processing, Performance Tuning, Memory Management, Fault Tolerance, ScalabilityGood...


  • Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    TCS is Hiring For Spark & Scala DeveloperJob Location : BangaloreExperience : 6 - 12 yearsMust have skills :Must have 6+ years of IT experienceMust have good experience in Spark and ScalaGood to have experience instreaming systems like Spark streaming and StormExperience with Spark Data processing, Performance Tuning, Memory Management, Fault Tolerance,...

  • Scala Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Avisoft Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    We are looking for a highly skilled Scala Data Engineer to design, build, and optimize large-scale data platforms. The ideal candidate will have deep expertise in Scala, Spark, and SQL, with proven experience delivering scalable and high-performance data solutions in cloud-native environments.Key ResponsibilitiesDesign, build, and optimize scalable data...


  • Bengaluru, Karnataka, India The It Mind Services Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Data Engineer - SPARKApache Spark (Scala), Hadoop, and Hive, performance tuning, debugging, and optimization , Functional Programming, OOP principles, SOLID design, and design patterns, RDBMS, NoSQL database (preferably PostgreSQL), build tools (Maven), version control systems (Git), and CI/CD pipelines (Jenkins preferred), unit and integration tests using...

  • Scala Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Xebia It Architects Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Role: Scala Data EngineerLocation: Bangalore (Hybrid, 3 days office/week)Experience: 4 6 yearsEmployment Type: Full-TimeKey Responsibilities:Develop, maintain, and optimize data pipelines using Scala & SparkWork with HDFS, Hive, Yarn to manage large datasets efficientlyBuild and manage workflows with OozieApply unit testing practices to ensure code quality...

  • Data Engineer

    1 week ago


    Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...


  • Bengaluru, Karnataka, India POWER IT SERVICES Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    Big Data Engineer – Spark & AnalyticsLocation- ( Bangalore, Maharashtra)Job Description•Gather operational intel on business processes and policies from multiple sources•Prepare periodical and ad-hoc reports using operational data•Develop semantic core to align data with business processes•Support operations team's work streams for data processing,...

  • Data Engineer

    5 days ago


    Bengaluru, Karnataka, India CODERS BRAIN Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    About the RoleWe are seeking a highly skilled Data Engineer to join our team, working on end-to-end data engineering and data science use cases. The ideal candidate should have strong expertise in Python or Scala, Spark (Databricks), and SQL, with proven experience in building scalable and efficient data pipelines on Azure.ResponsibilitiesDesign, build, and...