Senior Spark Developer

4 weeks ago


Pune, India Consulting Pandits Full time

Spark Rockstar Wanted

Imagine a world where big data is not just a buzzword, but a playground where you get to build innovative solutions that make a real impact. We're on the hunt for a Senior Big Data Engineer who's passionate about Spark and has the skills to prove it. If you're a Spark aficionado with a knack for designing and developing scalable data pipelines, we want to hear from you

Responsibilities :

- Lead the charge in building and maintaining our Spark-based data ecosystem, with a focus on batch and streaming processing

- Collaborate with our team of data wizards to design and implement ETL workflows that migrate seamlessly to Spark

- Whip up Python, Scala, and Java code like a pro, and use your expertise to integrate Spark with Hadoop, HDFS, YARN, and Kafka

- Streamline our workflows using Airflow or other workflow engines, and make Jupyter notebooks do your bidding

- Bonus points if you can conjure up workflows with NiFi, Flink, and Kudu streaming

Qualifications :

- 5+ years of experience in Spark development, with a strong portfolio of projects that showcase your expertise

- Hands-on experience with Spark, Hadoop, HDFS, YARN, Kafka, and Airflow (or other workflow engines)

- Programming prowess in Python, Scala, and Java

- Experience with ETL implementation and migration to Spark

- Good to have : Flink and Kudu streaming, automation of workflows CI/CD, Nifi streaming and transformation, and Informatica workflow migration


(ref:hirist.tech)

  • Pune, Maharashtra, India Consulting Pandits Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Spark Rockstar Wanted Imagine a world where big data is not just a buzzword, but a playground where you get to build innovative solutions that make a real impact. We're on the hunt for a Senior Big Data Engineer who's passionate about Spark and has the skills to prove it. If you're a Spark aficionado with a knack for designing and developing scalable data...

  • Spark Developer

    1 week ago


    Pune, India Infosys Limited Full time

    Job Description Job Description: - Spark Expertise - Expert proficiency in Spark - Ability to design and implement efficient data processing workflows - Experience with Spark SQL and DataFrames - Good exposure to Big Data architectures and good understanding of Big Data eco system - Experience with some framework building experience on Hadoop - Good with DB...

  • Spark Developer

    5 days ago


    Pune, Maharashtra, India Infosys Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Spark DeveloperSpark, Scala, Bigdata • Ability to develop value-creating strategies and models that enable clients to innovate, drive growth and increase their business profitability • Good knowledge on software configuration management systems • Awareness of latest technologies and Industry trends • Logical thinking and problem-solving skills along...


  • Pune, India Cyanous Software Private Limited Full time

    Detailed JD (Roles and Responsibilities) :- At least 8+ years of experience and strong knowledge in Scala programming language.- Able to write clean, maintainable and efficient Scala code following best practices.- Good knowledge on the fundamental Data Structures and their usage - At least 8+ years of experience in designing and developing large scale,...

  • Java Spark Developer

    4 weeks ago


    Pune, India Wipro Full time

    Java+Spark Primary skill - Apache Spark Secondary skill - Java Strong knowledge in Apache Spark framework Core Spark, Spark Data Frames, Spark streaming Hands-on experience in any one of the programming languages (Java) Good understanding of distributed programming concepts. Experience in optimizing Spark DAG, and Hive queries on Tez Experience using tools...


  • Pune, India Orcapod Consulting Services Full time

    Duration of contract* 12 months Total Yrs. of Experience* 3+ Relevant Yrs. of experience* 3+ Detailed JD *(Roles and Responsibilities) Develop and maintain legacy application and new platform (Roleswap, Migration, Patching, Mandatory upgrades) Mandatory skills* Python,Spark,Spark SQL,GIT,PostgreSQL,Airflow,Jenkins,Kubernetes Desired skills* Camunda knowledge...


  • Pune, Maharashtra, India Orcapod Consulting Services Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Duration of contract*12 monthsTotal Yrs. of Experience*3+Relevant Yrs. of experience*3+Detailed JD *(Roles and Responsibilities)Develop and maintain legacy application and new platform(Roleswap, Migration, Patching, Mandatory upgrades)Mandatory skills*Python,Spark,Spark SQL,GIT,PostgreSQL,Airflow,Jenkins,KubernetesDesired skills*Camunda knowledge of Camunda...


  • Bengaluru, Chennai, Pune, India Photon Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    About The Role   Develop, test, and deploy data processing applications using Apache Spark and Scala.Optimize and tune Spark applications for better performance on large-scale data sets.Work with the Cloudera Hadoop ecosystem (e.g., HDFS, Hive, Impala, HBase, Kafka) to build data pipelines and storage solutions.Collaborate with data scientists, business...


  • Pune, Maharashtra, India Barclays Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Join us as a Senior Spark Data Engineer at Barclays, where you'll take part in the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionize our digital offerings, ensuring unparalleled customer experiences. As a part of the team, you will deliver technology stack, using strong analytical...


  • Pune, India Citi Full time

    **Responsibilities**: - Bachelor’s degree (in Computer Science, Information Technology). - Must have good programming skills and should be aware of OOPS concept, design principles, data structures. - Good experience in Big Data technologies (A MUST) - HDFS. - Good knowledge of databases: Hive, Hbase. - Good knowledge in Apache Spark, Spark-SQL, UDF, Sqoop....