Senior Spark Developer

5 days ago

Pune, India Consulting Pandits Full time

Spark Rockstar Wanted

Imagine a world where big data is not just a buzzword, but a playground where you get to build innovative solutions that make a real impact. We're on the hunt for a Senior Big Data Engineer who's passionate about Spark and has the skills to prove it. If you're a Spark aficionado with a knack for designing and developing scalable data pipelines, we want to hear from you

Responsibilities :

- Lead the charge in building and maintaining our Spark-based data ecosystem, with a focus on batch and streaming processing

- Collaborate with our team of data wizards to design and implement ETL workflows that migrate seamlessly to Spark

- Whip up Python, Scala, and Java code like a pro, and use your expertise to integrate Spark with Hadoop, HDFS, YARN, and Kafka

- Streamline our workflows using Airflow or other workflow engines, and make Jupyter notebooks do your bidding

- Bonus points if you can conjure up workflows with NiFi, Flink, and Kudu streaming

Qualifications :

- 5+ years of experience in Spark development, with a strong portfolio of projects that showcase your expertise

- Hands-on experience with Spark, Hadoop, HDFS, YARN, Kafka, and Airflow (or other workflow engines)

- Programming prowess in Python, Scala, and Java

- Experience with ETL implementation and migration to Spark

- Good to have : Flink and Kudu streaming, automation of workflows CI/CD, Nifi streaming and transformation, and Informatica workflow migration


(ref:hirist.tech)

  • Java Spark Developer

    3 days ago

    Pune, India Wipro Full time

    Java+Spark Primary skill - Apache Spark Secondary skill - Java Strong knowledge in Apache Spark framework Core Spark, Spark Data Frames, Spark streaming Hands-on experience in any one of the programming languages (Java) Good understanding of distributed programming concepts. Experience in optimizing Spark DAG, and Hive queries on Tez Experience using tools...

  • Python and Spark developer

    4 days ago

    Pune, Maharashtra, India Orcapod Consulting Services Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Duration of contract*12 monthsTotal Yrs. of Experience*3+Relevant Yrs. of experience*3+Detailed JD *(Roles and Responsibilities)Develop and maintain legacy application and new platform(Roleswap, Migration, Patching, Mandatory upgrades)Mandatory skills*Python,Spark,Spark SQL,GIT,PostgreSQL,Airflow,Jenkins,KubernetesDesired skills*Camunda knowledge of Camunda...

  • Python and Spark developer

    3 days ago

    Pune, India Orcapod Consulting Services Full time

    Duration of contract* 12 months Total Yrs. of Experience* 3+ Relevant Yrs. of experience* 3+ Detailed JD *(Roles and Responsibilities) Develop and maintain legacy application and new platform (Roleswap, Migration, Patching, Mandatory upgrades) Mandatory skills* Python,Spark,Spark SQL,GIT,PostgreSQL,Airflow,Jenkins,Kubernetes Desired skills* Camunda knowledge...

  • Java/scala +spark Developer-3

    1 week ago

    Pune, India Citi Full time

    **Responsibilities**: - Bachelor’s degree (in Computer Science, Information Technology). - Must have good programming skills and should be aware of OOPS concept, design principles, data structures. - Good experience in Big Data technologies (A MUST) - HDFS. - Good knowledge of databases: Hive, Hbase. - Good knowledge in Apache Spark, Spark-SQL, UDF,...

  • Java Spark

    2 weeks ago

    Pune, India Citi Full time

    Requirements - 6+ years of progressive experience in development with Java/J2EE technologies. - Expertise in development with Apache Spark streaming and batch framework - Proficiency in programming SparkSQL, Java for Spark. - Experience with Kafka / high performance streaming frameworks - Should have experience working with Hadoop ecosystem components -...

  • Spark SQL Developer

    7 days ago

    Pune, Maharashtra, India Tech Mahindra Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    Roles and ResponsibilitiesCollaborate with cross-functional teams to gather requirements and design ETL processes for extracting, transforming, and loading (ETL) of large datasets from various sources into the target database.Develop complex SQL queries to optimize performance of existing databases and troubleshoot issues related to query execution...

  • Spark SQL Developer

    3 days ago

    Pune, India Tech Mahindra Full time

    Roles and Responsibilities Collaborate with cross-functional teams to gather requirements and design ETL processes for extracting, transforming, and loading (ETL) of large datasets from various sources into the target database. Develop complex SQL queries to optimize performance of existing databases and troubleshoot issues related to query execution times....

  • Senior Big Data/Spark Engineer

    2 weeks ago

    Pune, Maharashtra, India Barclays Full time US$ 1,00,000 - US$ 1,50,000 per year

    Join us as a Senior Big Data/Spark Engineer at Barclays, where you will be responsible for supporting the successful delivery of location strategy projects to plan, budget, agreed quality and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to...

  • Senior Big Data/Spark Engineer

    3 days ago

    Pune, India Barclays Full time

    Join us as a Senior Big Data/Spark Engineer at Barclays, where you will be responsible for supporting the successful delivery of location strategy projects to plan, budget, agreed quality and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to...

  • Senior Data Engineer

    5 days ago

    Pune, India Right Move Staffing Solutions Private Limited Full time

    Job Summary :We are looking for a highly skilled and experienced Senior Data Engineer to join our data team.The ideal candidate will have over 4 years of experience in building and maintaining robust data pipelines and infrastructure.This role requires expertise in cloud data services, big data processing frameworks, and a strong understanding of data...