
Senior Spark Developer
4 weeks ago
Spark Rockstar Wanted
Imagine a world where big data is not just a buzzword, but a playground where you get to build innovative solutions that make a real impact. We're on the hunt for a Senior Big Data Engineer who's passionate about Spark and has the skills to prove it. If you're a Spark aficionado with a knack for designing and developing scalable data pipelines, we want to hear from you
Responsibilities :
- Lead the charge in building and maintaining our Spark-based data ecosystem, with a focus on batch and streaming processing
- Collaborate with our team of data wizards to design and implement ETL workflows that migrate seamlessly to Spark
- Whip up Python, Scala, and Java code like a pro, and use your expertise to integrate Spark with Hadoop, HDFS, YARN, and Kafka
- Streamline our workflows using Airflow or other workflow engines, and make Jupyter notebooks do your bidding
- Bonus points if you can conjure up workflows with NiFi, Flink, and Kudu streaming
Qualifications :
- 5+ years of experience in Spark development, with a strong portfolio of projects that showcase your expertise
- Hands-on experience with Spark, Hadoop, HDFS, YARN, Kafka, and Airflow (or other workflow engines)
- Programming prowess in Python, Scala, and Java
- Experience with ETL implementation and migration to Spark
- Good to have : Flink and Kudu streaming, automation of workflows CI/CD, Nifi streaming and transformation, and Informatica workflow migration
(ref:hirist.tech)
-
Senior Spark Developer
1 week ago
Pune, Maharashtra, India Consulting Pandits Full time ₹ 20,00,000 - ₹ 25,00,000 per yearSpark Rockstar Wanted Imagine a world where big data is not just a buzzword, but a playground where you get to build innovative solutions that make a real impact. We're on the hunt for a Senior Big Data Engineer who's passionate about Spark and has the skills to prove it. If you're a Spark aficionado with a knack for designing and developing scalable data...
-
Spark Developer
1 week ago
Pune, India Infosys Limited Full timeJob Description Job Description: - Spark Expertise - Expert proficiency in Spark - Ability to design and implement efficient data processing workflows - Experience with Spark SQL and DataFrames - Good exposure to Big Data architectures and good understanding of Big Data eco system - Experience with some framework building experience on Hadoop - Good with DB...
-
Spark Developer
5 days ago
Pune, Maharashtra, India Infosys Full time ₹ 20,00,000 - ₹ 25,00,000 per yearSpark DeveloperSpark, Scala, Bigdata • Ability to develop value-creating strategies and models that enable clients to innovate, drive growth and increase their business profitability • Good knowledge on software configuration management systems • Awareness of latest technologies and Industry trends • Logical thinking and problem-solving skills along...
-
Spark/Scala Developer
3 days ago
Pune, India Cyanous Software Private Limited Full timeDetailed JD (Roles and Responsibilities) :- At least 8+ years of experience and strong knowledge in Scala programming language.- Able to write clean, maintainable and efficient Scala code following best practices.- Good knowledge on the fundamental Data Structures and their usage - At least 8+ years of experience in designing and developing large scale,...
-
Java Spark Developer
4 weeks ago
Pune, India Wipro Full timeJava+Spark Primary skill - Apache Spark Secondary skill - Java Strong knowledge in Apache Spark framework Core Spark, Spark Data Frames, Spark streaming Hands-on experience in any one of the programming languages (Java) Good understanding of distributed programming concepts. Experience in optimizing Spark DAG, and Hive queries on Tez Experience using tools...
-
Python and Spark developer
4 weeks ago
Pune, India Orcapod Consulting Services Full timeDuration of contract* 12 months Total Yrs. of Experience* 3+ Relevant Yrs. of experience* 3+ Detailed JD *(Roles and Responsibilities) Develop and maintain legacy application and new platform (Roleswap, Migration, Patching, Mandatory upgrades) Mandatory skills* Python,Spark,Spark SQL,GIT,PostgreSQL,Airflow,Jenkins,Kubernetes Desired skills* Camunda knowledge...
-
Python and Spark developer
7 days ago
Pune, Maharashtra, India Orcapod Consulting Services Full time ₹ 9,00,000 - ₹ 12,00,000 per yearDuration of contract*12 monthsTotal Yrs. of Experience*3+Relevant Yrs. of experience*3+Detailed JD *(Roles and Responsibilities)Develop and maintain legacy application and new platform(Roleswap, Migration, Patching, Mandatory upgrades)Mandatory skills*Python,Spark,Spark SQL,GIT,PostgreSQL,Airflow,Jenkins,KubernetesDesired skills*Camunda knowledge of Camunda...
-
Scala/Spark Developer
3 days ago
Bengaluru, Chennai, Pune, India Photon Full time ₹ 10,00,000 - ₹ 25,00,000 per yearAbout The Role Develop, test, and deploy data processing applications using Apache Spark and Scala.Optimize and tune Spark applications for better performance on large-scale data sets.Work with the Cloudera Hadoop ecosystem (e.g., HDFS, Hive, Impala, HBase, Kafka) to build data pipelines and storage solutions.Collaborate with data scientists, business...
-
Senior Spark Data Engineer
3 days ago
Pune, Maharashtra, India Barclays Full time ₹ 8,00,000 - ₹ 24,00,000 per yearJoin us as a Senior Spark Data Engineer at Barclays, where you'll take part in the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionize our digital offerings, ensuring unparalleled customer experiences. As a part of the team, you will deliver technology stack, using strong analytical...
-
Java/scala +spark Developer-3
6 days ago
Pune, India Citi Full time**Responsibilities**: - Bachelor’s degree (in Computer Science, Information Technology). - Must have good programming skills and should be aware of OOPS concept, design principles, data structures. - Good experience in Big Data technologies (A MUST) - HDFS. - Good knowledge of databases: Hive, Hbase. - Good knowledge in Apache Spark, Spark-SQL, UDF, Sqoop....