Apache Spark Expert

7 days ago


Pune, India Info Origin Full time

Position Title:

- Apache Spark Expert**Location**:

- Pune, MH, OnsitePosition Type:

- Full-TimeInterview Process:

- Webcam/SkypeReq Id:

- 3314Salary/Rate:

- MarketPosition Duration:

- Permanent**Key Responsibilities**:

- Provide architectural consultancy to clients and develop implementation roadmaps for Apache Spark and Kafka solutions.
- Translate architectural strategies into code and facilitate knowledge transfer to junior developers.
- Demonstrate in-depth knowledge of AWS services, particularly MSK (Managed Streaming for Kafka), Spark EMR, and working with various data sources, sinks, and transformations.
- Proficiency in utilizing S3, Athena, Glue, and Step Functions for efficient data processing and analysis.
- Experience with Kinesis, Firehose, and the Kafka Connect Framework, including the development of custom connectors.

**Qualifications and Skills**:

- Strong expertise in Apache Spark and Kafka with a focus on architectural understanding.
- Proven hands-on experience in developing and implementing solutions using Apache Spark and Kafka Streaming.
- Sound knowledge of AWS services, including MSK, Spark EMR, S3, Athena, Glue, Step Functions, Kinesis, and Firehose.
- Proficiency in the Kafka Connect Framework, especially in the development of custom connectors.

**Preferred**:

- Familiarity with Apache Airflow is considered a plus.
- Experience in (Real-time) Data analytics is also welcome.



  • Pune, India Deutsche Telekom Full time

    Your Role :We are looking for a Senior Cloudera Developer (Data Engineer) with extensive experience in Spark, the Hadoop ecosystem, and GCP services to design and implement data :- Design, develop, and implement data solutions using Cloudera technologies such as Hadoop, Spark, and Hive.- Collaborate with data engineers to optimize data pipelines and data...


  • Pune, India Coditas Technologies Pvt. Ltd. Full time

    We are looking for Senior data engineers who have the right attitude, aptitude, skills, empathy, compassion, and hunger for learning.Build products in the data analytics space.Roles and Responsibilities :- Develop and manage robust ETL pipelines using Apache Spark (Scala).- Understand Spark concepts, performance optimization techniques, and governance...


  • Bengaluru, Chennai, Pune, India Photon Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    About The Role   Develop, test, and deploy data processing applications using Apache Spark and Scala.Optimize and tune Spark applications for better performance on large-scale data sets.Work with the Cloudera Hadoop ecosystem (e.g., HDFS, Hive, Impala, HBase, Kafka) to build data pipelines and storage solutions.Collaborate with data scientists, business...

  • Data Streaming

    1 week ago


    Pune, India ACE Talent Consulting Full time

    Job Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Data Streaming

    1 week ago


    Pune, India ACE Talent Consulting Full time

    ob Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Tcs data Streaming

    5 days ago


    Pashan, Pune, Maharashtra, India Response Informatics Full time

    JOB DESCRIPTION: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. SKILLS: Pyspark Scala (with a focus on the functional programming paradigm) Apache Spark 2.x, 3.x -...

  • Java Spark

    3 days ago


    Pune, India Citi Full time

    Requirements - 6+ years of progressive experience in development with Java/J2EE technologies. - Expertise in development with Apache Spark streaming and batch framework - Proficiency in programming SparkSQL, Java for Spark. - Experience with Kafka / high performance streaming frameworks - Should have experience working with Hadoop ecosystem components -...

  • Lead Data Engineer

    1 week ago


    Pune, India CliqHR Full time

    We are seeking an experienced Lead Data Engineer to design, build, and optimize large-scale data solutions. This role involves working with modern cloud and big data technologies to enable data-driven decision-making, while leading initiatives that ensure scalability, reliability, and efficiency.Key Responsibilities :- Lead the design and deployment of...

  • Spark/Scala Developer

    3 weeks ago


    Pune, India Cyanous Software Private Limited Full time

    Detailed JD (Roles and Responsibilities) :- At least 8+ years of experience and strong knowledge in Scala programming language.- Able to write clean, maintainable and efficient Scala code following best practices.- Good knowledge on the fundamental Data Structures and their usage - At least 8+ years of experience in designing and developing large scale,...

  • Spark Admin

    2 weeks ago


    Pune, Maharashtra, India Virtusa Full time

    Spark Administration: Install, configure, and manage Apache Spark 3.x. clusters. Monitor and optimize Spark cluster performance. Troubleshoot and resolve Spark-related issues. Manage Spark job scheduling and resource allocation. Ensure high availability and reliability of Spark clusters. DevOps Practices: Implement and manage CI/CD pipelines using tools like...