Apache Spark Expert

6 days ago


Pune, India Info Origin Full time

Position Title:

- Apache Spark Expert**Location**:

- Pune, MH, OnsitePosition Type:

- Full-TimeInterview Process:

- Webcam/SkypeReq Id:

- 3314Salary/Rate:

- MarketPosition Duration:

- Permanent**Key Responsibilities**:

- Provide architectural consultancy to clients and develop implementation roadmaps for Apache Spark and Kafka solutions.
- Translate architectural strategies into code and facilitate knowledge transfer to junior developers.
- Demonstrate in-depth knowledge of AWS services, particularly MSK (Managed Streaming for Kafka), Spark EMR, and working with various data sources, sinks, and transformations.
- Proficiency in utilizing S3, Athena, Glue, and Step Functions for efficient data processing and analysis.
- Experience with Kinesis, Firehose, and the Kafka Connect Framework, including the development of custom connectors.

**Qualifications and Skills**:

- Strong expertise in Apache Spark and Kafka with a focus on architectural understanding.
- Proven hands-on experience in developing and implementing solutions using Apache Spark and Kafka Streaming.
- Sound knowledge of AWS services, including MSK, Spark EMR, S3, Athena, Glue, Step Functions, Kinesis, and Firehose.
- Proficiency in the Kafka Connect Framework, especially in the development of custom connectors.

**Preferred**:

- Familiarity with Apache Airflow is considered a plus.
- Experience in (Real-time) Data analytics is also welcome.


  • Apache Iceberg

    1 week ago


    Hyderabad, Pune, Bhubaneswar, India Tata Consultancy Services Limited Full time

    Job Description Must Have Skills: - Good experience in Apache Iceberg, Apache Spark, Trino - Proficiency in SQL and data modeling Nice To Have Skills: Technical/Functional Skills Key Qualifications - Experience with open Data Lakehouse using Apache Iceberg - Experience with Data Lakehouse architecture with Apache Iceberg and Trino - Experience with...


  • Pune, India People Prime Worldwide Full time

    About Client :- Our client is a French multinational information technology (IT) services and consulting company, headquartered in Paris, France. Founded in 1967, It has been a leader in business transformation for over 50 years, leveraging technology to address a wide range of business needs, from strategy and design to managing operations. The company is...

  • Hinduja Tech

    3 weeks ago


    Pune, India HINDUJA TECH LIMITED Full time

    Description :Data Engineer will be responsible for development and deployment of end-to-end data ingestion pipelines using Azure Databricks, Apache Spark, and related technologies, ensuring scalability, performance, and efficiency.Key responsibilities and tasks :- Design & Architecture : Design high-performance, resilient, and scalable data architectures for...


  • Pune, India Bohiyaanam Technologies Pvt Ltd Full time

    **Position**:Databricks Java Spark **Location: Pune/Bangalore/Mumbai** **Experience: 6 to 9 Years** **Skills required**: Big Data-Apache Spark-Java **Responsibilities**: - **Primary skillset: Databricks, Spark,**: - **Exp - 5+ yrs**: - Spark, Scala or PySpark pipelines development in multihop pipeline architecture - Strong understanding of full CI/CD...

  • Tcs data Streaming

    4 days ago


    Pashan, Pune, Maharashtra, India Response Informatics Full time

    JOB DESCRIPTION: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. SKILLS: Pyspark Scala (with a focus on the functional programming paradigm) Apache Spark 2.x, 3.x -...

  • Spark Admin

    2 weeks ago


    Pune, Maharashtra, India Virtusa Full time

    Spark Administration: Install, configure, and manage Apache Spark 3.x. clusters. Monitor and optimize Spark cluster performance. Troubleshoot and resolve Spark-related issues. Manage Spark job scheduling and resource allocation. Ensure high availability and reliability of Spark clusters. DevOps Practices: Implement and manage CI/CD pipelines using tools like...


  • Pune, Maharashtra, India Systems Plus Full time ₹ 2,40,001 - ₹ 9,00,001 per year

    Location: PuneEmployment Type: full-timeJob Detailsob Description: Apache Flink Engineer (minimum 4+ Years Experience)Position: Apache Flink EngineerExperience: 4+ years (Min. 2 years in Apache Flink)Location: RemoteType: Full-time/ContractJob RoleWe are looking for an experienced Data Engineer with strong hands-on expertise in Apache Flink, large-scale data...

  • Spark Developer

    2 weeks ago


    Bengaluru, Hyderabad, Pune, India Infosys Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking an experienced Spark Developer with 5+ years of expertise in big data processing and distributed computing. The ideal candidate will design, develop, and optimize data pipelines using Apache Spark and related technologies, ensuring high performance and scalability for large datasets.Key ResponsibilitiesDesign and implement Spark-based ETL...


  • Pune, India Ixceed Solutions Full time

    Description :Job Description :We are looking for an experienced Senior Spark Developer with a strong background in Scala, Java, and Big Data technologies. The ideal candidate will have 5+ years of development experience, including expertise in Spark-based data processing, Java server-side development, and working in fast-paced, low-latency environments. A...

  • Software Engineer

    4 weeks ago


    Pune, India Alp Consulting Ltd. Full time

    Job Description: 5.5- 7.5 Years of in Big Data & Data related technology experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop Experience with building stream-processing systems, using technologies...