Apache Spark Expert

6 days ago

Pune, India Info Origin Full time

Position Title:

- Apache Spark Expert**Location**:

- Pune, MH, OnsitePosition Type:

- Full-TimeInterview Process:

- Webcam/SkypeReq Id:

- 3314Salary/Rate:

- MarketPosition Duration:

- Permanent**Key Responsibilities**:

- Provide architectural consultancy to clients and develop implementation roadmaps for Apache Spark and Kafka solutions.
- Translate architectural strategies into code and facilitate knowledge transfer to junior developers.
- Demonstrate in-depth knowledge of AWS services, particularly MSK (Managed Streaming for Kafka), Spark EMR, and working with various data sources, sinks, and transformations.
- Proficiency in utilizing S3, Athena, Glue, and Step Functions for efficient data processing and analysis.
- Experience with Kinesis, Firehose, and the Kafka Connect Framework, including the development of custom connectors.

**Qualifications and Skills**:

- Strong expertise in Apache Spark and Kafka with a focus on architectural understanding.
- Proven hands-on experience in developing and implementing solutions using Apache Spark and Kafka Streaming.
- Sound knowledge of AWS services, including MSK, Spark EMR, S3, Athena, Glue, Step Functions, Kinesis, and Firehose.
- Proficiency in the Kafka Connect Framework, especially in the development of custom connectors.

**Preferred**:

- Familiarity with Apache Airflow is considered a plus.
- Experience in (Real-time) Data analytics is also welcome.

  • Data Engineer//Python/Azure Databrick/Apache Spark

    3 days ago

    Pune, Maharashtra, India OptimHire Full time US$ 90,000 - US$ 1,20,000 per year

    Job DescriptionData Software Engineer - Spark, Python, (AWS, Kafka or Azure Databricks or GCP)Job Description:5-12 Years of in Big Data & Data related technology experienceExpert level understanding of distributed computing principlesExpert level knowledge and experience in Apache SparkHands on programming with PythonProficiency with Hadoop v2, Map Reduce,...

  • Senior Technical Lead – Apache Spark, Scala, Python

    3 days ago

    Pune, Maharashtra, India HireWand Technologies Full time US$ 1,50,000 - US$ 2,00,000 per year

    Job Title: Senior Technical Lead Apache Spark, Scala, PythonLocation: PuneJob Type: Full TimeJob BriefWe are looking for a skilled Senior Technical Lead to join our client's team. In this role, you will guide technical teams, lead projects, and create smart and scalable solutions using Apache Spark, Scala, and Python. You will play a key role in shaping our...

  • Java Spark Developer

    3 days ago

    Pune, Maharashtra, India Wipro Full time US$ 90,000 - US$ 1,20,000 per year

    Java+SparkPrimary skill - Apache Spark Secondary skill - JavaStrong knowledge in Apache Spark framework Core Spark, Spark Data Frames, Spark streamingHands-on experience in any one of the programming languages (Java)Good understanding of distributed programming concepts.Experience in optimizing Spark DAG, and Hive queries on TezExperience using tools like...

  • Scala Spark Developer

    3 days ago

    Chennai, Pune, India Cloudxtreme Full time US$ 90,000 - US$ 1,20,000 per year

    Role : Scala Spark DeveloperLocation : Pune/ChennaiCompany : MNCRequired Skills & Qualifications:Minimum 5 years of professional experience in Data Engineering, with a strong focus on big data technologies.Proficiency in Scala for developing big data applications and transformations, especially with Apache Spark.Expert-level proficiency in SQL; ability to...

  • Spark Scala Developer

    3 days ago

    Chennai, Pune, India Cloudxtreme Full time US$ 1,20,000 - US$ 2,00,000 per year

    Mandatory - Experience and knowledge in designing, implementing, and managing non-relational data stores (e.g., MongoDB, Cassandra, DynamoDB), focusing on flexible schema design, scalability, and performance optimization for handling large volumes of unstructured or semi-structured data.Mainly client needs No SQL DB, either MongoDB or HBaseData Pipeline...

  • Java Spark Developer

    3 days ago

    Chennai, Pune, India Cloudxtreme Full time US$ 90,000 - US$ 1,20,000 per year

    Key Responsibilities:Collaborate with senior developers and architects to design and develop robust and scalable data processing applications using Apache Spark.Write efficient, testable, and maintainable code primarily in Java, transitioning to Spark as you gain expertise.Participate in code reviews, testing, and deployment of Spark...

  • Databricks, Java Spark

    1 week ago

    Pune, India Bohiyaanam Technologies Pvt Ltd Full time

    **Position**:Databricks Java Spark **Location: Pune/Bangalore/Mumbai** **Experience: 6 to 9 Years** **Skills required**: Big Data-Apache Spark-Java **Responsibilities**: - **Primary skillset: Databricks, Spark,**: - **Exp - 5+ yrs**: - Spark, Scala or PySpark pipelines development in multihop pipeline architecture - Strong understanding of full CI/CD...

  • Data Streaming

    7 days ago

    Pune, India ACE Talent Consulting Full time

    Job Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Data Streaming

    7 days ago

    Pune, India ACE Talent Consulting Full time

    ob Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Tcs data Streaming

    4 days ago

    Pashan, Pune, Maharashtra, India Response Informatics Full time

    JOB DESCRIPTION: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. SKILLS: Pyspark Scala (with a focus on the functional programming paradigm) Apache Spark 2.x, 3.x -...