See more Collapse

Senior Spark Developer

2 months ago

BangaloreAnywhere in IndiaMultiple Locations CYBERSAPIENT TECHNOLOGIES PRIVATE LIMITED Full time

About the job :

Job Description / Duties & Responsibilities :

- Develop, program, and maintain applications using the Apache Spark open-source framework.

- Work with different aspects of the Spark ecosystem, including Spark SQL, DataFrames, Datasets, and streaming.

- Spark Developer must have strong programming skills in Java, Scala, or Python.

- Familiar with big data processing tools and techniques.

- Adhere to the ISMS policies and procedures.

Job Specification/Skills and Competencies :

- Proven experience as a Spark Developer or a related role.

- Strong programming skills in Scala/PySpark/Python.

- Familiarity with big data processing tools and techniques.

- Understanding in :

1. Apache Spark 2. x, Spark 3. x

2. Apache Spark RDD API

3. Apache Spark SQL Data Frame API

4. Apache Spark Streaming API

5. Spark query tuning and performance optimization

6. Database integration - Cassandra, Elasticsearch

- Experience working with HDFS.

- Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus).

- Experience with the Hadoop ecosystem.

- Good understanding of distributed systems.

- Experience with streaming data platforms.

- Excellent analytical and problem-solving skills.

(ref:hirist.tech)

We have other current jobs related to this field that you can find below

  • Senior Python Developer

    3 weeks ago

    Anywhere in India/Multiple Locations Varite Full time

    Salary : 12-20 LPA Responsibilities : Develop pipeline objects using PySpark, Flink, Apache Spark, and Scala. Design and implement real-time data streams utilizing Flink and PySpark. Utilize AWS services such as Glue, EMR, and VPC for infrastructure management and deployment. Design and develop data pipeline architectures using Confluent Kafka, Spark,...

  • Senior Python Developer

    2 months ago

    Anywhere in India/Multiple Locations Varite Full time

    Salary : 12-20 LPA Responsibilities : Develop pipeline objects using PySpark, Flink, Apache Spark, and Scala. Design and implement real-time data streams utilizing Flink and PySpark. Utilize AWS services such as Glue, EMR, and VPC for infrastructure management and deployment. Design and develop data pipeline architectures using Confluent Kafka, Spark,...

  • Application Developer

    2 months ago

    Anywhere in India/Multiple Locations, IN Optivalue tek consulting Full time

    Summary : As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements using Python and Apache Spark. Your typical day will involve working with PySpark, performing data analysis and munging to extract insights and patterns from large data sets.Roles &...

  • Application Developer

    2 months ago

    Anywhere in India,Multiple Locations Optivalue tek consulting Full time

    Summary : As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements using Python and Apache Spark. Your typical day will involve working with PySpark, performing data analysis and munging to extract insights and patterns from large data sets.Roles &...

  • Application Developer

    3 weeks ago

    Anywhere in India/Multiple Locations Optivalue tek consulting Full time

    Summary : As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements using Python and Apache Spark. Your typical day will involve working with PySpark, performing data analysis and munging to extract insights and patterns from large data sets.Roles &...

  • Application Developer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Optivalue tek consulting Full time

    Summary : As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements using Python and Apache Spark. Your typical day will involve working with PySpark, performing data analysis and munging to extract insights and patterns from large data sets.Roles &...

  • Big Data Developer

    3 weeks ago

    Anywhere in India/Multiple Locations Risk Resources LLP Full time

    Responsibilities :- Lead and mentor a team of Big Data engineers in designing, developing, and deploying big data solutions on GCP- Define the technical vision and roadmap for big data architecture within the organization- Architect and implement scalable, high-performance big data pipelines using technologies like MapReduce, Hive, Spark, and BigQuery-...

  • Big Data Developer

    2 months ago

    Anywhere in India,Multiple Locations Risk Resources LLP Full time

    Responsibilities :- Lead and mentor a team of Big Data engineers in designing, developing, and deploying big data solutions on GCP- Define the technical vision and roadmap for big data architecture within the organization- Architect and implement scalable, high-performance big data pipelines using technologies like MapReduce, Hive, Spark, and BigQuery-...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Paralleldots Full time

    Job Description :As a Senior Data Engineer at ParallelDots, you will be responsible for overseeing the design and development of advanced data solutions. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure meets the highest standards of scalability, reliability, and :- Lead the design,...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations Paralleldots Full time

    Job Description : As a Senior Data Engineer at ParallelDots, you will be responsible for overseeing the design and development of advanced data solutions. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure meets the highest standards of scalability, reliability, and : - Lead the design,...

  • Senior Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations BLossomJob Full time

    Senior Data Engineers (Scala, Spark/PySpark, SQL) x2 headcounts- Primarily experts of Spark 3.X, SCALA, Delta lake implementation, streaming solution implementationfor IOT in Spark streaming. Kafka expertise.Any experience with MFG BI, DWH and Datalake implementation would be a bonus. Location : WFH from IndiaDomain : Semicon- 1x new headcount in 7-10 yrs...

  • Senior Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN BLossomJob Full time

    Senior Data Engineers (Scala, Spark/PySpark, SQL) x2 headcounts- Primarily experts of Spark 3.X, SCALA, Delta lake implementation, streaming solution implementationfor IOT in Spark streaming. Kafka expertise.Any experience with MFG BI, DWH and Datalake implementation would be a bonus. Location : WFH from IndiaDomain : Semicon- 1x new headcount in 7-10 yrs...

  • Senior Data Engineer- Spark/Python

    2 months ago

    Anywhere in India/Multiple Locations/Bangalore/Chennai, IN iMind Your Business Solutions Private Limited Full time

    We are hiring senior data engineers for the leaders in analytics and cloud applications, headquartered in San Francisco and with offices in India (Bangalore and Chennai).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud applications, big data,...

  • Senior Data Engineer- Spark/Python

    2 months ago

    Anywhere in India,Multiple Locations,Bangalore,Chennai iMind Your Business Solutions Private Limited Full time

    We are hiring senior data engineers for the leaders in analytics and cloud applications, headquartered in San Francisco and with offices in India (Bangalore and Chennai).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud applications, big data,...

  • Senior Data Engineer- Spark/Python

    3 weeks ago

    Anywhere in India/Multiple Locations/Bangalore/Chennai iMind Your Business Solutions Private Limited Full time

    We are hiring senior data engineers for the leaders in analytics and cloud applications, headquartered in San Francisco and with offices in India (Bangalore and Chennai).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud applications, big data,...

  • Senior Data Engineer- Spark/Python

    3 weeks ago

    Anywhere in India/Multiple Locations/Bangalore/Chennai, IN iMind Your Business Solutions Private Limited Full time

    We are hiring senior data engineers for the leaders in analytics and cloud applications, headquartered in San Francisco and with offices in India (Bangalore and Chennai).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud applications, big data,...

  • Data Engineer

    3 weeks ago

    Metros/Anywhere in India/Multiple Locations WEN Full time

    Role : Data Engineer - Spark StreamingExperience : 5+ yearsResponsibilities :- Develop and maintain real-time data pipelines using Spark Streaming.- Collaborate with cross-functional teams to understand and define data requirements for streaming applications.- Design and implement scalable and efficient data processing solutions using Python and Pyspark.-...

  • Data Engineer

    3 weeks ago

    Metros/Anywhere in India/Multiple Locations, IN WEN Full time

    Role : Data Engineer - Spark StreamingExperience : 5+ yearsResponsibilities :- Develop and maintain real-time data pipelines using Spark Streaming.- Collaborate with cross-functional teams to understand and define data requirements for streaming applications.- Design and implement scalable and efficient data processing solutions using Python and Pyspark.-...

  • Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations Shankheshwar Enterprises Full time

    Our client is a leading player in the Digital Transformation space have 17 centers in USA and India Job Description : - Analyze and organize raw data, developing and maintaining data systems and pipelines, and collaborating closely with data scientists and business analysts to ensure data quality and reliability. - Quickly grasp and translate business...

  • Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Shankheshwar Enterprises Full time

    Our client is a leading player in the Digital Transformation space have 17 centers in USA and IndiaJob Description :- Analyze and organize raw data, developing and maintaining data systems and pipelines, and collaborating closely with data scientists and business analysts to ensure data quality and reliability.- Quickly grasp and translate business...