Apache Spark Admin

2 weeks ago

Madhapur Hyderabad Telangana, India Glansa Solutions Full time

An Apache Spark administrator's responsibilities include:

- **Designing and implementing Spark jobs**: Defining, scheduling, monitoring, and controlling processes
- **Optimizing Spark jobs**: Maximizing speed and scalability while remaining data-use compliant
- **Managing data pipelines**: Managing data pipelines and acquisition processes
- **Performing data processing and analysis**: Performing data processing and analysis
- **Building machine learning models**: Building machine learning models using Spark or MapReduce to visualize and present results
- **Developing REST APIs**: Developing REST APIs for Spark jobs
- **Backing up and restoring**: Choosing the correct method to back up and restore IBM Software Hub

Apache Spark is an open-source engine for in-memory processing of big data at large-scale.

**Job Types**: Full-time, Permanent

Pay: ₹286,026.51 - ₹1,455,599.75 per year

Schedule:

- Day shift

**Experience**:

- total work: 3 years (preferred)

Work Location: In person

  • Data Engineer

    4 weeks ago

    Hyderabad, India BOHIYAANAM TECHNOLOGY LLP Full time

    Description :Key Responsibilities :- Design, develop, and deploy end-to-end data engineering solutions using Databricks, Apache Spark, PySpark, Python, and SQL.- Build scalable and efficient ETL/ELT pipelines for data ingestion, transformation, and integration from various sources.- Work with data warehousing solutions and ensure high-performance data...

  • Softility - Big Data Engineer - Apache Spark

    7 days ago

    Hyderabad, India Softility Full time

    Big Data Engineer :- Design, build, and optimize robust and scalable data pipelines for both batch and real-time data ingestion using Apache Spark (Streaming + Batch), Apache Nifi, and Apache Kafka.Data Storage and Management :- Manage and maintain data storage solutions on Hadoop/HDFS. Implement data models and schemas in Hive for a data warehouse and...

  • Spark, Scala

    6 days ago

    Hyderabad, Telangana, India Cognizant Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Key skills: Spark, ScalaExperience: 6 to 12 yearsLocation: AIA HyderabadJob descriptionDevelop, test, and deploy data processing applications using Apache Spark and Scala.Optimize and tune Spark applications for better performance on large-scale data sets.Work with the Cloudera Hadoop ecosystem (e.g., HDFS, Hive, Impala, HBase, Kafka) to build data...

  • Spark, Java

    6 days ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Skills-Java, Spark, SQLExperience: 6 to 12 yearsLocation: AIA-HyderabadResponsibilities and JD in brief along with additional criteria to be considered (if any):At least 6+ years of experience and strong knowledge in Java and related frameworks.Expertise in building robust, scalable and maintainable applications with Java and Spark.Able to write clean,...

  • Spark Developer

    5 days ago

    Bengaluru, Chennai, Hyderabad, India ti Steps Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    About the Role:We are looking for a skilled and passionate Spark Developer to design, build, and optimize large-scale data processing systems. The ideal candidate will have strong expertise in Apache Spark, Big Data ecosystems, and distributed computing frameworks.Key Responsibilities:Design, develop, and maintain data processing pipelines using Apache Spark...

  • Apache Flink Engineer(Data Engineer)

    5 days ago

    Chennai, Hyderabad, India People Impact Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role & responsibilitiesRoles - Apache Flink EngineerPrimary Skills - Apache Flink, Apache Kafka & Spark (Data Engineer background profiles only)Experience - 5 to 12 yearsShift - 12PM to 9PM ISTNotice Period - Immediate to 30 days onlyLocations - Chennai & Hyderbad only

  • Java developer with Spark

    4 weeks ago

    Hyderabad, India VHL Technologies Inc Full time

    Looking for immediate joiners, 30 days or less 3-5 years experience Key Requirements : Experience with the following technologies: Java, REST APIs, and Apache Spark Practical expertise in building and managing streaming applications Proven track record in developing microservices and integrating with batch data sources Strong hands-on programming skills in...

  • Java developer with Spark

    4 weeks ago

    Hyderabad, India VHL Technologies Inc Full time

    Looking for immediate joiners, 30 days or less 3-5 years experience Key Requirements: Experience with the following technologies: Java, REST APIs, and Apache Spark Practical expertise in building and managing streaming applications Proven track record in developing microservices and integrating with batch data sources Strong hands-on programming skills in...

  • Grid Dynamics

    4 weeks ago

    Hyderabad, India GRID DYNAMICS PRIVATE LIMITED Full time

    Description : Big Data Functions : - Design and implement data pipelines for migration from HDFS/Hive to cloud object storage (e.g., S3, Ceph). - Optimize Spark (and optionally Flink) jobs for performance and scalability in a Kubernetes environment.- Ensure data consistency, schema evolution, and governance with Apache Iceberg or equivalent table formats. -...

  • Java developer with Spark

    4 weeks ago

    Hyderabad, India VHL Technologies Inc Full time

    Looking for immediate joiners, 30 days or less3-5 years experienceKey Requirements:Experience with the following technologies: Java, REST APIs, and Apache SparkPractical expertise in building and managing streaming applicationsProven track record in developing microservices and integrating with batch data sourcesStrong hands-on programming skills in Java