Data Streaming

12 hours ago

Kochi, India ACE Talent Consulting Full time

ob Description:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster.

Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data.

**Responsibilities**:
Create Scala/Spark/Pyspark jobs for data transformation and aggregation

Produce unit tests for Spark transformations and helper methods

Write Scaladoc-style documentation with all code

Design data processing pipelines

**Skills**:Pyspark

Scala (with a focus on the functional programming paradigm)

Apache Spark 2.x, 3.x
- Apache Spark RDD API
- Apache Spark SQL DataFrame API
- Apache Spark Streaming API

Spark query tuning and performance optimization

SQL database integration (Postgres, and/or MySQL)

Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)

CI-CD Pipleline, Jenkins, Gitlab /Bitbucket

Deep understanding of distributed systems (e.g. partitioning, replication, consistency, and consensus)

**Salary**: ₹300,000.00 - ₹1,500,000.00 per year

Schedule:

- Day shift
- Monday to Friday

Ability to commute/relocate:

- Kochi, Kerala: Reliably commute or planning to relocate before starting work (required)

**Experience**:

- total work: 3 years (preferred)

Work Location: One location

  • Java Developer

    2 weeks ago

    Kochi, India Nazztec Private Limited Full time

    Job Title : Java Developer - Streaming & Data IntegrationWork Locations (Hybrid - 3 Days from Office) Description :We are seeking a proactive and technically strong Java Developer with experience in real-time data streaming and integration. This role will focus on developing enterprise-grade applications using Java, Spring Boot, and Apache Flink, with data...

  • Data Streaming

    12 hours ago

    Kochi, India ACE Talent Consulting Full time

    Job Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Data Engineer

    1 week ago

    Kochi, India Focaloid Technologies Full time

    Strong proficiency in SQL; experience with Snowflake is a strong plus.Proficiency in Python for data processing, automation, and ETL workflows.Ability to design, build, and optimize complex data pipelines and procedures for data flow.Experience with data modeling, schema design, and query optimization.System UnderstandingKnowledge of data warehousing...

  • Data Engineer

    1 week ago

    Kochi, India Focaloid Technologies Full time

    - Strong proficiency in SQL; experience with Snowflake is a strong plus.- Proficiency in Python for data processing, automation, and ETL workflows.- Ability to design, build, and optimize complex data pipelines and procedures for data flow.- Experience with data modeling, schema design, and query optimization.- System Understanding- Knowledge of data...

  • Data Engineer

    1 week ago

    Kochi, India Focaloid Technologies Full time

    Strong proficiency in SQL; experience with Snowflake is a strong plus.Proficiency in Python for data processing, automation, and ETL workflows.Ability to design, build, and optimize complex data pipelines and procedures for data flow.Experience with data modeling, schema design, and query optimization.System UnderstandingKnowledge of data warehousing...

  • Data Engineer

    1 week ago

    Kochi, India Focaloid Technologies Full time

    Strong proficiency in SQL; experience with Snowflake is a strong plus.Proficiency in Python for data processing, automation, and ETL workflows.Ability to design, build, and optimize complex data pipelines and procedures for data flow.Experience with data modeling, schema design, and query optimization.System UnderstandingKnowledge of data warehousing...

  • Data Engineer

    2 weeks ago

    Kochi, India Focaloid Technologies Full time

    Strong proficiency in SQL; experience with Snowflake is a strong plus.Proficiency in Python for data processing, automation, and ETL workflows.Ability to design, build, and optimize complex data pipelines and procedures for data flow.Experience with data modeling, schema design, and query optimization.System UnderstandingKnowledge of data warehousing...

  • Data Engineer

    1 week ago

    kochi, India Focaloid Technologies Full time

    Strong proficiency in SQL; experience with Snowflake is a strong plus.Proficiency in Python for data processing, automation, and ETL workflows.Ability to design, build, and optimize complex data pipelines and procedures for data flow.Experience with data modeling, schema design, and query optimization.System UnderstandingKnowledge of data warehousing...

  • NOMD Technologies

    6 days ago

    Kochi, India NOMD TECHNOLOGIES Full time

    AI Data Engineer-4-5 yrs experience-Immediate Joiners. The AI Data Engineer designs, develops, and maintains the data pipelines and infrastructure essential for AI and machine learning projects. The role bridges traditional data engineering with the specific requirements of AIensuring models are trained on high-quality, well-prepared data and that data flows...

  • Data Engineer

    1 week ago

    Kochi, India B2N Management Consulting Pvt. Ltd. Full time

    Overview : We are seeking a highly skilled and hands-on Data Lake Implementation Specialist with expertise in building data lake solutions on Oracle Autonomous Database. The ideal candidate will have deep technical knowledge of Oracle Cloud Infrastructure (OCI) Data Integration, Oracle Data Integrator (ODI), Oracle Data Transform, ETL job scheduling, and...