Data Streaming

1 month ago


Chennai, India ACE Talent Consulting Full time

ob Description:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster.

Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data.

**Responsibilities**:
Create Scala/Spark/Pyspark jobs for data transformation and aggregation

Produce unit tests for Spark transformations and helper methods

Write Scaladoc-style documentation with all code

Design data processing pipelines

**Skills**:Pyspark

Scala (with a focus on the functional programming paradigm)

Apache Spark 2.x, 3.x
- Apache Spark RDD API
- Apache Spark SQL DataFrame API
- Apache Spark Streaming API

Spark query tuning and performance optimization

SQL database integration (Postgres, and/or MySQL)

Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)

CI-CD Pipleline, Jenkins, Gitlab /Bitbucket

Deep understanding of distributed systems (e.g. partitioning, replication, consistency, and consensus)

**Salary**: ₹300,000.00 - ₹1,500,000.00 per year

Schedule:

- Day shift
- Monday to Friday

Ability to commute/relocate:

- Chennai, Tamil Nadu: Reliably commute or planning to relocate before starting work (required)

**Experience**:

- total work: 3 years (preferred)

Work Location: One location


  • Data Scientist

    1 month ago


    Chennai, India Crayon Data Full time

    Role: Data Scientist Experience level: 3+ years Location: Chennai Who are we? Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform, maya.ai, helps...

  • Data Architect

    2 months ago


    Chennai, India iXceed Solutions Full time

    **Job Title**:Data Architect **Job Location**:WFH from India **Job Mode**: Remote (need to follow Europe Time zone) Primarily experts of **Spark 3.X, SCALA,** Delta lake implementation, streaming solution implementation for IOT in Spark streaming. Kafka expertise. Any experience with MFG BI, DWH and Datalake implementation would be a bonus. **Job Type**:...

  • Power App

    18 hours ago


    Chennai, India NTT DATA Full time

    **Req ID**: 245061 We are currently seeking a Power App to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). **Educational Qualification**: B.E (Computer / IT related Stream) Exp: 3+yrs. **Mandatory Skills**: (Please try for Option 1 first, if not found go for Option 2. Option 3 we will start only after 2.) Option 1: Power Apps Developer with...

  • Senior Data Engineer

    2 months ago


    Chennai, India hubwork Full time

    Summary :We are looking for a data engineer who has experience in building data pipelines for large data platforms. This role will be working with a team building a large data platform catering to the needs of a wealth manager for their application and analytics needs. This role will be augmented with a customer of Intellect and other third parties in...

  • AWS Data Engineer

    1 month ago


    Chennai, India CliqHR Full time

    Job Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...

  • AWS Data Engineer

    1 month ago


    Chennai, India CliqHR Full time

    Job Description : - Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...

  • AWS Data Engineer

    1 month ago


    Chennai, Tamil Nadu, India CliqHR Full time

    Job Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...


  • chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Job Description - Data Engineer (Databricks)About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for...


  • Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Job Role : Sr. Data Engineer (Databricks)About Eucloid :Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some of...

  • Data Engineer

    3 days ago


    Chennai, India Hiring Infinity Full time

    Role : Data EngineerWe are looking for a curious Data Engineer to join our extremely fast growing Tech Team About the organization :Founded in 2021, leverages GenAI to enable businesses to effortlessly create and launch ads in seconds.Businesses can manage ads across channels like Facebook, Instagram, Google and WhatsApp in a single platform.Proprietary...

  • Data Engineer

    3 days ago


    Chennai, India Hiring Infinity Full time

    Role : Data EngineerWe are looking for a curious Data Engineer to join our extremely fast growing Tech Team About the organization :Founded in 2021, leverages GenAI to enable businesses to effortlessly create and launch ads in seconds.Businesses can manage ads across channels like Facebook, Instagram, Google and WhatsApp in a single platform.Proprietary...

  • Data Engineer

    3 days ago


    chennai, India Hiring Infinity Full time

    Role : Data EngineerWe are looking for a curious Data Engineer to join our extremely fast growing Tech Team About the organization :Founded in 2021, leverages GenAI to enable businesses to effortlessly create and launch ads in seconds.Businesses can manage ads across channels like Facebook, Instagram, Google and WhatsApp in a single platform.Proprietary...

  • Data Engineer

    24 minutes ago


    Chennai, Tamil Nadu, India Hiring Infinity Full time

    Role : Data EngineerWe are looking for a curious Data Engineer to join our extremely fast growing Tech Team About the organization :Founded in 2021, leverages GenAI to enable businesses to effortlessly create and launch ads in seconds.Businesses can manage ads across channels like Facebook, Instagram, Google and WhatsApp in a single platform.Proprietary...

  • Big Data Developer

    1 month ago


    Chennai, India RiskInsight Consulting Pvt Ltd Full time

    ResponsibilitiesDesign, develop, and deploy scalable Big Data applications using Apache Spark.Collaborate with data scientists and business analysts to understand requirements and translate them into technical solutions.Write efficient and optimized code to process and analyze large volumes of data.Implement data ingestion processes from various data sources...


  • chennai, India NTT DATA Full time

    Job Description Req ID:  278316  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Systems Integration Specialist Advisor to join our team in Chennai, Tamil Nādu...


  • Chennai, India NTT DATA Full time

    Job Description Req ID: 278316 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Systems Integration Specialist Advisor to join our team in Chennai, Tamil Nādu (IN-TN),...


  • Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Role : Lead Data Engineer (Databricks) About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some of...


  • chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Role : Lead Data Engineer (Databricks) About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some of...


  • Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Role : Lead Data Engineer (Databricks)About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some of...

  • Big Data Tester

    4 weeks ago


    Chennai, India Durus Consulting Private Limited Full time

    **JOB DESCRIPTION FOR BIG DATA ENGINEER** **Job Role ** : Big Data Testing **Job Type** : FTE **Location** : Hybrid **Notice Period** : Immediate to 30 days **Experience ** : 3+ YRS **Desired Skills and Experience.** - Demonstrated testing & automation experience of 3-6 Years in Big data technology stack - Hadoop, HDFS & Spark - Good understanding of...