
Data Streaming
2 days ago
Job Description:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster.
Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data.
**Responsibilities**:
Create Scala/Spark/Pyspark jobs for data transformation and aggregation
Produce unit tests for Spark transformations and helper methods
Write Scaladoc-style documentation with all code
Design data processing pipelines
**Skills**:Pyspark
Scala (with a focus on the functional programming paradigm)
Apache Spark 2.x, 3.x
- Apache Spark RDD API
- Apache Spark SQL DataFrame API
- Apache Spark Streaming API
Spark query tuning and performance optimization
SQL database integration (Postgres, and/or MySQL)
Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)
CI-CD Pipleline, Jenkins, Gitlab /Bitbucket
Deep understanding of distributed systems (e.g. partitioning, replication, consistency, and consensus)
**Salary**: ₹300,000.00 - ₹1,500,000.00 per year
Schedule:
- Day shift
Ability to commute/relocate:
- Bengaluru, Karnataka: Reliably commute or planning to relocate before starting work (required)
**Experience**:
- total work: 3 years (preferred)
Work Location: One location
-
Data Streaming Engineer
1 week ago
Bengaluru, Karnataka, India Innova ESI Full time ₹ 15,00,000 - ₹ 25,00,000 per yearRole: Data Streaming EngineerExperience: 6+yearsLocation: Pan IndiaImmediate joiners onlyWe are looking for a Data Streaming Engineer to design, develop, and test real-time data pipelines leveraging Solace, Kafka, Databricks Delta Live Tables (DLT), and Apache Spark Structured Streaming.The role involves building ingestion frameworks, streaming data readers,...
-
Senior Streaming Analytics Engineer
15 hours ago
Bengaluru, India BairesDev Full timeJob Description At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this...
-
Kafka Developer
2 weeks ago
Bengaluru, India People Prime World Wide Full timeKey Responsibilities :- Design, develop, and maintain scalable and reliable Apache Kafka-based data streaming platforms.- Utilize Kafka Connect, Kafka Streams, and Kafka client libraries to build real-time data pipelines and event-driven applications.- Independently resolve issues related to infrastructure deployment and application setup on cloud services...
-
Data Engineer
2 weeks ago
Bengaluru, India NTT DATA, Inc. Full timeReq ID: We are currently seeking a Data Engineer (Talend &Pyspark) to join our team in Bangalore, Karntaka (IN-KA), India (IN). "Job Duties: Key Responsibilities: Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack...
-
Urgent Search: Staff Streaming Analytics Engineer
15 hours ago
Bengaluru, India BairesDev Full timeJob Description At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this...
-
Bengaluru, India NTT DATA, Inc. Full time6+ years of experience in Data Engineering or related role. Hands-on experience with Snowflake (data modelling, performance tuning, query optimization, Snowpipe, Time Travel, Streams & Tasks). Strong expertise in AWS Glue for ETL job development, orchestration, and optimization. Proficiency with AWS services such as S3, Lambda, Redshift, Athena, Step...
-
Data Engineer
2 weeks ago
Bengaluru, India NTT Data Full timeJob Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN). - 6+ years of experience in Data...
-
Offshore Data Engineer
17 hours ago
Bengaluru, India NTT DATA Full timeJob Description Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Offshore Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Key...
-
Offshore Data Engineer
12 hours ago
Bengaluru, India NTT DATA Full timeJob Description Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Offshore Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Key...
-
Data Streaming
2 days ago
Bengaluru, India ACE Talent Consulting Full timeob Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...