Data Streaming
1 month ago
ob Description:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster.
Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data.
**Responsibilities**:
Create Scala/Spark/Pyspark jobs for data transformation and aggregation
Produce unit tests for Spark transformations and helper methods
Write Scaladoc-style documentation with all code
Design data processing pipelines
**Skills**:Pyspark
Scala (with a focus on the functional programming paradigm)
Apache Spark 2.x, 3.x
- Apache Spark RDD API
- Apache Spark SQL DataFrame API
- Apache Spark Streaming API
Spark query tuning and performance optimization
SQL database integration (Postgres, and/or MySQL)
Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)
CI-CD Pipleline, Jenkins, Gitlab /Bitbucket
Deep understanding of distributed systems (e.g. partitioning, replication, consistency, and consensus)
**Salary**: ₹300,000.00 - ₹1,500,000.00 per year
Schedule:
- Day shift
- Monday to Friday
Ability to commute/relocate:
- Chennai, Tamil Nadu: Reliably commute or planning to relocate before starting work (required)
**Experience**:
- total work: 3 years (preferred)
Work Location: One location
-
Data Scientist
1 month ago
Chennai, India Crayon Data Full timeRole: Data Scientist Experience level: 3+ years Location: Chennai Who are we? Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform, maya.ai, helps...
-
Data Architect
2 months ago
Chennai, India iXceed Solutions Full time**Job Title**:Data Architect **Job Location**:WFH from India **Job Mode**: Remote (need to follow Europe Time zone) Primarily experts of **Spark 3.X, SCALA,** Delta lake implementation, streaming solution implementation for IOT in Spark streaming. Kafka expertise. Any experience with MFG BI, DWH and Datalake implementation would be a bonus. **Job Type**:...
-
Power App
18 hours ago
Chennai, India NTT DATA Full time**Req ID**: 245061 We are currently seeking a Power App to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). **Educational Qualification**: B.E (Computer / IT related Stream) Exp: 3+yrs. **Mandatory Skills**: (Please try for Option 1 first, if not found go for Option 2. Option 3 we will start only after 2.) Option 1: Power Apps Developer with...
-
Senior Data Engineer
2 months ago
Chennai, India hubwork Full timeSummary :We are looking for a data engineer who has experience in building data pipelines for large data platforms. This role will be working with a team building a large data platform catering to the needs of a wealth manager for their application and analytics needs. This role will be augmented with a customer of Intellect and other third parties in...
-
AWS Data Engineer
1 month ago
Chennai, India CliqHR Full timeJob Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...
-
AWS Data Engineer
1 month ago
Chennai, India CliqHR Full timeJob Description : - Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...
-
AWS Data Engineer
1 month ago
Chennai, Tamil Nadu, India CliqHR Full timeJob Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...
-
Eucloid - Data Engineer - Azure Databricks
5 days ago
chennai, India Eucloid Data Solutions Pvt Ltd Full timeJob Description - Data Engineer (Databricks)About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for...
-
Chennai, India Eucloid Data Solutions Pvt Ltd Full timeJob Role : Sr. Data Engineer (Databricks)About Eucloid :Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some of...
-
Data Engineer
3 days ago
Chennai, India Hiring Infinity Full timeRole : Data EngineerWe are looking for a curious Data Engineer to join our extremely fast growing Tech Team About the organization :Founded in 2021, leverages GenAI to enable businesses to effortlessly create and launch ads in seconds.Businesses can manage ads across channels like Facebook, Instagram, Google and WhatsApp in a single platform.Proprietary...
-
Data Engineer
3 days ago
Chennai, India Hiring Infinity Full timeRole : Data EngineerWe are looking for a curious Data Engineer to join our extremely fast growing Tech Team About the organization :Founded in 2021, leverages GenAI to enable businesses to effortlessly create and launch ads in seconds.Businesses can manage ads across channels like Facebook, Instagram, Google and WhatsApp in a single platform.Proprietary...
-
Data Engineer
3 days ago
chennai, India Hiring Infinity Full timeRole : Data EngineerWe are looking for a curious Data Engineer to join our extremely fast growing Tech Team About the organization :Founded in 2021, leverages GenAI to enable businesses to effortlessly create and launch ads in seconds.Businesses can manage ads across channels like Facebook, Instagram, Google and WhatsApp in a single platform.Proprietary...
-
Data Engineer
24 minutes ago
Chennai, Tamil Nadu, India Hiring Infinity Full timeRole : Data EngineerWe are looking for a curious Data Engineer to join our extremely fast growing Tech Team About the organization :Founded in 2021, leverages GenAI to enable businesses to effortlessly create and launch ads in seconds.Businesses can manage ads across channels like Facebook, Instagram, Google and WhatsApp in a single platform.Proprietary...
-
Big Data Developer
1 month ago
Chennai, India RiskInsight Consulting Pvt Ltd Full timeResponsibilitiesDesign, develop, and deploy scalable Big Data applications using Apache Spark.Collaborate with data scientists and business analysts to understand requirements and translate them into technical solutions.Write efficient and optimized code to process and analyze large volumes of data.Implement data ingestion processes from various data sources...
-
Systems Integration Specialist Advisor
4 days ago
chennai, India NTT DATA Full timeJob Description Req ID: 278316 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Systems Integration Specialist Advisor to join our team in Chennai, Tamil Nādu...
-
Systems Integration Specialist Advisor
4 days ago
Chennai, India NTT DATA Full timeJob Description Req ID: 278316 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Systems Integration Specialist Advisor to join our team in Chennai, Tamil Nādu (IN-TN),...
-
Chennai, India Eucloid Data Solutions Pvt Ltd Full timeRole : Lead Data Engineer (Databricks) About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some of...
-
chennai, India Eucloid Data Solutions Pvt Ltd Full timeRole : Lead Data Engineer (Databricks) About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some of...
-
Chennai, India Eucloid Data Solutions Pvt Ltd Full timeRole : Lead Data Engineer (Databricks)About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some of...
-
Big Data Tester
4 weeks ago
Chennai, India Durus Consulting Private Limited Full time**JOB DESCRIPTION FOR BIG DATA ENGINEER** **Job Role ** : Big Data Testing **Job Type** : FTE **Location** : Hybrid **Notice Period** : Immediate to 30 days **Experience ** : 3+ YRS **Desired Skills and Experience.** - Demonstrated testing & automation experience of 3-6 Years in Big data technology stack - Hadoop, HDFS & Spark - Good understanding of...