Spark / Data Pipeline Engineer

Data Engineer - SPARKApache Spark (Scala), Hadoop, and Hive, performance tuning, debugging, and optimization , Functional Programming, OOP principles, SOLID design, and design patterns, RDBMS, NoSQL database (preferably PostgreSQL), build tools (Maven), version control systems (Git), and CI/CD pipelines (Jenkins preferred), unit and integration tests using...

Big Data Orchestration: Airflow, Spark on Kubernetes, Yarn, Oozie Big DataProcessing: Hadoop, Kafka, Spark & Spark Structured Streaming. Experienceon SOLID & DRY principles with Good Software Architecture & Designimplementation experience Advanced Scala experience (e.g. FunctionalProgramming, using Case classes, Complex Data Structures &...

Job description:Your Experience5 years of experience in large scale realtime stream processing using Apache Spark with messaging infrastructure like KafkaPulsar7 years of experience on Data Pipeline development ETL and processing of structured and unstructured data5 years of experience using NoSQL systems like MongoDB DynamoDB and SnowFlake AthenaExperience...

About the RoleWe are seeking a highly skilled Data Engineer to join our team, working on end-to-end data engineering and data science use cases. The ideal candidate should have strong expertise in Python or Scala, Spark (Databricks), and SQL, with proven experience in building scalable and efficient data pipelines on Azure.ResponsibilitiesDesign, build, and...

Job Description:Scala, Spark, Apache Sqoop, Cloudera, Unix Shell scripting, CAAS.Experience: 4-6 yearsAt least four years of data engineering experience working with sophisticated data pipelines using Spark , Scala , ApacheSqoop, Cloudera , Unix Shell scripting and CAAS.Scala, Spark, Apache Sqoop, Cloudera, Unix Shell scripting, CAAS.Experience: 4-6 yearsAt...

Industry & SectorA technology services & software exports company operating in the Enterprise Data Engineering and Big Data Analytics sector. We build scalable data platforms, real-time streaming pipelines, and ETL systems that power analytics and data products for global enterprise customers.Primary Title (best-performing standardized title):Senior Data...

Below is the complete job descriptionData EngineerBangalore, IndiaHybridMust Have skills :Spark, Spark SQL, working experience with spark. Very Strong SQL, Batch Exp is a must have. Good Project Experience.Python is a plus not a must have.Data Engineer Job Description:Hands-on experience with Apache Spark (especially Spark SQL and batch processing).Strong...

As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...

Take a moment to read the full description before you apply — it'll help ensure it's the right fit for youRole:Sr. Software EngineerLocation: Bengaluru, India (Hybrid) - Must be based out of BengaluruDomain: Retail/eCommerceNote:We are looking for someone who canjoin within 15–30 days.Must Have:Proficient in Java, Spring Boot, RESTful APIs, and Data...

Responsibilities:* Design, develop, and maintain data pipelines using Spark, Python, Scala, and Java.* Write efficient and optimized SQL queries for data extraction, transformation, and loading (ETL) processes.* Work with DataFrames to manipulate and analyze large datasets.* Implement data storage and processing solutions using cloud technologies (preferably...