Spark Scala Developer

2 weeks ago

bangalore, India Corporate Resources Full time
Job Description
  • 4+ years of experience with Java, Spark, Bigdata & Hadoop with strong Unix background
  • Create Scala/Spark jobs for data transformation and aggregation
  • Experience of writing code in spark and Scala
  • Produce unit tests for Spark transformations and helper methods
  • Write Scaladoc-style documentation with all code
  • Design data processing pipelines
  • Scala (with a focus on the functional programming paradigm)
  • Spark query tuning and performance optimization
  • SQL database integration {{ Microsoft, Oracle, Postgres, and/or MySQL }}
  • Experience working with {{ HDFS, S3, Cassandra, and/or DynamoDB }}
  • Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)
  • Scala, Spark, Hive, Hadoop & spark Architecture and its components such as HDFS, Job Tracker, Task Tracker, executor cores and memory params.
  • Spark and Spark Streaming creating RDDs, applying operations -Transformation and Actions.
  • Code optimizes to fine tune the applications.
  • Develop Shell scripts to orchestrate execution of all other scripts and move the data files within and outside of HDFS.
  • Industry
  • IT-Software / Software Services
  • Functional Area
  • IT Software - Application Programming, Maintenance
  • Job Role
  • Software Developer
  • Job Type
  • Permanent
Qualification
  • UG Qualification
  • B.Tech/B.E - Computers
  • PG Qualification
  • Any Post Graduate - Any Specialization
  • Doctorate
  • Doctorate Not Required - None
Desired Candidate Profile Please refer to the Job description above

  • Scala/Spark Developer

    2 months ago

    Bangalore, Karnataka, India Edge In Asia Recruitment Private Limited Full time

    Our client is a global investment banking company having its headquarters in the US with 75000+ employees globally and is looking for a Scala / Spark to join their Bangalore regional team.Note : Looking for candidates who can join within 30daysWork Mode : Hybrid Experience : 5 to 8 yearsSome of the key responsibilities will include :- Understand and...

  • Scala/Spark Developer

    2 weeks ago

    Bangalore, India Edge In Asia Recruitment Private Limited Full time

    Our client is a global investment banking company having its headquarters in the US with 75000+ employees globally and is looking for a Scala / Spark to join their Bangalore regional team.Note : Looking for candidates who can join within 30daysWork Mode : Hybrid Experience : 5 to 8 yearsSome of the key responsibilities will include : - Understand and...

  • Scala/Spark Developer

    2 months ago

    Bangalore, India Edge In Asia Recruitment Private Limited Full time

    Our client is a global investment banking company having its headquarters in the US with 75000+ employees globally and is looking for a Scala / Spark to join their Bangalore regional team.Note : Looking for candidates who can join within 30daysWork Mode : Hybrid Experience : 5 to 8 yearsSome of the key responsibilities will include : - Understand and...

  • Scala/Spark Developer

    2 weeks ago

    Bangalore, Karnataka, India Edge In Asia Recruitment Private Limited Full time

    Our client is a global investment banking company having its headquarters in the US with 75000+ employees globally and is looking for a Scala / Spark to join their Bangalore regional team.Note : Looking for candidates who can join within 30daysWork Mode : Hybrid Experience : 5 to 8 yearsSome of the key responsibilities will include :- Understand and...

  • Scala Developer

    1 week ago

    Bangalore/Pune/Hyderabad, India T D Newton Full time

    Urgent requirement for Scala Developer.Experience : 4-7 YrsJob Location : Pune/Bangalore/HyderabadMust have : Scala, Spark, CI/CD, Airflow, Hadoop, SQL Job Description :As a key member of the technical team alongside Engineers, Data Scientists and Data Users, you will be expected to define and contribute at a high-level to many aspects of our collaborative...

  • Scala Developer

    2 days ago

    Bangalore/Pune/Hyderabad, IN T D Newton Full time

    Urgent requirement for Scala Developer.Experience : 4-7 YrsJob Location : Pune/Bangalore/HyderabadMust have : Scala, Spark, CI/CD, Airflow, Hadoop, SQLJob Description :As a key member of the technical team alongside Engineers, Data Scientists and Data Users, you will be expected to define and contribute at a high-level to many aspects of our collaborative...

  • Big Data Consultant

    2 months ago

    bangalore, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Big Data Consultant

    2 months ago

    Bangalore, Karnataka, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Big Data Consultant

    2 weeks ago

    Bangalore, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Big Data Consultant

    2 months ago

    Bangalore, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Big Data Consultant

    2 weeks ago

    Bangalore, Karnataka, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Data Engineer

    2 weeks ago

    bangalore, India VDart Software Services Pvt. Ltd. Full time

    Job Description: Mandatory - Hands on spark scala programming, experience in working in data lakes, data migration project, AWS cloud Strong proficiency in Scala, with a minimum of 2 years' experience in a data lake environment -- MUST Expertise in technologies such as Impala, Spark, Python, Java, and SQL. Cloud migration experience, particularly with...

  • Data Engineer

    3 weeks ago

    bangalore, India VDart Software Services Pvt. Ltd. Full time

    Job Description: Mandatory - Hands on spark scala programming, experience in working in data lakes, data migration project, AWS cloud Strong proficiency in Scala, with a minimum of 2 years' experience in a data lake environment -- MUST Expertise in technologies such as Impala, Spark, Python, Java, and SQL. Cloud migration experience, particularly with...

  • Developer - Spark/Scala

    2 months ago

    Pune/Bangalore/Hyderabad/Mumbai/Delhi NCR, India TD Newton & Associates Full time

    Job Description : Main duties and responsibilities :- Delivering on critical business priorities while evolving platform towards its strategic vision- Working closely with product managers, end users, and business analysts to understand business objectives and key success measures - Partnering with technology teams to design and architect solutions that...

  • Senior Spark Developer

    2 months ago

    Bangalore,Anywhere in India,Multiple Locations CYBERSAPIENT TECHNOLOGIES PRIVATE LIMITED Full time

    About the job :Job Description / Duties & Responsibilities :- Develop, program, and maintain applications using the Apache Spark open-source framework.- Work with different aspects of the Spark ecosystem, including Spark SQL, DataFrames, Datasets, and streaming.- Spark Developer must have strong programming skills in Java, Scala, or Python.- Familiar with...

  • Quantexa Developer

    2 months ago

    bangalore, India IBU Consulting Full time

    Roles And Responsibilities :- Quantexa ETL and Scoring Development, configuration, and enhancements.- Technical delivery of Quantexa implementation utilizing the below support tools and methods.- Agile / DevOps delivery mode.- CI/CD pipelines.- Work with Kubernetes.- Basic understanding of MS Azure and Azure DevOps is a plus .- Hands-on experience with Scala...

  • Quantexa Developer

    2 weeks ago

    Bangalore, Karnataka, India IBU Consulting Full time

    Roles And Responsibilities :- Quantexa ETL and Scoring Development, configuration, and enhancements.- Technical delivery of Quantexa implementation utilizing the below support tools and methods.- Agile / DevOps delivery mode.- CI/CD pipelines.- Work with Kubernetes.- Basic understanding of MS Azure and Azure DevOps is a plus .- Hands-on experience with Scala...

  • Quantexa Developer

    2 months ago

    Bangalore, Karnataka, India IBU Consulting Full time

    Roles And Responsibilities :- Quantexa ETL and Scoring Development, configuration, and enhancements.- Technical delivery of Quantexa implementation utilizing the below support tools and methods.- Agile / DevOps delivery mode.- CI/CD pipelines.- Work with Kubernetes.- Basic understanding of MS Azure and Azure DevOps is a plus .- Hands-on experience with Scala...

  • Quantexa Developer

    2 months ago

    Bangalore, India IBU Consulting Full time

    Roles And Responsibilities :- Quantexa ETL and Scoring Development, configuration, and enhancements.- Technical delivery of Quantexa implementation utilizing the below support tools and methods.- Agile / DevOps delivery mode.- CI/CD pipelines.- Work with Kubernetes.- Basic understanding of MS Azure and Azure DevOps is a plus .- Hands-on experience with Scala...

  • Data Engineer III

    2 months ago

    bangalore, India Freelance Full time

    Job Description : Key Responsibilities :- Develop, construct, test, and maintain data architectures (e.g., databases, large-scale processing systems)- Build high-quality data pipelines using Spark/Scala- Implement best software engineering practices, including Git version control, code reviews, and unit testing- Collaborate with data scientists and...