Spark Scala Developer

1 month ago


pune, India Corporate Resources Full time
Job Description
  • 4+ years of experience with Java, Spark, Bigdata & Hadoop with strong Unix background
  • Create Scala/Spark jobs for data transformation and aggregation
  • Experience of writing code in spark and Scala
  • Produce unit tests for Spark transformations and helper methods
  • Write Scaladoc-style documentation with all code
  • Design data processing pipelines
  • Scala (with a focus on the functional programming paradigm)
  • Spark query tuning and performance optimization
  • SQL database integration {{ Microsoft, Oracle, Postgres, and/or MySQL }}
  • Experience working with {{ HDFS, S3, Cassandra, and/or DynamoDB }}
  • Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)
  • Scala, Spark, Hive, Hadoop & spark Architecture and its components such as HDFS, Job Tracker, Task Tracker, executor cores and memory params.
  • Spark and Spark Streaming creating RDDs, applying operations -Transformation and Actions.
  • Code optimizes to fine tune the applications.
  • Develop Shell scripts to orchestrate execution of all other scripts and move the data files within and outside of HDFS.
  • Industry
  • IT-Software / Software Services
  • Functional Area
  • IT Software - Application Programming, Maintenance
  • Job Role
  • Software Developer
  • Job Type
  • Permanent
Qualification
  • UG Qualification
  • B.Tech/B.E - Computers
  • PG Qualification
  • Any Post Graduate - Any Specialization
  • Doctorate
  • Doctorate Not Required - None
Desired Candidate Profile Please refer to the Job description above

  • Pune, India Radioactive Technologies Full time

    Spark & Scala DeveloperExp- 7-13 yrsShift- Malaysian / EURFull time Contract OpportunityFully Remoteplease share resume at Responsibilities-Create Scala/Spark jobs for data transformation and aggregationProduce unit tests for Spark transformations and helper methodsWrite Scaladoc-style documentation with all codeDesign data processing pipelinesSkills-Scala...


  • Pune, India Radioactive Technologies Full time

    Spark & Scala DeveloperExp- 7-13 yrsShift- Malaysian / EURFull time Contract OpportunityFully Remoteplease share resume at Khalil@weradioactive.comResponsibilities-Create Scala/Spark jobs for data transformation and aggregationProduce unit tests for Spark transformations and helper methodsWrite Scaladoc-style documentation with all codeDesign data processing...


  • Pune, India Radioactive Technologies Full time

    Spark & Scala Developer Exp- 7-13 yrs Shift- Malaysian / EUR Full time Contract Opportunity Fully Remote please share resume at Responsibilities- Create Scala/Spark jobs for data transformation and aggregation Produce unit tests for Spark transformations and helper methods Write Scaladoc-style documentation with all code Design data processing...


  • Pune, India Radioactive Technologies Full time

    Spark & Scala DeveloperExp- 7-13 yrsShift- Malaysian / EURFull time Contract OpportunityFully Remoteplease share resume at Khalil@weradioactive.comResponsibilities-Create Scala/Spark jobs for data transformation and aggregationProduce unit tests for Spark transformations and helper methodsWrite Scaladoc-style documentation with all codeDesign data processing...

  • Big Data Developer

    1 month ago


    Pune, India CloudQ IT Services Full time

    Looking for a candidate to help with migration of on-premise capital markets data lake to data bricks. Key skills : Spark, Scala, Pyspark- Good to have experience in Databricks - 4+ experience with various Hadoop/Databricks ecosystem components, 3+ years hands-on experience in Spark (Scala). - Working experience with Spark based data processing in Scala-...


  • Pune/Bangalore/Hyderabad/Mumbai/Delhi NCR, India TD Newton & Associates Full time

    Job Description : Main duties and responsibilities :- Delivering on critical business priorities while evolving platform towards its strategic vision- Working closely with product managers, end users, and business analysts to understand business objectives and key success measures - Partnering with technology teams to design and architect solutions that...


  • Pune, India CGI Information Systems and Management Consultants Full time

    Job Description :- Experience in application development experience in Scala, Spark, scala test/Junit libraries with maven.- Experience on Hive, Spark using Cloudera data platform.- Knowledge of performance optimization on Hadoop platform, Spark and Scala.- Ability to write optimized SQL queries.- Ability to learn & adapt.- Ability to communicate in clear &...

  • Data Streaming

    4 weeks ago


    Pune, India ACE Talent Consulting Full time

    ob Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Data Streaming

    4 weeks ago


    Pune, India ACE Talent Consulting Full time

    Job Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Data Engineer

    2 weeks ago


    Pune/Hyderabad, India SKYGATE CONSULTING Full time

    Role : Data EngineerExp : 5-12yrsLocation : Pune/HyderabadNotice Period : Immediate Joiners/ServingJob Description : Candidates who have good exposure in : - Strong Experience in Scala-Spark Development. - Strong experience in Apache, PySpark, Hadoop, Scala, Apache spark .- Extensive experience in writing advanced SQL statements and performance tuning...


  • Pune,Bangalore,Chennai,Hyderabad, India CGI Information Systems and Management Consultants Full time

    Job Description : - Experience in application development experience in Scala, Spark, scala test/Junit libraries with maven.- Experience on Hive, Spark using Cloudera data platform.- Knowledge of performance optimization on Hadoop platform, Spark and Scala.- Ability to write optimized SQL queries.- Ability to learn & adapt.- Ability to communicate in clear...


  • Pune/Bangalore/Chennai/Hyderabad, IN CGI Information Systems and Management Consultants Full time

    Job Description :- Experience in application development experience in Scala, Spark, scala test/Junit libraries with maven.- Experience on Hive, Spark using Cloudera data platform.- Knowledge of performance optimization on Hadoop platform, Spark and Scala.- Ability to write optimized SQL queries.- Ability to learn & adapt.- Ability to communicate in clear &...

  • WovV Technologies

    4 weeks ago


    Pune, Maharashtra, India Wovv Technology Full time

    Job Description: Big Data Pune (Hybrid Work module compulsory, WFH not 5-7 years.Contract Duration: Contract position for 6 months (can be Technology: Product development, Application development, Spark, Apache Airflow, Snowflake, BigQuery.Role and is a mandatory skill here. Please ensure that the candidate has worked into Scala before sharing the profiles....

  • WovV Technologies

    1 month ago


    Pune, India Wovv Technology Full time

    Job Description: Big Data Pune (Hybrid Work module compulsory, WFH not 5-7 years.Contract Duration: Contract position for 6 months (can be Technology: Product development, Application development, Spark, Apache Airflow, Snowflake, BigQuery.Role and is a mandatory skill here. Please ensure that the candidate has worked into Scala before sharing the...

  • WovV Technologies

    1 month ago


    Pune, India Wovv Technology Full time

    Job Description: Big Data Pune (Hybrid Work module compulsory, WFH not 5-7 years. Contract Duration: Contract position for 6 months (can be Technology: Product development, Application development, Spark, Apache Airflow, Snowflake, BigQuery. Role and is a mandatory skill here. Please ensure that the candidate has worked into Scala before sharing the...

  • Big Data Developer

    4 weeks ago


    Pune, India SP Software Pvt. Ltd. Full time

    Job Description :- Should have hands on experience in Software development- Responsible for designing, deploying, and managing high quality data solutions in the AWS cloud ecosystem.- Create and maintain optimal data architecture pipeline.- Collaborate with various onsite and client stakeholders in Agile environment to identify data engineering requirements...

  • Data Engineer

    1 month ago


    Pune, India TD NEWTON Full time

    Job Description : Must have Requirements : To be successful in this role, you should meet the following requirements : - Scala development and design using Scala 2.10+ or Java development and design using Java 1.8+. - Experience with most of the following technologies : Apache Hadoop, Scala, Apache Spark, Spark streaming, YARN, Kafka, Hive, Python, ETL...

  • WovV Technologies

    1 month ago


    Pune, India WovV Technologies Full time

    Location : Pune (Hybrid Work module compulsory, WFH not available).Experience : 5-7 years.Contract Duration : Contract position for 6 months (can be extended).Mandatory Technology : Product development, Application development, Spark, Apache Airflow, Snowflake, BigQuery.Role and Responsibilities :- Scala is a mandatory skill here.- Please ensure that the...

  • WovV Technologies

    1 month ago


    Pune, India WovV Technologies Full time

    Location : Pune (Hybrid Work module compulsory, WFH not available).Experience : 5-7 years.Contract Duration : Contract position for 6 months (can be extended).Mandatory Technology : Product development, Application development, Spark, Apache Airflow, Snowflake, BigQuery.Role and Responsibilities :- Scala is a mandatory skill here.- Please ensure that the...

  • WovV Technologies

    4 weeks ago


    Pune, Maharashtra, India WovV Technologies Full time

    Location : Pune (Hybrid Work module compulsory, WFH not available).Experience : 5-7 years.Contract Duration : Contract position for 6 months (can be extended).Mandatory Technology : Product development, Application development, Spark, Apache Airflow, Snowflake, BigQuery.Role and Responsibilities :- Scala is a mandatory skill here.- Please ensure that the...