Bigdata Scala Spark Data Engineer

4 weeks ago

Hyderabad, India LTIMindtree Full time
Primary Skill –

Scala SparkTotal Exp – 5.2 Years to 13 YearsNotice Period – ONLY 0 to 60 Days JoinersJob Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, DelhiJob Description:

Job Description:

JD: Spark + scala Tech LeadJob DescriptionWe are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully exploit the potential of Spark cluster. You will be responsible to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data to feature developers and business analysts.This involves both ad-hoc requests as well as data pipelines that are embedded in production environment.Responsibilities• Create Scala/Spark jobs for data transformation and aggregation• Produce unit tests for Spark transformations and helper methods• Write Scaladoc-style documentation with all code• Design data processing pipelinesWhat you’ll be doing:• Building distributed and highly parallelized Big data processing pipeline which process massive amount of data (both structured and unstructured data) in near real-time• Leveraging Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics• Working closely with analysts and business stakeholders to develop analytics models• Continuous delivering on Hadoop and other Big Data Platforms• Automating?processes where possible and are repeatable and reliable• Working closely with QA teamAs an expert Data Engineer, you are expected to have:• Excellent Programming language skills: Scala, Python, Bash, SQL, Bash• Excellent understanding of Hadoop and HDFS.• Excellent understanding of file types, their pros and cons• Excellent understanding of Build Tools especially SBT/Gradle• Very good understanding and good experience in implementing CICD e.g. Building and Maintaining Jenkins Pipelines• Working experience in SAFe/Agile• As an expert, you are also responsible for Juniors writing maintainable code• Are collaborative and achieve your expectations through communication and teamwork• Are curious, responsive and can understand the needs of others to ensure delivery of the desired results• Work qualitatively and strive to always do things betterSkills• Spark + Scala Scala (with a focus on the programming )• Apache Spark 2.x• Spark query tuning and performance optimization• Understanding on Hadoop Architecture• Experience working with Hive, SQL and HDFS• Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)• Good to have experience in Scala• Experience working with tools like Jenkins, Jira, Bitbucket and Git• Experience in writing shell scripts and working with Linux platforms.As an advantage, familiarity with one or more of the below technologies:• Experience?working in Agile/SAFe• Experience/certification with AWS Cloud

  • Bigdata Scala Spark Data Engineer

    1 month ago

    Hyderabad, India LTIMindtree Full time

    Primary Skill – Scala Spark Total Exp – 5.2 Years to 13 Years Notice Period – ONLY 0 to 60 Days Joiners Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, Delhi Job Description: Job Description: JD: Spark + scala Tech Lead Job Description We are looking for a Spark Scala Tech Lead with 5+ Years experience who...

  • Bigdata Scala Spark Data Engineer

    1 month ago

    Hyderabad, India LTIMindtree Full time

    Primary Skill – Scala SparkTotal Exp – 5.2 Years to 13 YearsNotice Period – ONLY 0 to 60 Days Joiners Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, DelhiJob Description:Job Description:JD: Spark + scala Tech LeadJob DescriptionWe are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully...

  • Bigdata Scala Spark Data Engineer

    1 month ago

    Hyderabad, India LTIMindtree Full time

    Primary Skill – Scala SparkTotal Exp – 5.2 Years to 13 YearsNotice Period – ONLY 0 to 60 Days Joiners Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, DelhiJob Description:Job Description:JD: Spark + scala Tech LeadJob DescriptionWe are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully...

  • Spark Scala Developer

    3 weeks ago

    Hyderabad, India Corporate Resources Full time

    Job Description4+ years of experience with Java, Spark, Bigdata & Hadoop with strong Unix backgroundCreate Scala/Spark jobs for data transformation and aggregationExperience of writing code in spark and ScalaProduce unit tests for Spark transformations and helper methodsWrite Scaladoc-style documentation with all codeDesign data processing pipelinesScala...

  • Spark- Scala

    1 month ago

    Hyderabad, India Nityo Infotech Full time

    • working experience in Scala• experience using Spark programming model in Scala.• 3-4 years of experience in Advanced Java development and MongoDB• Experience in working with Core Spark, Spark Streaming, Data Frame API, Data set API, Data Source API & Spark SQL• Experience in working with Apache Kafka, developing consumers & producers using...

  • Data Engineer

    2 weeks ago

    Pune/Hyderabad, India SKYGATE CONSULTING Full time

    Role : Data EngineerExp : 5-12yrsLocation : Pune/HyderabadNotice Period : Immediate Joiners/ServingJob Description : Candidates who have good exposure in : - Strong Experience in Scala-Spark Development. - Strong experience in Apache, PySpark, Hadoop, Scala, Apache spark .- Extensive experience in writing advanced SQL statements and performance tuning...

  • Bigdata

    1 month ago

    Hyderabad, India Bloom Consulting Services Full time

    Big Data, AWS, Python, Scala - Hyderabad, Chennai - Full Time Experience **05.0 - 10.0 years** Offered Salary **15.00 - 24.00** Notice Period **Not Disclosed** **Bigdata**: **Experience : 5-10 years** **Location : Hyderabad,chennai** - Expertise in building AWS Data Engineering pipelines with AWS Glue -> Athena -> Quick sight - Experience in...

  • Bigdata

    4 weeks ago

    Hyderabad, India Bloom Consulting Services Full time

    Hyderabad, Chennai - Full Time - Big Data, AWS, Python, Scala Experience **05.0 - 10.0 Years** Offered Salary **15.00 - 24.00 Lakh** Notice Period **Not Disclosed** **Bigdata**: **Experience : 5-10 years** **Location : Hyderabad,chennai** - Expertise in building AWS Data Engineering pipelines with AWS Glue -> Athena -> Quick sight - Experience in...

  • Data Engineer with Scala

    18 hours ago

    hyderabad, India Brillius Full time

    Role name:     Data Engineer with ScalaRole Description:Data Engineer with Scala• Exceptional skills in Scala and Spark• Hands on experience with data processing technologies, ETL processes and feature engineering• A track record of developing scalable pipelines and delivering data promptly in a collaborative team environmentAdditional Nice to...

  • Data Engineer with Scala

    1 day ago

    Hyderabad, India Brillius Full time

    Role name:     Data Engineer with Scala Role Description:       Data Engineer with Scala• Exceptional skills in Scala and Spark• Hands on experience with data processing technologies, ETL processes and feature engineering• A track record of developing scalable pipelines and delivering data promptly in a collaborative team...

  • Data Engineer with Scala

    18 hours ago

    hyderabad, India Brillius Full time

    Role name:     Data Engineer with Scala   Role Description:        Data Engineer with Scala • Exceptional skills in Scala and Spark • Hands on experience with data processing technologies, ETL processes and feature engineering • A track record of developing scalable pipelines and delivering data promptly in a collaborative team environment...

  • Software Engineer

    1 week ago

    hyderabad, India Trinity Infotech Inc. Full time

    5+ years of experience working in a software product development organization building modern and scalable big data applicationsExpertise with Scala functional programming (intermediate/advanced level is a mandate)Data structures and algorithms in ScalaFluency in Hadoop technologies around handling big data processing e.g. SparkExperience implementing...

  • CGI - Big Data Engineer - Scala/Spark

    1 month ago

    Pune,Bangalore,Chennai,Hyderabad, India CGI Information Systems and Management Consultants Full time

    Job Description : - Experience in application development experience in Scala, Spark, scala test/Junit libraries with maven.- Experience on Hive, Spark using Cloudera data platform.- Knowledge of performance optimization on Hadoop platform, Spark and Scala.- Ability to write optimized SQL queries.- Ability to learn & adapt.- Ability to communicate in clear...

  • CGI - Big Data Engineer - Scala/Spark

    4 weeks ago

    Pune/Bangalore/Chennai/Hyderabad, IN CGI Information Systems and Management Consultants Full time

    Job Description :- Experience in application development experience in Scala, Spark, scala test/Junit libraries with maven.- Experience on Hive, Spark using Cloudera data platform.- Knowledge of performance optimization on Hadoop platform, Spark and Scala.- Ability to write optimized SQL queries.- Ability to learn & adapt.- Ability to communicate in clear &...

  • Software Engineer

    1 week ago

    Hyderabad, India Trinity Infotech Inc. Full time

    5+ years of experience working in a software product development organization building modern and scalable big data applicationsExpertise with Scala functional programming (intermediate/advanced level is a mandate)Data structures and algorithms in ScalaFluency in Hadoop technologies around handling big data processing e.g. SparkExperience implementing...

  • Software Engineer

    1 week ago

    hyderabad, India Trinity Infotech Inc. Full time

    5+ years of experience working in a software product development organization building modern and scalable big data applications Expertise with Scala functional programming (intermediate/advanced level is a mandate) Data structures and algorithms in Scala Fluency in Hadoop technologies around handling big data processing e.g. Spark Experience implementing...

  • Software Engineer

    1 week ago

    Hyderabad, India Trinity Infotech Inc. Full time

    5+ years of experience working in a software product development organization building modern and scalable big data applicationsExpertise with Scala functional programming (intermediate/advanced level is a mandate)Data structures and algorithms in ScalaFluency in Hadoop technologies around handling big data processing e.g. SparkExperience implementing...

  • Big data spark

    3 weeks ago

    Hyderabad, India Virtusa Full time

    Big data spark - CREQ178115 DescriptionPosition: Big data spark developerPrimary skills: Big data spark & JavaLocation: Hyderabad & BangaloreEssential:5+ years of IT experience in Bigdata tech stack (Java Spark, Hadoop, Hive)Experience in writing SQL for any one relation databaseExtensive knowledge in HiveSQL Hadoop, SQLExperience in DevOps process (CICD,...

  • Big data spark

    4 weeks ago

    hyderabad, India Virtusa Full time

    Big data spark - CREQ178115 Description Position: Big data spark developer Primary skills: Big data spark & Java Location: Hyderabad & Bangalore Essential:5+ years of IT experience in Bigdata tech stack (Java Spark, Hadoop, Hive)Experience in writing SQL for any one relation databaseExtensive knowledge in HiveSQL Hadoop, SQLExperience in DevOps...

  • Big data spark

    1 month ago

    Hyderabad, India Virtusa Full time

    Big data spark - CREQ178115 Description Position: Big data spark developer Primary skills: Big data spark & Java Location: Hyderabad & Bangalore Essential: 5+ years of IT experience in Bigdata tech stack (Java Spark, Hadoop, Hive) Experience in writing SQL for any one relation database Extensive knowledge in HiveSQL Hadoop, SQL Experience in DevOps...