Bigdata Scala Spark Data Engineer

3 weeks ago

Hyderabad, India LTIMindtree Full time

Primary Skill – Scala Spark

Total Exp – 5.2 Years to 13 Years

Notice Period – ONLY 0 to 60 Days Joiners

Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, Delhi

Job Description:


Job Description:


JD: Spark + scala Tech Lead

Job Description

We are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully exploit the potential of Spark cluster. You will be responsible to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data to feature developers and business analysts.

This involves both ad-hoc requests as well as data pipelines that are embedded in production environment.

Responsibilities

• Create Scala/Spark jobs for data transformation and aggregation

• Produce unit tests for Spark transformations and helper methods

• Write Scaladoc-style documentation with all code

• Design data processing pipelines

What you’ll be doing:

• Building distributed and highly parallelized Big data processing pipeline which process massive amount of data (both structured and unstructured data) in near real-time

• Leveraging Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics

• Working closely with analysts and business stakeholders to develop analytics models

• Continuous delivering on Hadoop and other Big Data Platforms

• Automating?processes where possible and are repeatable and reliable

• Working closely with QA team

As an expert Data Engineer, you are expected to have:

• Excellent Programming language skills: Scala, Python, Bash, SQL, Bash

• Excellent understanding of Hadoop and HDFS.

• Excellent understanding of file types, their pros and cons

• Excellent understanding of Build Tools especially SBT/Gradle

• Very good understanding and good experience in implementing CICD e.g. Building and Maintaining Jenkins Pipelines

• Working experience in SAFe/Agile

• As an expert, you are also responsible for Juniors writing maintainable code

• Are collaborative and achieve your expectations through communication and teamwork

• Are curious, responsive and can understand the needs of others to ensure delivery of the desired results

• Work qualitatively and strive to always do things better

Skills

• Spark + Scala Scala (with a focus on the programming )

• Apache Spark 2.x

• Spark query tuning and performance optimization

• Understanding on Hadoop Architecture

• Experience working with Hive, SQL and HDFS

• Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)

• Good to have experience in Scala

• Experience working with tools like Jenkins, Jira, Bitbucket and Git

• Experience in writing shell scripts and working with Linux platforms.

As an advantage, familiarity with one or more of the below technologies:

• Experience?working in Agile/SAFe

• Experience/certification with AWS Cloud

  • Bigdata Scala Spark Data Engineer

    2 months ago

    Hyderabad, India LTIMindtree Full time

    Primary Skill –Scala SparkTotal Exp – 5.2 Years to 13 YearsNotice Period – ONLY 0 to 60 Days JoinersJob Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, DelhiJob Description:Job Description:JD: Spark + scala Tech LeadJob DescriptionWe are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully...

  • Bigdata Scala Spark Data Engineer

    3 weeks ago

    hyderabad, India LTIMindtree Full time

    Primary Skill – Scala SparkTotal Exp – 5.2 Years to 13 YearsNotice Period – ONLY 0 to 60 Days JoinersJob Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, DelhiJob Description:Job Description:JD: Spark + scala Tech LeadJob DescriptionWe are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully...

  • Bigdata Scala Spark Data Engineer

    3 weeks ago

    Hyderabad, India LTIMindtree Full time

    Primary Skill – Scala Spark Total Exp – 5.2 Years to 13 Years Notice Period – ONLY 0 to 60 Days Joiners Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, Delhi Job Description: Job Description: JD: Spark + scala Tech Lead Job Description We are looking for a Spark Scala Tech Lead with 5+ Years experience who...

  • Bigdata Scala Spark Data Engineer

    3 weeks ago

    Hyderabad, India LTIMindtree Full time

    Primary Skill – Scala SparkTotal Exp – 5.2 Years to 13 YearsNotice Period – ONLY 0 to 60 Days Joiners Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, DelhiJob Description:Job Description:JD: Spark + scala Tech LeadJob DescriptionWe are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully...

  • Bigdata Scala Spark Data Engineer

    2 months ago

    Hyderabad, India LTIMindtree Full time

    Primary Skill – Scala SparkTotal Exp – 5.2 Years to 13 YearsNotice Period – ONLY 0 to 60 Days Joiners Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, DelhiJob Description:Job Description:JD: Spark + scala Tech LeadJob DescriptionWe are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully...

  • Bigdata Scala Spark Data Engineer

    2 months ago

    Hyderabad, India LTIMindtree Full time

    Primary Skill – Scala Spark Total Exp – 5.2 Years to 13 Years Notice Period – ONLY 0 to 60 Days Joiners Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, Delhi Job Description: Job Description: JD: Spark + scala Tech Lead Job Description We are looking for a Spark Scala Tech Lead with 5+ Years experience who...

  • Bigdata Scala Spark Data Engineer

    1 week ago

    Hyderabad, Telangana, India LTIMindtree Full time

    Primary Skill – Scala Spark Total Exp – 5.2 Years to 13 Years Notice Period – ONLY 0 to 60 Days Joiners Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, Delhi Job Description:Job Description: JD: Spark + scala Tech Lead Job Description We are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to...

  • Spark Scala Engineer

    2 weeks ago

    Hyderabad, India NucleusTeq Full time

    We are looking for exceptional Spark Scala engineer with 5+ yrs experience who will be responsible for:Experience- 5+ YrsLocation- Hyderabad & IndoreResponsibilities:Implementing the large scale Spark applications and finetune at runtimeDesign and implement high volume and cardinality data using IcebergImplement and optimize pipelines using...

  • Spark Scala Engineer

    1 week ago

    Hyderabad, Telangana, India NucleusTeq Full time

    We are looking for exceptional Spark Scala engineer with 5+ yrs experience who will be responsible for:Experience- 5+ YrsLocationHyderabad & IndoreResponsibilities:Implementing the large scale Spark applications and finetune at runtimeDesign and implement high volume and cardinality data using IcebergImplement and optimize pipelines using AirflowExperience...

  • Spark Scala Engineer

    7 days ago

    hyderabad, India NucleusTeq Full time

    We are looking for exceptional Spark Scala engineer with 5+ yrs experience who will be responsible for: Experience- 5+ Yrs Location- Hyderabad & Indore Responsibilities: Implementing the large scale Spark applications and finetune at runtime Design and implement high volume and cardinality data using Iceberg...

  • Spark Scala Engineer

    1 week ago

    Hyderabad, Telangana, India NucleusTeq Full time

    We are looking for exceptional Spark Scala engineer with 5+ yrs experience who will be responsible for: Experience- 5+ Yrs Location- Hyderabad & Indore Responsibilities:Implementing the large scale Spark applications and finetune at runtimeDesign and implement high volume and cardinality data using IcebergImplement and optimize pipelines using...

  • Spark Scala Engineer

    2 weeks ago

    Hyderabad, India NucleusTeq Full time

    We are looking for exceptional Spark Scala engineer with 5+ yrs experience who will be responsible for: Experience- 5+ Yrs Location- Hyderabad & Indore Responsibilities: Implementing the large scale Spark applications and finetune at runtimeDesign and implement high volume and cardinality data using...

  • Revrr - Data Engineer - Scala/Spark

    4 weeks ago

    Hyderabad, India REVRR TECHNOLOGIES PRIVATE LIMITED Full time

    Company : Revrr TechnologiesEmployment Type : Full-TimeRole : Software Development EngineerPosition : Developer, Senior Developer, and Lead DeveloperProject : Maps and Advertising Platforms ProductExperience: Sr SDE: 4-8 YearsKey Skills : Spark Core/Streaming with Any programming language Scala/ Java/ Python, Good Understanding of Data Structures,...

  • Data Engineer

    11 hours ago

    Hyderabad, India Tech Mahindra Full time

    About the CompanyTech Mahindra offers innovative and customer-centric digital experiences, enabling enterprises, associates and the society to Rise. We are focused on leveraging next-generation technologies including 5G, Blockchain, Cybersecurity, Artificial Intelligence, and more, to enable end-to-end digital transformation for global customers. Tech...

  • Data Engineer

    2 days ago

    Hyderabad, India Tech Mahindra Full time

    About the CompanyTech Mahindra offers innovative and customer-centric digital experiences, enabling enterprises, associates and the society to Rise. We are focused on leveraging next-generation technologies including 5G, Blockchain, Cybersecurity, Artificial Intelligence, and more, to enable end-to-end digital transformation for global customers. Tech...

  • Data Engineer

    18 hours ago

    hyderabad, India Tech Mahindra Full time

    About the Company Tech Mahindra offers innovative and customer-centric digital experiences, enabling enterprises, associates and the society to Rise. We are focused on leveraging next-generation technologies including 5G, Blockchain, Cybersecurity, Artificial Intelligence, and more, to enable end-to-end digital transformation for global customers. Tech...

  • Spark- Scala

    2 months ago

    Hyderabad, India Nityo Infotech Full time

    • working experience in Scala• experience using Spark programming model in Scala.• 3-4 years of experience in Advanced Java development and MongoDB• Experience in working with Core Spark, Spark Streaming, Data Frame API, Data set API, Data Source API & Spark SQL• Experience in working with Apache Kafka, developing consumers & producers using...

  • Spark- Scala

    3 weeks ago

    Hyderabad, India Nityo Infotech Full time

    • working experience in Scala• experience using Spark programming model in Scala.• 3-4 years of experience in Advanced Java development and MongoDB• Experience in working with Core Spark, Spark Streaming, Data Frame API, Data set API, Data Source API & Spark SQL• Experience in working with Apache Kafka, developing consumers & producers using...

  • Software Engineer

    3 weeks ago

    Hyderabad, India Whitefield Careers Full time

    Notice Period : Max 30 daysWe are searching for a talented Software Engineer to join our growing team. In this role, you will be responsible for designing, developing, and maintaining large-scale data processing applications using Scala and Spark. You will work closely with data scientists and engineers to develop solutions that transform raw data into...

  • Software Engineer

    4 weeks ago

    Hyderabad, India Whitefield Careers Full time

    Notice Period : Max 30 daysWe are searching for a talented Software Engineer to join our growing team. In this role, you will be responsible for designing, developing, and maintaining large-scale data processing applications using Scala and Spark. You will work closely with data scientists and engineers to develop solutions that transform raw data into...