Spark/Scala

1 day ago


Pune, Maharashtra, India Tata Consultancy Services (TCS) Full time

Role:Senior Developer

Required Technical Skill Set: Spark/Scala/Unix

Experience Range: 5-8 years

Location of Requirement : Pune

Must-Have

  1. Minimum 4+ years of experience in development of Spark Scala

  2. Experience in designing and development of solutions for Big Data using Hadoop ecosystem technologies such as with Hadoop Bigdata components like HDFS, Spark, Hive Parquet File format, YARN, MapReduce, Sqoop

  3. Good Experience in writing and optimizing Spark Jobs, Spark SQL etc. Should have worked on both batch and streaming data processing.

  4. Experience in writing and optimizing complex Hive and SQL queries to process huge data. good with UDFs, tables, joins, Views etc

  5. Experience in debugging the Spark code

  6. Working knowledge of basic UNIX commands and shell script

  7. Experience of Autosys, Gradle

Good-to-Have 1. Good analytical and debugging skills

  1. Ability to coordinate with SMEs, stakeholders, manage timelines, escalation & provide on time status

  2. Write clear and precise documentation / specification

  3. Work in an agile environment

  4. Create documentation and document all developed mappings

SN Responsibility of / Expectations from the Role

1 Create Scala/Spark jobs for data transformation and aggregation

2 Produce unit tests for Spark transformations and helper methods

3 Write Scaladoc-style documentation with all code

4 Design data processing pipelines

Location

Pune

Job Function

TECHNOLOGY

Role

Engineer

Job Id

381936

Desired Skills

Scala | Spark

Desired Candidate Profile

Qualifications : BACHELOR OF ENGINEERING


  • Sr. Scala Engineer

    2 weeks ago


    Pune, Maharashtra, India Persistent Full time

    About Position:As a Senior Scala Engineer, youll be part of a team of smart, highly skilled technologists who are passionate about learning and supporting utting-edge technologies such as Spark, Scala, Pyspark, Databricks, Airflow, SQL, Docker, Kubernetes, and other Data engineering tools.These technologies are deployed using DevOps pipelines leveraging...


  • Pune, Maharashtra, India Citi Full time

    Lead the architecture, design, and development of high-performance, scalable, and reliable big data processing systems using Scala and Apache Spark. Drive technical vision and strategy for big data initiatives, ensuring alignment with overall company goals and industry best practices. Evaluate and recommend new technologies and tools to enhance our big data...


  • Pune, Maharashtra, India HSBC Full time

    Some careers shine brighter than others.If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.HSBC is one of the largest banking and...


  • Pune, Maharashtra, India OPUS Technologies Full time

    Who are we?Opus Technologies is a technology company with a focus on shaping the future of payment technology. With experience building highly innovative solutions and products, we combine our deep technology proficiency with unmatched domain expertise in payments and fintech, enabling us to deliver unparalleled quality and value in everything we do. For the...


  • Pune, Maharashtra, India Citi Full time

    Vice President, Big Data Scala EngineerLocation: Pune About the Role:We are seeking an experienced and highly skilled Vice President, Big Data Scala Engineer to lead the design, development, and implementation of our next-generation big data platforms and applications. The ideal candidate will be a technical leader with a strong background in Scala, Spark,...


  • Pune, Maharashtra, India CRUTZ LEELA ENTERPRISES Full time

    Sr. Spark Big Data DeveloperPFB the detailed JD:�� Experience: 5 to 7+ years (Hands-on)�� Notice Period: 0-30 DaysMust Have: Proficiency in at least one of the following programming languages:Java/Scala/Python Good understanding of SQL Experience of development and deployment of at least one end-to-end datastorage/processing pipeline...

  • Lead Data Engineer

    6 days ago


    Pune, Maharashtra, India CG-VAK Software & Exports Ltd. Full time

    Job Title:Data Engineer (Scala, Spark, Pyspark, Databricks)Remark - final round will be the F2FExperience: 10+ Years (Lead Role)ResponsibilitiesDesign, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform.Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data...

  • Big Data Engineer

    2 weeks ago


    Pune, Maharashtra, India Digital Placement Full time

    Primary – Scala with Spark Hdfs  ,Spark with scala , Scala  (Functional Programming ), Hbase , Sql , Sqoop ,Shell Script Optional :     Apache  Kafka , Akka Http , TeamCity , U-deploy ,  Autosys , SBT Good to have  :     Pivotal Gemfire 7+ years of strong experience as big data developer. Hands on knowledge of Shell Scripting. Familiar with...

  • Aws Data Engineer

    1 week ago


    Pune, Maharashtra, India Tech Mahindra Full time

    AWS Data EngineerExperience: 3-8 YearsLocation: Pune (Kharadi)About the RoleWe are looking for a highly skilled Spark SQL Developer with a solid background in Data Engineering. The candidate should have hands-on experience in building and optimizing large-scale data pipelines using Apache Spark, along with strong expertise in SQL and ETL processes.Key...

  • Data Engineer

    1 week ago


    Pune, Maharashtra, India HSBC Full time

    Some careers shine brighter than others.If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.HSBC is one of the largest banking and...