BigData- Spark/Scala

5 hours ago


Chennai, Tamil Nadu, India Prodapt Full time ₹ 20,00,000 - ₹ 25,00,000 per year
Overview

  

We are looking for a highly skilled Big Data Engineer with extensive experience in Spark and Scala to join our team. The ideal candidate will play a crucial role in designing, developing, and optimizing large-scale data processing systems. You will work closely with data scientists, analysts, and other stakeholders to deliver high-quality data solutions.

Responsibilities

Key Responsibilities: -

  • Design, develop, and maintain scalable data pipelines using Apache Spark and Scala.
  • Collaborate with cross-functional teams to understand data requirements and deliver data solutions that meet business needs.
  • Optimize Spark jobs for performance and cost-efficiency in a distributed computing environment.
  • Implement best practices for data modeling, ETL processes, and data governance.
  • Monitor and troubleshoot data processing workflows to ensure data integrity and availability.
  • Work with cloud platforms (AWS, Azure, or GCP) to implement big data solutions.
  • Stay up to date with industry trends and emerging technologies in big data and analytics.
  
Requirements
  • 7-10 years of experience in Big Data technologies, with a strong focus on Apache Spark and Scala.
  • Proficiency in data processing frameworks (Hadoop, Spark) and languages (Scala, Java).
  • Experience with data warehousing solutions (Snowflake, Redshift, etc.) and SQL.
  • Knowledge of data modeling, ETL processes, and data visualization tools (Tableau, Power BI).
  • Familiarity with cloud services (AWS, Azure, Google Cloud) and containerization (Docker, Kubernetes).
  • Strong analytical skills and the ability to work with large datasets.
  • Excellent communication and teamwork skills.
  
  • BigData- Spark/Scala

    2 weeks ago


    Chennai, Tamil Nadu, India Prodapt Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Overview Big Data, Spark Scala strong  Responsibilities Big Data, Spark Scala strong  Requirements Big Data, Spark Scala strong 

  • Spark Scala Python

    1 week ago


    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Job SummaryJoin our dynamic team as a Developer where you will leverage your expertise in Spark Scala Python and SparkSQL to drive innovative solutions. With a hybrid work model and day shifts you will collaborate with cross-functional teams to enhance data processing capabilities. Your contributions will directly impact our companys growth and societal...


  • Chennai, Tamil Nadu, India Prodapt Solutions Private Limited Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Big Data Engineer - Spark/Scala Exp : 7 to 10 years Location : Chennai Overview : We are looking for a highly skilled Big Data Engineer with extensive experience in Spark and Scala to join our team. The ideal candidate will play a crucial role in designing, developing, and optimizing large-scale data processing systems. You will work closely with...


  • Chennai, Tamil Nadu, India BNP Paribas Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Position Purpose The Senior Spark Developer will be responsible for integrating and maintaining the Quantexa platform, a spark based software provided by a UK fintech, into our existing systems to enhance our anti-money laundering capabilities. This role requires a deep expertise in Spark development, as well as an ability to analyze and understand...

  • BigData Developer

    4 hours ago


    Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Must Have :Hands on development experience in programming languages such as JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scriptingShould be comfortable with Unix File system as well as HDFS commandsShould have worked on query languages such as Oracle SQL, Hive SQL, Spark SQL, Impala, HBase DBShould be flexibleShould have good communication...

  • Scala Lead Engineer

    7 days ago


    Chennai, Tamil Nadu, India Techominds Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Summary:Skills: Lead Data Eng with Scala, Spark, Pyspark & DatabricksJob Title: Data Engineer (Scala, Spark, Pyspark, Databricks)Location: ChennaiResponsibilities:Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform.Implement ETL (Extract, Transform, Load) processes for various data...


  • Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Overall Minimum 6+ years of post-college working experience as a developer in Bigdata, Hadoop or Data Warehousing Tools and Cloud Computing (Added Skill)Minimum 3+ Years of experience Hadoop, Hive SQL's, Spark, Bigdata Eco System Tools.Preferable to have good technical knowledge on Cloud computing, AWS or Azure Cloud Services.Strong conceptual and creative...


  • Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Key responsibilitiesThe role requires a hands-on technologist with expertise in Data Engineering with strong programming background in Hadoop/Hive/Sparks/ScalaShould have hands-on knowledge on GCP cloud platforms.Provide technical leadership and hands-on implementation role in the areas of data engineering including data ingestion, data access, modelling,...


  • Chennai, Tamil Nadu, India Cognizant Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking a highly experienced Sr. Architect with 16+ years of experience to join our team.The ideal candidate will have expertise in Python, Splunk, Spark, Hadoop, Data & Solution Architect, Integration Architect, and Solution Design & Architecture.This hybrid role requires a deep understanding of technical architecture and the ability to design and...

  • Scala Developer

    4 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Role : Scala DeveloperExp : 4+ year Location : Pune,Chennai, Kolkata, Hyderabad, Gurgoan, Functional Skills: Experience in Credit Risk/Regulatory risk domain Technical Skills: Spark ,PySpark, Python, Hive, Scala, MapReduce, Unix shell scripting Good to Have Skills: Exposure to Machine Learning TechniquesJob Description:4+ Years of experience with...