BigData- Spark/Scala

1 day ago


Chennai, Tamil Nadu, India Prodapt Full time ₹ 20,00,000 - ₹ 25,00,000 per year
Overview

  

We are looking for a highly skilled Big Data Engineer with extensive experience in Spark and Scala to join our team. The ideal candidate will play a crucial role in designing, developing, and optimizing large-scale data processing systems. You will work closely with data scientists, analysts, and other stakeholders to deliver high-quality data solutions.

Responsibilities

Key Responsibilities: -

  • Design, develop, and maintain scalable data pipelines using Apache Spark and Scala.
  • Collaborate with cross-functional teams to understand data requirements and deliver data solutions that meet business needs.
  • Optimize Spark jobs for performance and cost-efficiency in a distributed computing environment.
  • Implement best practices for data modeling, ETL processes, and data governance.
  • Monitor and troubleshoot data processing workflows to ensure data integrity and availability.
  • Work with cloud platforms (AWS, Azure, or GCP) to implement big data solutions.
  • Stay up to date with industry trends and emerging technologies in big data and analytics.
  
Requirements
  • 7-10 years of experience in Big Data technologies, with a strong focus on Apache Spark and Scala.
  • Proficiency in data processing frameworks (Hadoop, Spark) and languages (Scala, Java).
  • Experience with data warehousing solutions (Snowflake, Redshift, etc.) and SQL.
  • Knowledge of data modeling, ETL processes, and data visualization tools (Tableau, Power BI).
  • Familiarity with cloud services (AWS, Azure, Google Cloud) and containerization (Docker, Kubernetes).
  • Strong analytical skills and the ability to work with large datasets.
  • Excellent communication and teamwork skills.
  
  • BigData- Spark/Scala

    24 hours ago


    Chennai, Tamil Nadu, India Prodapt Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Overview:We are looking for a highly skilled Big Data Engineer with extensive experience in Spark and Scala to join our team. The ideal candidate will play a crucial role in designing, developing, and optimizing large-scale data processing systems. You will work closely with data scientists, analysts, and other stakeholders to deliver high-quality data...

  • Spark Scala Python

    1 week ago


    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Job SummaryJoin our dynamic team as a Developer where you will leverage your expertise in Spark Scala Python and SparkSQL to drive innovative solutions. With a hybrid work model and day shifts you will collaborate with cross-functional teams to enhance data processing capabilities. Your contributions will directly impact our companys growth and societal...

  • Spark Scala Developer

    2 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Dear Candidates,Greetings from TCS!!!!TCS is looking for Spark Scala DeveloperExperience: 6-8 yearsLocation: ChennaiMUST HAVE SKILLS:Data Engineer Design and develop scalable and efficient solutions using Spark, Scala, Python, Airflow.Good code and configuration spark knowledge (at least 1 or 2 profiles should be very good at spark configuration and cluster...


  • Chennai, Tamil Nadu, India Virtusa Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    7+ years of experience in Big Data with strong expertise in Spark and ScalaMandatory Skills: Big Data Primarily Spark and ScalaStrong Knowledge in HDFS, Hive, Impala with knowledge on Unix , Oracle, Autosys,Good to Have : Agile Methodology and Banking ExpertiseStrong Communication SkillsNot limited to Spark batch, need Spark streaming experienceNo SQL DB...

  • Scala Lead Engineer

    1 week ago


    Chennai, Tamil Nadu, India Techominds Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Summary:Skills: Lead Data Eng with Scala, Spark, Pyspark & DatabricksJob Title: Data Engineer (Scala, Spark, Pyspark, Databricks)Location: ChennaiResponsibilities:Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform.Implement ETL (Extract, Transform, Load) processes for various data...

  • Scala Developer

    1 week ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Role : Scala DeveloperExp : 4+ year Location : Pune,Chennai, Kolkata, Hyderabad, Gurgoan, Functional Skills: Experience in Credit Risk/Regulatory risk domain Technical Skills: Spark ,PySpark, Python, Hive, Scala, MapReduce, Unix shell scripting Good to Have Skills: Exposure to Machine Learning TechniquesJob Description:4+ Years of experience with...

  • Bigdata Engineer

    24 hours ago


    Chennai, Tamil Nadu, India Virtusa Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Hands on knowledge of SPARK, scala, etc.Hands on knowledge of RDBMS like MS-SQL/OracleKnowledge of CI/CD tools like uDeployUnderstanding of Vanguard and its landscape will be additional advantageAbout VirtusaTeamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000...

  • Bigdata Developer

    2 weeks ago


    Chennai, Tamil Nadu, India Virtusa Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    We are seeking a highly skilled and motivated Python , AWS, Big Data Engineer to join our data engineering team. The ideal candidate will have hands-on experience with Hadoop ecosystem, Apache Spark, and programming expertise in Python (PySpark), Scala, and Java. You will be responsible for designing, developing, and optimizing scalable data pipelines and...

  • Big Data Developer

    1 day ago


    Chennai, Tamil Nadu, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Greetings from TCSTCS has been a great pioneer in feeding the fire of young Techies like you. We are a global leader in the technology arena and there's nothing that can stop us from growing together.Your role is of key importance, as it lays down the foundation for the entire project.Position:Bigdata DeveloperJob Location:Bangalore, ChennaiExperience:5+Must...


  • Chennai, Tamil Nadu, India Cognizant Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key skills: Azure SQL, AZURE Synapse, SparkExperience: 6 to 13 yearsLocation: KolkataJob Description:Solution Design and Development:Collaborate with business stakeholders and data architects to understand data requirements and design scalable and efficient solutions using Azure Synapse Analytics.Develop and implement data pipelines, data warehouses, and...