Spark Scala Developer

14 hours ago

Chennai Pune, India Cloudxtreme Full time US$ 1,20,000 - US$ 2,00,000 per year

Mandatory - Experience and knowledge in designing, implementing, and managing non-relational data stores (e.g., MongoDB, Cassandra, DynamoDB), focusing on flexible schema design, scalability, and performance optimization for handling large volumes of unstructured or semi-structured data.

Mainly client needs No SQL DB, either MongoDB or HBase

  • Data Pipeline Development: Design, develop, test, and deploy robust, high-performance, and scalable ETL/ELT data pipelines using Scala and Apache Spark to ingest, process, and transform large volumes of structured and unstructured data from diverse sources.
  • Big Data Expertise: Leverage expertise in the Hadoop ecosystem (HDFS, Hive, etc.) and distributed computing principles to build efficient and fault-tolerant data solutions.
  • Advanced SQL: Write complex, optimized SQL queries and stored procedures.
  • Performance Optimization: Continuously monitor, analyze, and optimize the performance of data pipelines and data stores. Troubleshoot complex data-related issues, identify bottlenecks, and implement solutions for improved efficiency and reliability.
  • Data Quality & Governance: Implement data quality checks, validation rules, and reconciliation processes to ensure the accuracy, completeness, and consistency of data. Contribute to data governance and security best practices.
  • Automation & CI/CD: Implement automation for data pipeline deployment, monitoring, and alerting using tools like Apache Airflow, Jenkins, or similar CI/CD platforms.
  • Documentation: Create and maintain comprehensive technical documentation for data architectures, pipelines, and processes.

Required Skills & Qualifications:

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
  • Minimum 5 years of professional experience in Data Engineering, with a strong focus on big data technologies.
  • Proficiency in Scala for developing big data applications and transformations, especially with Apache Spark.
  • Expert-level proficiency in SQL; ability to write complex queries, optimize performance, and understand database internals.
  • Extensive hands-on experience with Apache Spark (Spark SQL, DataFrames, RDDs) for large-scale data processing and analytics.
  • Mandatory - Experience and knowledge in designing, implementing, and managing non-relational data stores (e.g., MongoDB, Cassandra, DynamoDB), focusing on flexible schema design, scalability, and performance optimization for handling large volumes of unstructured or semi-structured data.
  • Solid understanding of distributed computing concepts and experience with the Hadoop ecosystem (HDFS, Hive).
  • Experience with building and optimizing ETL/ELT processes and data warehousing concepts.
  • Strong understanding of data modeling techniques (e.g., Star Schema, Snowflake Schema).
  • Familiarity with version control systems (e.g., Git).
  • Excellent problem-solving, analytical, and communication skills.
  • Ability to work independently and collaboratively in an Agile team environment.

  • Scala Spark Developer

    14 hours ago

    Chennai, Tamil Nadu, India Path Mentors Staffing Solution Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Scala/Spark Developer: 5+ yrs experience in designing scalable data pipelines with Scala, Spark, Hadoop, Kafka. Skilled in data formats(Avro,Parquet),HDFS, SQL, NoSQL. Proficient in troubleshooting, optimization, delivering reliable bigdata solutions Required Candidate profileScala/Spark Developer with 5+ years' experience in building scalable data...

  • Spark Scala Developer

    7 days ago

    Chennai, Tamil Nadu, India Infotel UK Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Position: Spark Scala DeveloperCompany: Infotel IndiaAt Infotel UK, a premier technology consulting firm, we are seeking a skilled Spark Scala Developer to join our dynamic team. In this role, you will leverage your expertise in Spark and Scala to design and implement advanced data processing solutions that drive insights and provide value to our clients....

  • Spark/Scala Developer

    2 days ago

    Chennai, Tamil Nadu, India HARP Technologies and Services Full time

    Notice Period : Immediate to 30 DaysJob Summary :We are looking for experienced Spark Scala Developers to join our growing team. The ideal candidate will have hands-on expertise in big data processing using Apache Spark with Scala, along with proficiency in workflow management tools like Oozie and data flow platforms such as Apache NiFi.Key Responsibilities...

  • Scala Spark Developer

    12 hours ago

    Chennai, Pune, India Cloudxtreme Full time US$ 90,000 - US$ 1,20,000 per year

    Role : Scala Spark DeveloperLocation : Pune/ChennaiCompany : MNCRequired Skills & Qualifications:Minimum 5 years of professional experience in Data Engineering, with a strong focus on big data technologies.Proficiency in Scala for developing big data applications and transformations, especially with Apache Spark.Expert-level proficiency in SQL; ability to...

  • Spark/scala

    4 days ago

    Pune, India Codvo.ai Full time

    **Spark /Scala** **Company Overview**: At Codvo, software and people transformations go hand-in-hand. We are a global empathy-led technology services company. Product innovation and mature software engineering are part of our core DNA. Respect, Fairness, Growth, Agility, and Inclusiveness are the core values that we aspire to live by each day. - We continue...

  • Spark/Scala Developer

    11 hours ago

    Pune, Maharashtra, India Cyanous Software Private Limited Full time US$ 1,50,000 - US$ 2,00,000 per year

    Detailed JD (Roles And Responsibilities)At least 8+ years of experience and strong knowledge in Scala programming language.Able to write clean, maintainable and efficient Scala code following best practices.Good knowledge on the fundamental Data Structures and their usageAt least 8+ years of experience in designing and developing large scale, distributed...

  • Spark-Scala Position

    13 hours ago

    Bengaluru, Chennai, Pune, India Orcapod Consulting Services Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Position: Spark-ScalaWork location : Bangalore , Chennai, Hyderabad, Pune, Chandigarh.Type: C2H (Convert into Full-Time Employment (FTE) within 90 days)Experience: 8-10 yrs.Skillsets:Mandatory skills - Spark & ScalaDesired skills - Spark Streaming, Hadoop, Hive, SQL, Sqoop, ImpalaJoining: Immediate joiner or 15 to 30 days

  • Spark, Scala Developer

    14 hours ago

    Bengaluru, Chennai, Hyderabad, India SRS Infoway Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Role & responsibilitiesAt least 5+ years of experience and strong knowledge in Scala programming language.Able to write clean, maintainable and efficient Scala code following best practices.Good knowledge on the fundamental Data Structures and their usageAt least 5+ years of experience in designing and developing large scale, distributed data processing...

  • Spark-Scala Professional

    15 hours ago

    Pune, Maharashtra, India IDESLABS PRIVATE LIMITED Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    skills Spark-Scala Experience 6+ Years Location PAN INDIA Job type Contract to Hire Work Model HybridHands on experience of 4-5 years on Spark-Scala combinationAwareness of Hive DBAwareness and experience of working on Hadoop platformGood SQL knowledgeNP

  • Scala Developer

    4 weeks ago

    Chennai, Tamil Nadu, India Techmynd Consulting Full time

    Pyspark, Scala DeveloperLocation : Bangalore, ChennaiExperience : 6- 8 YearsMust-Have :- Experience with Scala and Spark & its architecture.- Good SQL Knowledge.Job Description :We are seeking a highly skilled Scala & Spark Developer with strong expertise in distributed data processing and big data technologies. The ideal candidate will have hands-on...