Apache Spark Senior Technical Lead

3 days ago

Chennai, Tamil Nadu, India Sopra Steria Full time US$ 1,50,000 - US$ 2,00,000 per year

Company Description

About Sopra Steria

Sopra Steria, a major Tech player in Europe with 50,000 employees in nearly 30 countries, is recognised for its consulting, digital services and solutions. It helps its clients drive their digital transformation and obtain tangible and sustainable benefits. The Group provides end-to-end solutions to make large companies and organisations more competitive by combining in-depth knowledge of a wide range of business sectors and innovative technologies with a collaborative approach. Sopra Steria places people at the heart of everything it does and is committed to putting digital to work for its clients in order to build a positive future for all. In 2024, the Group generated revenues of €5.8 billion.

The world is how we shape it.

Job Description

Job Title: Technical Lead/Senior Technical Lead

Location: Chennai

Role Overview:

We are looking for a skilled Data Engineer to build and maintain scalable data pipelines, manage large datasets, and support analytics initiatives. This role involves working with Big Data technologies like Spark, Scala, SQL/PLSQL, and cloud-ready environments.

Experience:

  • 8+ years of relevant experience in data engineering / big data development
  • Proven track record of handling large-scale data solutions

Key Responsibilities:

  • Maintain, develop, and design data ingestion and transformation pipelines using Spark and Scala
  • Process and optimize SQL and PL/SQL queries for large datasets
  • Integrate and process data from multiple sources for analytics and reporting
  • Collaborate with BI and Analytics teams to support reporting and dashboard needs
  • Ensure data quality, performance, and security
  • Provide innovative solutions to complex business/technical problems
  • Identify and resolve technical, integration and development issues
  • Implement best practices, standards and processes to ensure quality of the final product
  • Responsible for development of features independently
  • Continuous build, automated testing, and release management
  • Participate in status meetings and provide regular updates

Required Skills:

Technical:

  • Expertise in Spark and Scala for large-scale data processing
  • Proficiency in Python and SQL, including hands-on experience with PL/SQL
  • Experience in data integration/ETL workflows;
  • Strong knowledge of relational databases such as Oracle and PostgreSQL

Professional Competencies:

  • Excellent verbal and written communication skills
  • Result-oriented with strong analytical and problem-solving skills
  • Ability to work in a fast-paced, agile, and collaborative environment

Desired Skills:

  • Exposure to cloud data services (AWS preferred)
  • Proficiency in Linux/Unix shell scripting for automation and data operation
  • Experience working in Agile environments – Scrum and/or SAFe frameworks
  • Exposure to DevOps culture and practices, including CI/CD and automation
  • Understanding of software engineering lifecycle – Development, Testing, and Production Support
  • Experience in production issue handling and transformation support management

Good to Have Skills:

  • Exposure to Dataiku preferred
  • Awareness of cloud and big data architecture principles

Total Experience Expected: 08-10 years

Qualifications

BE / B.Tech/ MS/ M.Tech/ MCA

Additional Information

At our organization, we are committed to fighting against all forms of discrimination. We foster a work environment that is inclusive and respectful of all differences.

All of our positions are open to people with disabilities.

  • Senior Apache Spark Scala Developer

    1 day ago

    Chennai, Tamil Nadu, India beBeeDevelopment Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Apache Spark Scala DeveloperJob Description:We are seeking a skilled and experienced Senior Apache Spark Scala developer to join our team. The successful candidate will have expertise in designing, developing, and deploying scalable data pipelines using Apache Spark Core, SQL, Streaming, MLlib, etc., with Scala.Key Responsibilities:Design and develop...

  • Apache Spark Senior Technical Lead

    1 week ago

    Chennai, Tamil Nadu, India Sopra Steria Full time US$ 1,50,000 - US$ 2,00,000 per year

    Company Description About Sopra SteriaSopra Steria, a major Tech player in Europe with 50,000 employees in nearly 30 countries, is recognised for its consulting, digital services and solutions. It helps its clients drive their digital transformation and obtain tangible and sustainable benefits. The Group provides end-to-end solutions to make large companies...

  • Highly Skilled Apache Spark Developer Wanted

    4 days ago

    Chennai, Tamil Nadu, India beBeeBigData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Opportunity:This is an exciting opportunity to work as a skilled developer with expertise in Apache Spark and Scala programming language.You will be responsible for designing and developing large-scale distributed data processing pipelines using Apache Spark and related technologies.The ideal candidate will have strong knowledge of Spark Core, Spark SQL,...

  • Spark/Scala Developer

    4 days ago

    Chennai, Tamil Nadu, India HARP Technologies and Services Full time

    Notice Period : Immediate to 30 DaysJob Summary :We are looking for experienced Spark Scala Developers to join our growing team. The ideal candidate will have hands-on expertise in big data processing using Apache Spark with Scala, along with proficiency in workflow management tools like Oozie and data flow platforms such as Apache NiFi.Key Responsibilities...

  • Highly Experienced Java and Spark Developer Wanted

    4 days ago

    Chennai, Tamil Nadu, India beBeeJavaDeveloper Full time US$ 1,00,000 - US$ 1,50,000

    Job DescriptionAs a Senior Java/Spark Developer, you will be responsible for designing and developing large-scale, distributed data processing pipelines using Apache Spark and related technologies. You will work with our team to build robust, scalable, and maintainable applications using Java and Spark.",

  • Big Data Engineering Lead

    2 days ago

    Chennai, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Big Data EngineerWe are seeking a highly skilled Senior Big Data Engineer to lead our data engineering team and drive the development of scalable, secure, and high-performance big data solutions.Experience in leading distributed teams across multiple geographies and time zones is essential.Strong technical expertise in Scala, Python, Apache Spark...

  • Spark Scala, Aws Kolkata&hyderabad

    1 day ago

    Chennai, Tamil Nadu, India Diverse Lynx Full time

    Hiring for Spark Scala AWS KOLKATA HYDERABAD Design and develop robust scalable data pipelines using Apache Spark Core SQL Streaming MLlib etc with Scala Write clean modular production-grade Scala code adhering to coding standards and best practices Optimize Spark jobs for performance reliability and cost-efficiency including tuning Spark...

  • Spark Developer

    3 days ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 18,12,500 - ₹ 23,40,000

    Senior Data Processing SpecialistElevate your career in data engineering and drive business growth as a Senior Data Processing Specialist. This exciting role offers the opportunity to design, implement, and optimize distributed data processing jobs to handle large-scale data in Hadoop Distributed File System (HDFS) using Apache Spark and Python.This position...

  • Spark Scala Developer

    1 week ago

    Chennai, Tamil Nadu, India Infotel UK Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Position: Spark Scala DeveloperCompany: Infotel IndiaAt Infotel UK, a premier technology consulting firm, we are seeking a skilled Spark Scala Developer to join our dynamic team. In this role, you will leverage your expertise in Spark and Scala to design and implement advanced data processing solutions that drive insights and provide value to our clients....

  • Technical Lead

    1 week ago

    Chennai, Tamil Nadu, India Antal International Full time

    Summary role description Company description Role details Title Designation Lead Software Engineer Java - Banking Reporting Manager Senior Director Location Chennai Work Mode Hybrid 3 days office 2 days work-from-home Role responsibilities Client Interaction Requirements Gathering Collaborating with clients to translate business...