Scala Lead Engineer

2 days ago

Chennai, Tamil Nadu, India Techominds Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Job Summary:

Skills: Lead Data Eng with Scala, Spark, Pyspark & Databricks

Job Title: Data Engineer (Scala, Spark, Pyspark, Databricks)

Location: Chennai

Responsibilities:

  • Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform.
  • Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency.
  • Optimize Spark applications for performance and cost-efficiency within the Databricks environment.
  • Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning.
  • Collaborate with data scientists, analysts, and other engineering teams to understand data requirements and deliver solutions.
  • Implement data governance and security best practices within Databricks.
  • Troubleshoot and resolve data-related issues, ensuring data availability and reliability.
  • Stay updated with the latest advancements in Spark, Scala, Databricks, and related big data technologies.

Required Skills and Experience:

  • Proven experience as a Data Engineer with a strong focus on big data technologies.
  • Expertise in Scala programming language for data processing and Spark application development.
  • In-depth knowledge and hands-on experience with Apache Spark, including Spark SQL, Spark Streaming, and Spark Core.
  • Proficiency in using Databricks platform features, including notebooks, jobs, workflows, and Unity Catalog.
  • Experience with Delta Lake and its capabilities for building data lakes.
  • Strong understanding of data warehousing concepts, data modeling, and relational databases.
  • Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services.
  • Experience with version control systems like Git.
  • Excellent problem-solving and analytical skills.
  • Ability to work independently and as part of a team.

Preferred Qualifications (Optional):

  • Experience with other big data technologies like Kafka, Flink, or Hadoop ecosystem components.
  • Knowledge of data visualization tools.
  • Understanding of DevOps principles and CI/CD pipelines for data engineering.
  • Relevant certifications in Spark or Databricks.

  • Scala Developer

    1 day ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Role : Scala DeveloperExp : 4+ year Location : Pune,Chennai, Kolkata, Hyderabad, Gurgoan, Functional Skills: Experience in Credit Risk/Regulatory risk domain Technical Skills: Spark ,PySpark, Python, Hive, Scala, MapReduce, Unix shell scripting Good to Have Skills: Exposure to Machine Learning TechniquesJob Description:4+ Years of experience with...

  • Spark Scala Developer

    5 days ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Dear Candidates,Greetings from TCS!!!!TCS is looking for Spark Scala DeveloperExperience: 6-8 yearsLocation: ChennaiMUST HAVE SKILLS:Data Engineer Design and develop scalable and efficient solutions using Spark, Scala, Python, Airflow.Good code and configuration spark knowledge (at least 1 or 2 profiles should be very good at spark configuration and cluster...

  • Lead Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Prodapt Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    OverviewAs a Lead Data Engineer, you will design, build, and maintain scalable backend and distributed systems that power real-time and data-intensive applications. You will have the autonomy to use your expertise in Java, Scala, or Pythoncrafting robust microservices, APIs, and integrating event-driven and streaming solutions. This role blends hands-on...

  • Lead Engineer

    6 days ago

    Chennai, Tamil Nadu, India Prodapt Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    OverviewAs a Lead Data Engineer, you will design, build, and maintain scalable backend and distributed systems that power real-time and data-intensive applications. You will have the autonomy to use your expertise in Java, Scala, or Python—crafting robust microservices, APIs, and integrating event-driven and streaming solutions. This role blends hands-on...

  • Data Engineer with Spark and Scala

    2 weeks ago

    Chennai, Tamil Nadu, India KRG Technologies Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Roles and ResponsibilitiesDesign, develop, test, deploy, and maintain large-scale data pipelines using Scala programming language.Collaborate with cross-functional teams to gather requirements and deliver high-quality ETL solutions.Develop complex SQL queries to extract insights from various databases (Oracle, Postgres) and transform them into actionable...

  • Lead Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India Vijay Venkatesh ( Proprietor of Cynosure Corporate Solutions ) Full time

    Job DescriptionAbout the RoleWe are seeking a highly skilled and experienced Senior / Lead Data Engineer to design, develop, and maintain scalable, reliable, and efficient data pipelines and ETL solutions. The role requires strong expertise across multi-cloud environments, modern data warehousing platforms, programming languages, and data orchestration...

  • Lead Data Engineer

    6 days ago

    Chennai, Tamil Nadu, India ResourceTree Global Services Pvt Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Company DescriptionResourceTree Global Services Pvt Ltd. (ResourceTree) is a Talent Builder focused on offering careers in various industries, including Information & Technology, Manufacturing, Services, Banking, Agri-tech, Edu-tech, and others. The company collaborates with industry Leaders to fulfill their talent needs effectively. Over the past 7 years,...

  • Lead Data Engineer

    7 hours ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About The RoleWe are seeking a highly skilled and experienced Senior / Lead Data Engineer to design, develop, and maintain scalable, reliable, and efficient data pipelines and ETL solutions. The role requires strong expertise across multi-cloud environments, modern data warehousing platforms, programming languages, and data orchestration tools. You will play...

  • Lead Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time

    About the Role : We are seeking a highly skilled and experienced Senior / Lead Data Engineer to design, develop, and maintain scalable, reliable, and efficient data pipelines and ETL solutions. The role requires strong expertise across multi-cloud environments, modern data warehousing platforms, programming languages, and data orchestration tools. You will...

  • Data Engineering Lead

    2 weeks ago

    Chennai, Tamil Nadu, India Star Health Insurance Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Data Engineering Lead:Design and implement scalable data pipelines using technologies such as Apache Spark, Kafka, and AWS Glue.Develop and maintain ETL processes to ingest, transform, and store structured and unstructured data.Optimize data storage solutions, including data lakes, data warehouses, and relational databases.Collaborate with cross functional...