Lead Data Engineer

1 day ago

Chennai, Tamil Nadu, India Ixceed Solutions Full time

About the Role :

We are seeking a highly skilled and experienced Lead Data Engineer/Scala Developer with strong hands-on expertise in Scala programming, Apache Spark, Databricks, Python, and PySpark. The ideal candidate will lead the design, development, and deployment of scalable big data solutions, working closely with cross-functional teams to deliver high-quality data processing pipelines and analytics platforms.

Key Responsibilities :

- Lead the development and implementation of big data solutions using Scala and Apache Spark on Databricks.

- Design, develop, and optimize Spark jobs and ETL pipelines using Scala, Python, and PySpark.

- Collaborate with data engineers, data scientists, and business stakeholders to understand requirements and translate them into technical solutions.

- Mentor and guide junior developers and team members on best practices in Scala and Spark development.

- Ensure code quality, performance tuning, and scalability of Spark applications.

- Participate in architectural discussions and contribute to technology roadmap decisions.

- Troubleshoot and resolve production issues related to Spark jobs and data pipelines.

- Implement CI/CD pipelines for Spark applications and automate deployment processes.

- Stay updated with the latest trends and advancements in Scala, Spark, and big data technologies.

Required Skills and Qualifications :

- Strong hands-on experience in Scala programming with a deep understanding of functional programming concepts.

- Extensive experience with Apache Spark (RDDs, DataFrames, Datasets) and Databricks platform.

- Proficient in Python and PySpark for big data processing.

- Experience in building and optimizing large-scale ETL pipelines and data workflows.

- Solid understanding of distributed computing and big data ecosystem.

- Familiarity with cloud platforms such as AWS, Azure, or GCP, especially Databricks on cloud.

- Experience with version control systems (Git) and CI/CD tools.

- Strong problem-solving skills and ability to work in an agile environment.

- Excellent communication and leadership skills to manage and mentor a development team.

(ref:hirist.tech)

  • Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time US$ 80,000 - US$ 1,20,000 per year

    Req ID: 336026NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). "Job Duties: Key Responsibilities:...

  • Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India Quantrail Data Full time

    Location: Chennai, India (On-site)Duration: 3 monthsStipend: UnpaidEligibility: 2025 graduates onlyConversion: Opportunity for a full-time role based on performance Note: Please apply only if you are serious about joining us. This is a learning-focused internship, and we expect interns to be committed to upskilling themselves and contributing to real-world...

  • Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India Quantrail Data Full time US$ 60,000 - US$ 1,00,000 per year

    Location: Chennai, India (On-site)Duration: 3 monthsStipend: UnpaidEligibility:2025 graduates onlyConversion: Opportunity for a full-time role based on performance Note: Please apply only if you are serious about joining us. This is a learning-focused internship, and we expect interns to be committed to upskilling themselves and contributing to real-world...

  • Data Engineer

    17 hours ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    HiRole - Azure Data engineerLocation - Chennai, Gurugram (5days Onsite)Full timeImmediate joiner Only (15days Notice period only)Technical Skills:o Proficiency in SQL for complex queries, database design, and optimization.o Strong experience with PySpark for data transformation and processing.o Hands-on experience with Databricks for building and managing...

  • Eucloid - Senior/Lead Data Engineer - Python

    3 weeks ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Pvt Ltd Full time

    Job Description : Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...

  • Career Opportunity

    6 days ago

    Chennai, Tamil Nadu, India beBeeData Engineer Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Summary">This role is responsible for participating as an individual contributor in project teams, troubleshooting operational issues, and providing technical solutions to operational problems.The ideal candidate will have experience with Python, SQL, and Bash Scripting, as well as knowledge of data pipeline fundamentals and general understanding of...

  • Lead Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India Ignitho Full time US$ 90,000 - US$ 1,20,000 per year

    Position: Lead Data EngineerLocation: ChennaiWork Hours: 12:00 PM to 9:00 PM IST (US Business Hours)Availability: ImmediateExperience: 6 + YearsAbout the Company:Ignitho Inc. is a leading AI and data engineering company with a global presence, including offices in the US, UK, India, and Costa Rica.Visit our website to learn more about our work and culture:...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key Responsibilities:•  ...

  • Data Engineer Lead

    1 day ago

    Chennai, Tamil Nadu, India beBeeEngineering Full time US$ 1,80,000 - US$ 2,50,000

    Lead Data EngineerWe are seeking an accomplished and seasoned Data Engineering professional to lead our data team in delivering impactful solutions within the healthcare and medical technology sectors.This role involves driving the development of scalable pipelines using Databricks, Azure Data Factory, and ETL tools for efficient data ingestion,...

  • data engineering lead

    1 week ago

    Chennai, Tamil Nadu, India Randstad Full time

    At least 5 years' experience as a Senior or Lead Data Engineer Proven and demonstratable experience of Python, AWS, and SQL/ Oracle Bachelor's honours degree preferably in computer science, or engineering field, or demonstrable technical experience in a related engineering discipline At least 3 years of ETL experience using flat files to transposed...