Big Data Developer

5 hours ago

Chennai Tamil Nadu India, Tamil Nadu HCLTech Full time

Position: Senior Data Engineer – Vendor Experience: 6–12 Years


Role Summary

We are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires strong hands-on development skills in data engineering, with the ability to deliver high-quality pipelines under guidance from internal leads.


Key Responsibilities


Develop and optimize data pipelines using Spark 3.5 and Python/Scala.

Migrate existing Hive, Spark, and Control-M jobs to Airflow and DBT-based workflows.

Integrate data pipelines with messaging systems (Kafka, Solace) and object stores (S3, MinIO).

Troubleshoot and optimize distributed jobs running in Kubernetes environments.

Collaborate closely with internal leads and architects to implement best practices.

Design and implement migration/acceleration framework to automate end to end migration.

Continuous enhancements to the frameworks to ensure the stability, scalability and support for diverse use cases and scenarios.

Work with various data applications to enable and support the migration process.

Deliver assigned migration tasks within agreed timelines.


Required Skills

6–9 years of hands-on data engineering experience.

Strong expertise in Apache Spark (batch + streaming) and Hive.

Proficiency in Python, Scala, or Java.

Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).

Experience working with Kafka, Solace, and object stores (S3, MinIO).

Exposure to Docker/Kubernetes for deployment.

Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi

  • Big Data Developer

    5 hours ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Big Data Developer - 4 - 10 YearsMust-Have Experience in Web development using React, NodeJs/NestJS Good-to-Have Knowledge in Mongo DB.Skills: HTML, CSS, JavaScript, React, NodeJs, UI/UX experience

  • Big Data Developer

    5 hours ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    TCS is Hiring Big Data Developer For Chennai and Bangalore locationExperience : 4 - 10 yearsMode of interview: VirtualLocation : Chennai, BangaloreJOB DESCRIPTION:Must have:Hadoop,Spark, Pyspark, Hive, ScalaGood to have:Agile and good communication skillsResponsibilities:Ingest data from disparate sources (Structured, unstructured and semi-structured) and...

  • Big Data Developer

    3 days ago

    Chennai, Tamil Nadu, India Chiselon Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    HiringWe are hiring for one of our clientPosition - Big Data Engineer Experience - 5+yearsLocation - Chennai, BangaloreWork Mode - HybridNotice Period - Immediate Joiners(Seving Notice Period within 15days only)Job OverviewWe are seeking a skilled Big Data Developer to join our team in a mid-level position. This full-time, hybrid role is based in Chennai...

  • Big Data Developer

    3 days ago

    Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Description for Big DataRequired DetailsTechnical skill - Big Data (PySpark & Hive)Role DeveloperIT experience 5+ yearsExperience in specific skill 5+ yearsLocation of requirement ChennaiDesired Competencies (Technical/Behavioral Competency)Must-Have • PySparkHiveGood-to-HaveSparkHBaseDQ toolAgile Scrum experienceExposure in data ingestion from...

  • Big Data

    3 days ago

    Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Role: Big data EngineerExperience: 6 to 8 YearsLocation: Chennai/Mumbai/Pune Skills:Big DataPysparkHiveLocationChennaiJob FunctionTECHNOLOGYRoleDeveloperJob Id373773Desired SkillsDigital : PySpark | Big Data | HiveDesired Candidate ProfileQualifications : BACHELOR OF ENGINEERING, BACHELOR OF TECHNOLOGY

  • Big Data Developer

    6 days ago

    Chennai, Tamil Nadu, India RB Consultancy Services Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Title:Developer - Big Data (PySpark, HIVE)Work Location:TCS -Bangalore, KA / Chennai, TN / Hyderabad, TGSkills Required:Big Data (PySpark, HIVE)Experience: 4 - 6 YearsJob Description: Big Data (PySpark, HIVE)Need to work as a developer in Bigdata, Hadoop or Data Warehousing Tools and Cloud Computing.Work on Hadoop, Hive SQL's, Spark, and Bigdata Eco...

  • Big Data Developer

    6 days ago

    Chennai, Tamil Nadu, India Natobotics Full time ₹ 12,00,000 - ₹ 18,00,000 per year

    Required Details Total ExperienceRelevant ExperienceCurrent Company:Current Designation:Current CTC LPAMonthly Take home:Expected CTC LPA:Expected Take home:Notice Period:Current LocationExpected Location:Home Pincode:Offer In hand:PAN Number (upload profiles to the portal):DOB (upload profiles to the portal):Reason for Job Change:UG...

  • Big Data Developer

    5 hours ago

    Chennai, Tamil Nadu, India, Tamil Nadu HCLTech Full time

    Position: Senior Data Engineer – Experience: 6–9 YearsRole SummaryWe are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires strong hands-on development skills in data engineering, with the ability to deliver high-quality...

  • Big Data Engineering Developer

    6 days ago

    Chennai, Tamil Nadu, India Citi Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    The Applications Development Senior Programmer/Lead Analyst is an senior level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming...

  • Big Data Lead Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Synechron Full time ₹ 1,20,000 - ₹ 3,00,000 per year

    Big Data Lead Engineer - Hadoop - Spark & GCP (BigQuery/DataProc) with Java & Python ExpertiseJob SummarySynechron is seeking an experienced Big Data Lead Engineer to drive the design, development, and management of large-scale data solutionsThis full-time leadership role involves guiding a team of data engineers and architects to deliver scalable,...