Big Data Developer

2 weeks ago

tamil nadu, India HCLTech Full time

Position: Senior Data Engineer – Vendor Experience: 6–12 YearsRole SummaryWe are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires strong hands-on development skills in data engineering, with the ability to deliver high-quality pipelines under guidance from internal leads.Key ResponsibilitiesDevelop and optimize data pipelines using Spark 3.5 and Python/Scala.Migrate existing Hive, Spark, and Control-M jobs to Airflow and DBT-based workflows.Integrate data pipelines with messaging systems (Kafka, Solace) and object stores (S3, MinIO).Troubleshoot and optimize distributed jobs running in Kubernetes environments.Collaborate closely with internal leads and architects to implement best practices.Design and implement migration/acceleration framework to automate end to end migration.Continuous enhancements to the frameworks to ensure the stability, scalability and support for diverse use cases and scenarios.Work with various data applications to enable and support the migration process.Deliver assigned migration tasks within agreed timelines.Required Skills6–9 years of hands-on data engineering experience.Strong expertise in Apache Spark (batch + streaming) and Hive.Proficiency in Python, Scala, or Java.Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).Experience working with Kafka, Solace, and object stores (S3, MinIO).Exposure to Docker/Kubernetes for deployment.Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi

  • Big Data Architect

    4 days ago

    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    **Responsibilities**: - Design and architect Big Data solutions to meet business requirements. - Collaborate with stakeholders to understand their data and analytics needs. - Create and maintain data models, data flow diagrams, and data integration strategies. - Develop and implement data ingestion, processing, and storage solutions using Big Data...

  • Big Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India CIEL HR Full time

    Job Responsibilities Responsible for administering big data systems like EMR HDFS YARN Spark Hive Oozie and others Work closely with development teams at the design phase of big data jobs and pipelines to improve operational efficiency Troubleshoot big data systems for capacity bottlenecks of memory CPU OS storage and network Performance tuning of Hadoop...

  • Big Data

    3 weeks ago

    tamil nadu, India Tata Consultancy Services Full time

    Greetings from TATA CONSULTANCY SERVICES! WALK-IN-DRIVE "Big Data" Interview Date: 8th November 2025 Years of Experience: 5 - 10 years Location: Chennai Job Description:- Hands on experience in installation, configuration, supporting and managing Hadoop clusters using Hortonworks distribution Hands on experience on clusters migration /Azure Cloud Upgrading...

  • Big Data

    2 weeks ago

    tamil nadu, India Tata Consultancy Services Full time

    Greetings from TATA CONSULTANCY SERVICES! Skill: Big Data Years of Experience: 4 - 10 years Location: Chennai/ Banglore Job Description:- Hands on experience in installation, configuration, supporting and managing Hadoop clusters using Hortonworks distribution Hands on experience on clusters migration /Azure Cloud Upgrading the clusters and configured the...

  • Big Data

    2 weeks ago

    tamil nadu, India Tata Consultancy Services Full time

    Greetings from TATA CONSULTANCY SERVICES!Skill: Big DataYears of Experience: 4 - 10 yearsLocation: Chennai/ BangloreJob Description:-Hands on experience in installation, configuration, supporting and managing Hadoop clusters using Hortonworks distributionHands on experience on clusters migration /Azure CloudUpgrading the clusters and configured the Kerberos...

  • Data Architect- Aws-big Data

    3 days ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect - AWS-Big Data **Purpose of the Position**: As a Data Architect, you will be responsible for designing and optimizing complex Big Data solutions using AWS services, ensuring a balance between cost and performance. Responsibilities include assessing existing technology, performing impact assessments, optimizing designs, and...

  • Big Data Developer

    2 weeks ago

    tamil nadu, India Tata Consultancy Services Full time

    Role: Lead/ArchitectRequired Technical Skill Set: Bigdata, Hadoop or Data Warehousing Tools and Cloud ComputingDesired Experience Range: 4 to10 YearsLocation: Chennai and BengaluruMust-Have:Working experience of Hadoop, Hive SQL’s, Spark, Bigdata Eco System Tools.Should be able to tweak queries and work on performance enhancement.The candidate will be...

  • Big Data Developer

    2 weeks ago

    tamil nadu, India HCLTech Full time

    Position: Senior Data Engineer – Experience: 6–9 YearsRole SummaryWe are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires strong hands-on development skills in data engineering, with the ability to deliver high-quality...

  • Big Data Developer

    1 week ago

    tamil nadu, India HCLTech Full time

    Position: Senior Data Engineer – Experience: 6–9 Years Role Summary We are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires strong hands-on development skills in data engineering, with the ability to deliver high-quality...

  • Digital : Big Data and Hadoop Ecosystems

    2 weeks ago

    tamil nadu, India Tata Consultancy Services Full time

    Greetings from TATA CONSULTANCY SERVICES!Skill: Digital : Big Data and Hadoop EcosystemsYears of Experience: 5 - 10 yearsLocation: Chennai/ BangaloreJob Description:-Hands on experience in installation, configuration, supporting and managing Hadoop clusters using Hortonworks distributionHands on experience on clusters migration /Azure CloudUpgrading the...