Big Data Developer

1 week ago

Chennai, Tamil Nadu, India Grid Dynamics Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Big Data DeveLoper

Responsibilities

  • Design and implement data pipelines for migration from HDFS/Hive to cloud object storage (e.g., S3, Ceph).
  • Optimize Spark (and optionally Flink) jobs for performance and scalability in a Kubernetes environment.
  • Ensure data consistency, schema evolution, and governance with Apache Iceberg or equivalent table formats.
  • Support migration strategy definition by providing technical input and identifying risks.
  • Mentor junior developers and review their code / design decisions.
  • Collaborate with platform engineers, cloud architects, and product stakeholders to align technical implementation with project goals.
  • Troubleshoot complex distributed system issues in data pipelines or storage integration.

Requirements

  • Experience 7 to 15 Years
  • Scala and Python
  • Apache Spark (batch & streaming) – must
  • Deep knowledge of HDFS internals and migration strategies.
  • Experience with Apache Iceberg (or similar table formats like Delta Lake / Apache Hudi) for schema evolution, ACID transactions, and time travel.
  • Running Spark and/or Flink jobs on Kubernetes (e.g., Spark-on-K8s operator, Flink-on-K8s).
  • Experience with distributed blob storages like Ceph or AWS S3 and similar
  • Building ingestion, transformation, and enrichment pipelines for large-scale datasets.
  • Infrastructure-as-Code (Terraform, Helm) for provisioning data infrastructure.
  • Ability to work independently while guiding juniors.

Nice to have

  • Experience with Apache Flink
  • Prior experience in migration projects or large-scale data platform modernization.
  • Apple experience preferred (to enable him/her to get up to speed on our tooling set quickly and more independently)

We offer

  • Opportunity to work on bleeding-edge projects
  • Work with a highly motivated and dedicated team
  • Competitive salary
  • Flexible schedule
  • Benefits package - medical insurance, sports
  • Corporate social events
  • Professional development opportunities
  • Well-equipped office

About us

Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.

  • Big Data Developer

    1 week ago

    Chennai, Tamil Nadu, India Chiselon Technologies Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    HiringWe are hiring for one of our clientPosition - Big Data Engineer Experience - 5+yearsLocation - Chennai, BangaloreWork Mode - HybridNotice Period - Immediate Joiners(Seving Notice Period within 15days only)Job OverviewWe are seeking a skilled Big Data Developer to join our team in a mid-level position. This full-time, hybrid role is based in Chennai...

  • Big Data

    1 week ago

    Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Role: Big data EngineerExperience: 6 to 8 YearsLocation: Chennai/Mumbai/Pune Skills:Big DataPysparkHiveLocationChennaiJob FunctionTECHNOLOGYRoleDeveloperJob Id373773Desired SkillsDigital : PySpark | Big Data | HiveDesired Candidate ProfileQualifications : BACHELOR OF ENGINEERING, BACHELOR OF TECHNOLOGY

  • Big Data Developer

    1 week ago

    Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Description for Big DataRequired DetailsTechnical skill - Big Data (PySpark & Hive)Role DeveloperIT experience 5+ yearsExperience in specific skill 5+ yearsLocation of requirement ChennaiDesired Competencies (Technical/Behavioral Competency)Must-Have • PySparkHiveGood-to-HaveSparkHBaseDQ toolAgile Scrum experienceExposure in data ingestion from...

  • Senior Big Data Developer

    4 days ago

    Chennai, Tamil Nadu, India MOURI Tech Foundation Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Looking for a Senior Big Data Developer to design, build, and optimize data pipelines and analytics solutions. Candidates with prior exposure to the Ericsson Wallet Platform will be given preference.Key ResponsibilitiesDevelop, test, and deploy high-performance data pipelines using Hadoop, Spark, Kudu, and HBase.Implement ETL/ELT workflows and ensure data...

  • Big Data Developer

    2 weeks ago

    Chennai, Tamil Nadu, India RB Consultancy Services Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Title:Developer - Big Data (PySpark, HIVE)Work Location:TCS -Bangalore, KA / Chennai, TN / Hyderabad, TGSkills Required:Big Data (PySpark, HIVE)Experience: 4 - 6 YearsJob Description: Big Data (PySpark, HIVE)Need to work as a developer in Bigdata, Hadoop or Data Warehousing Tools and Cloud Computing.Work on Hadoop, Hive SQL's, Spark, and Bigdata Eco...

  • Big Data Developer

    1 week ago

    Chennai, Tamil Nadu, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Greetings from TCSTCS has been a great pioneer in feeding the fire of young Techies like you. We are a global leader in the technology arena and there's nothing that can stop us from growing together.Your role is of key importance, as it lays down the foundation for the entire project.Position:Bigdata DeveloperJob Location:Bangalore, ChennaiExperience:5+Must...

  • Big Data Engineering Developer

    1 week ago

    Chennai, Tamil Nadu, India Citi Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    The Applications Development Senior Programmer/Lead Analyst is an senior level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming...

  • Big Data Hadoop Developer

    6 hours ago

    Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 4,00,000 - ₹ 6,00,000 per year

    Need to work as a developer in Cloudera Hadoop.Work on Hadoop, Python, PySpark, Hive SQL's, Bigdata Eco System Tools.Experience in working with teams in a complex organization involving multiple reporting lines.The candidate should have strong functional and technical knowledge to deliver what is required and he/she should be well acquainted with Banking...

  • Big Data Developer(Hive, Pyspark)

    2 days ago

    Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    TCS is conductingWeekend Virtual DriveforBig Data Developer(Hive, Pyspark)Years of Experience: 4 to 10 yrsLocation: ChennaiNotice Period: 0- 90 daysDate of Interview: 25th Oct, 2025Mode of Interview: Virtual Drive( Cisco or WebEx)Roles & Responsibilities:Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools.Should be able to...

  • Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data EngineerLocation - Chennai, Gurugram (5days Office)Full timeShift timing 2pm to 11pmKey Responsibilities:· Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks.· Collaborate with cross-functional teams to design and implement data models and data integration solutions.· Create and maintain robust SQL...