Big Data Engineer

2 days ago

Chennai, Tamil Nadu, India Citi Full time ₹ 12,00,000 - ₹ 36,00,000 per year
Discover your future at Citi

Working at Citi is far more than just a job. A career with us means joining a team of more than 230,000 dedicated people from around the globe. At Citi, you'll have the opportunity to grow your career, give back to your community and make a real impact.

Job Overview

Responsible for designing, developing, and optimizing data processing solutions using a combination of Big Data technologies. Focus on building scalable and efficient data pipelines for handling large datasets and enabling batch & real-time data streaming and processing.

Responsibilities:


Develop Spark applications using Scala or Python (Pyspark) for data transformation, aggregation, and analysis.

Develop and maintain Kafka-based data pipelines: This includes designing Kafka Streams, setting up Kafka Clusters, and ensuring efficient data flow.

Create and optimize Spark applications using Scala and PySpark: They leverage these languages to process large datasets and implement data transformations and aggregations.

Integrate Kafka with Spark for real-time processing: They build systems that ingest real-time data from Kafka and process it using Spark Streaming or Structured Streaming.

Collaborate with data teams: This includes data engineers, data scientists, and DevOps, to design and implement data solutions.

Tune and optimize Spark and Kafka clusters: Ensuring high performance, scalability, and efficiency of data processing workflows.

Write clean, functional, and optimized code: Adhering to coding standards and best practices.

Troubleshoot and resolve issues: Identifying and addressing any problems related to Kafka and Spark applications.

Maintain documentation: Creating and maintaining documentation for Kafka configurations, Spark jobs, and other processes.

Stay updated on technology trends: Continuously learning and applying new advancements in functional programming, big data, and related technologies.


Proficiency in:

Hadoop ecosystem big data tech stack(HDFS, YARN, MapReduce, Hive, Impala).

Spark (Scala, Python) for data processing and analysis.

Kafka for real-time data ingestion and processing.

ETL processes and data ingestion tools

Deep hands-on expertise in Pyspark, Scala, Kafka

Programming Languages:

Scala, Python, or Java for developing Spark applications.

SQL for data querying and analysis.

Other Skills:

Data warehousing concepts.

Linux/Unix operating systems.

Problem-solving and analytical skills.

Version control systems

-

Job Family Group:

Technology

-

Job Family:

Applications Development

-

Time Type:

Full time

-

Most Relevant Skills

Please see the requirements listed above.

-

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

-

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi's EEO Policy Statement and the Know Your Rights poster.

  • Big Data Lead Engineer

    4 days ago

    Chennai, Tamil Nadu, India Synechron Full time ₹ 1,20,000 - ₹ 3,00,000 per year

    Big Data Lead Engineer - Hadoop - Spark & GCP (BigQuery/DataProc) with Java & Python ExpertiseJob SummarySynechron is seeking an experienced Big Data Lead Engineer to drive the design, development, and management of large-scale data solutionsThis full-time leadership role involves guiding a team of data engineers and architects to deliver scalable,...

  • Python & Big Data Engineering

    2 weeks ago

    Chennai, Tamil Nadu, India Quantaleap Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Immediate opening for "Python & Big Data Engineering"Job Title: Python & Big Data EngineerJob Location: Chennai (Hybrid)Experience Required: 4 -9 YearsJob Requirement:Strong proficiency in Python with OOP and functional programming experience.Experience building multi-module, package-structured applications.Expertise in unit testing using pytest /...

  • Senior Big Data Developer

    2 weeks ago

    Chennai, Tamil Nadu, India MOURI Tech Foundation Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Looking for a Senior Big Data Developer to design, build, and optimize data pipelines and analytics solutions. Candidates with prior exposure to the Ericsson Wallet Platform will be given preference.Key ResponsibilitiesDevelop, test, and deploy high-performance data pipelines using Hadoop, Spark, Kudu, and HBase.Implement ETL/ELT workflows and ensure data...

  • Big Data Hadoop Developer

    7 days ago

    Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 4,00,000 - ₹ 6,00,000 per year

    Need to work as a developer in Cloudera Hadoop.Work on Hadoop, Python, PySpark, Hive SQL's, Bigdata Eco System Tools.Experience in working with teams in a complex organization involving multiple reporting lines.The candidate should have strong functional and technical knowledge to deliver what is required and he/she should be well acquainted with Banking...

  • Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India CG-VAK Software & Exports Ltd. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title:Data Engineer (Scala, Spark, Pyspark, Databricks)Remark - final round will be the F2FExperience: 10+ Years (Lead Role)ResponsibilitiesDesign, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform.Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data...

  • Data Engineer

    4 days ago

    Chennai, Tamil Nadu, India KaarTech Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Key Responsibilities· Data Pipeline Development: Design, develop, and maintain scalable data pipelines to support various data analytics and reporting needs.· Data Integration: Integrate data from multiple sources and ensure data quality and consistency.· Data Modelling: Build and optimize data models using Spark to support complex data transformations...

  • Data Entry

    2 days ago

    Chennai, Tamil Nadu, India Big Data Imaging Services Full time ₹ 1,20,000 - ₹ 1,80,000 per year

    Urgent Requirement Data Entry & Scanning ExecutiveMale & Female Candidates can apply.Qualification : +2 / ITI & above.Age : Below 30 yrs.Skills : MS Excel, Scanning, Typing.Duties : Data entry & Scanning.Salary : 15,000 Per Month.Job Openings : 25 vacancies AvailableMaples Capturing Technology Private LimitedContact Details : Address : No.695-B, 1st Street,...

  • Big Data Developer(Hive, Pyspark)

    1 week ago

    Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    TCS is conductingWeekend Virtual DriveforBig Data Developer(Hive, Pyspark)Years of Experience: 4 to 10 yrsLocation: ChennaiNotice Period: 0- 90 daysDate of Interview: 25th Oct, 2025Mode of Interview: Virtual Drive( Cisco or WebEx)Roles & Responsibilities:Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools.Should be able to...

  • Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India Randomtrees Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Were Hiring: Data Engineers Freshers (2024 & 2025 pass-outs)Company: Random TreesLocation: ChennaiWork Mode: Work from OfficeExperience: Freshers / Entry-Level (Internship experience will be counted)Job Description for Data Engineer -Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineer, Data Analytics or related fields...

  • Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India YO HR Consultancy Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Data Engineer/Senior Data EngineerLocation: ChennaiExperience: 2 - 8 YearsRole DescriptionThe Data Engineer will be responsible for designing and developing data engineering solutions, creating and maintaining data models, building Extract Transform Load (ETL) pipelines, implementing data warehousing, and performing data analytics on large...