Big Data Engineer

1 week ago

Chennai, Tamil Nadu, India Citi Full time ₹ 8,00,000 - ₹ 24,00,000 per year

Responsible for designing, developing, and optimizing data processing solutions using a combination of Big Data technologies. Focus on building scalable and efficient data pipelines for handling large datasets and enabling batch & real-time data streaming and processing. Responsibilities:> Develop Spark applications using Scala or Python (Pyspark) for data transformation, aggregation, and analysis.> Develop and maintain Kafka-based data pipelines: This includes designing Kafka Streams, setting up Kafka Clusters, and ensuring efficient data flow.> Create and optimize Spark applications using Scala and PySpark: They leverage these languages to process large datasets and implement data transformations and aggregations.> Integrate Kafka with Spark for real-time processing: They build systems that ingest real-time data from Kafka and process it using Spark Streaming or Structured Streaming.> Collaborate with data teams: This includes data engineers, data scientists, and DevOps, to design and implement data solutions.> Tune and optimize Spark and Kafka clusters: Ensuring high performance, scalability, and efficiency of data processing workflows.> Write clean, functional, and optimized code: Adhering to coding standards and best practices.> Troubleshoot and resolve issues: Identifying and addressing any problems related to Kafka and Spark applications.> Maintain documentation: Creating and maintaining documentation for Kafka configurations, Spark jobs, and other processes.> Stay updated on technology trends: Continuously learning and applying new advancements in functional programming, big data, and related technologies.Proficiency in:Hadoop ecosystem big data tech stack(HDFS, YARN, MapReduce, Hive, Impala).Spark (Scala, Python) for data processing and analysis.Kafka for real-time data ingestion and processing.ETL processes and data ingestion tools Deep hands-on expertise in Pyspark, Scala, Kafka Programming Languages:Scala, Python, or Java for developing Spark applications.SQL for data querying and analysis. Other Skills:Data warehousing concepts.Linux/Unix operating systems.Problem-solving and analytical skills.Version control systems Job Family Group: Technology Job Family:Applications Development Time Type:Full time Most Relevant Skills Please see the requirements listed above. Other Relevant Skills For complementary skills, please see above and/or contact the recruiter. Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.View Citi's EEO Policy Statement and the Know Your Rights poster.

  • Big Data Engineering

    3 days ago

    Chennai, Tamil Nadu, India DataHonDo Pte. Ltd. Full time ₹ 1,20,000 - ₹ 3,60,000 per year

     Location:Chennai Walk-in Date:20th September 2025 Joining:Immediate / Within 30 days Employment Type:Full-time / Contract (Open for both)Job Summary:We are hiring skilled and motivated Big Data Engineers with hands-on experience in SQL,Hadoop ecosystem, Spark, Python, and Kubernetes. The ideal candidates will demonstrate a strong learning...

  • Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data engineerLocation - Chennai, Gurugram (5days Onsite)Full timeImmediate joiner Only (15days Notice period only)Technical Skills:o Proficiency in SQL for complex queries, database design, and optimization.o Strong experience with PySpark for data transformation and processing.o Hands-on experience with Databricks for building and managing...

  • Big Data Lead Engineer

    3 days ago

    Chennai, Tamil Nadu, India Synechron Full time ₹ 1,20,000 - ₹ 3,00,000 per year

    Big Data Lead Engineer - Hadoop - Spark & GCP (BigQuery/DataProc) with Java & Python ExpertiseJob SummarySynechron is seeking an experienced Big Data Lead Engineer to drive the design, development, and management of large-scale data solutionsThis full-time leadership role involves guiding a team of data engineers and architects to deliver scalable,...

  • Big Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India Citi Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    The Data Engineer is accountable for developing high quality data products to support theBank's regulatory requirements and data driven decision making. A Data Engineer will serveas an example to other team members, work closely with customers, and remove orescalate roadblocks. By applying their knowledge of data architecture standards, datawarehousing, data...

  • Big Data Lead Engineer

    4 days ago

    Chennai, Tamil Nadu, India Synechron Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Big Data Lead Engineer — Hadoop, Spark & GCP (BigQuery/DataProc) with Java & Python Expertise Job Summary Synechron is seeking an experienced Big Data Lead Engineer to drive the design, development, and management of large-scale data solutions. This full-time leadership role involves guiding a team of data engineers and architects to deliver scalable,...

  • Big Data Engineer

    7 days ago

    Chennai, Tamil Nadu, India Citi Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Responsible for designing, developing, and optimizing data processing solutions using a combination of Big Data technologies. Focus on building scalable and efficient data pipelines for handling large datasets and enabling batch & real-time data streaming and processing. Responsibilities: Develop Spark applications using Scala or Python (Pyspark) for data...

  • Big Data Engineer

    4 hours ago

    Chennai, Tamil Nadu, India Citi Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Discover your future at CitiWorking at Citi is far more than just a job. A career with us means joining a team of more than 230,000 dedicated people from around the globe. At Citi, you'll have the opportunity to grow your career, give back to your community and make a real impact.Job OverviewResponsible for designing, developing, and optimizing data...

  • Big Data Developer

    3 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu IntraEdge Full time

    Job Title: Senior Big Data Engineer (Python)Experience: 5+ YearsLocations: Chennai, Bangalore, GurugramEmployment Type: Full-TimeNotice Period: Immediate to 30 Days PreferredJob SummaryWe are seeking an experienced Big Data Engineer with strong expertise in Python to design, build, and manage large-scale data pipelines and analytics solutions. The ideal...

  • Senior Big Data Developer

    2 weeks ago

    Chennai, Tamil Nadu, India MOURI Tech Foundation Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Looking for a Senior Big Data Developer to design, build, and optimize data pipelines and analytics solutions. Candidates with prior exposure to the Ericsson Wallet Platform will be given preference.Key ResponsibilitiesDevelop, test, and deploy high-performance data pipelines using Hadoop, Spark, Kudu, and HBase.Implement ETL/ELT workflows and ensure data...

  • Senior Big Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Luxoft Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Project descriptionWe've been engaged by an Investment Banking client in the Corporate, Commercial and Institutional Banking (CCIB) potfolio to work on the projects for managing analytics support for all the key initiatives of all segments/products .ResponsibilitiesChoosing the right technologies for our use cases, deploy and operate.Setting up Data stores...