Senior Data Engineer

6 days ago

Pune, Maharashtra, India Citi Full time US$ 1,50,000 - US$ 2,00,000 per year

Role Overview:
We are seeking a highly skilled and experienced Senior Data Engineer to join our team. The ideal candidate will have strong expertise in Python, PySpark, Apache Iceberg, Flink, and Kafka, with additional exposure to API development and integration. This role involves designing, building, and optimizing scalable data pipelines and systems to support our data-driven initiatives.

Key Responsibilities:

Data Pipeline Development:

  • Design, develop, and maintain scalable and efficient data pipelines using Python and PySpark.
  • Implement real-time and batch data processing workflows leveraging Apache Flink and Kafka.

Data Lakehouse Management:

  • Work with Apache Iceberg to manage large-scale datasets, including schema evolution, partitioning, and time-travel queries.
  • Optimize data storage and retrieval for performance and scalability.

API Development and Integration:

  • Develop and integrate APIs to enable seamless data access and interaction with downstream systems.
  • Ensure APIs are secure, scalable, and optimized for performance.

Data Streaming and Messaging:

  • Build and manage real-time data streaming solutions using Kafka.
  • Implement event-driven architectures to support business use cases.

Data Governance and Quality:

  • Ensure data quality, consistency, and security across all pipelines and systems.
  • Implement monitoring and alerting mechanisms to track data pipeline performance and reliability.

Collaboration and Mentorship:

  • Collaborate with cross-functional teams, including data scientists, analysts, and DevOps engineers.
  • Mentor junior engineers and contribute to best practices in data engineering.
Required Skills:

Programming and Frameworks:

  • Proficiency in Python and PySpark for data processing and pipeline development.
  • Strong understanding of distributed data processing frameworks.

Big Data Technologies:

  • Hands-on experience with Apache Iceberg for managing large-scale datasets.
  • Expertise in Apache Flink for real-time data processing.
  • Proficiency in Kafka for data streaming and messaging.

API Development:

  • Experience in developing and integrating RESTful APIs.
  • Knowledge of API security, scalability, and performance optimization.

Data Engineering:

  • Strong understanding of ETL/ELT processes and data transformation techniques.
  • Familiarity with data lakehouse architectures and best practices.

DevOps and CI/CD:

  • Experience with CI/CD pipelines for deploying and managing data pipelines.
  • Familiarity with containerization tools like Docker and orchestration platforms like Kubernetes.

Preferred Skills:

  • Experience with cloud platforms (AWS, Azure, or GCP) for data engineering solutions.
  • Knowledge of data governance frameworks and compliance requirements.
  • Familiarity with monitoring tools like Prometheus, Grafana, or similar.

Education:

  • Bachelor's degree/University degree or equivalent experience

This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.

Job Family Group:

Technology

Job Family:

Applications Development

Time Type:

Full time

Most Relevant Skills

Please see the requirements listed above.

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi's EEO Policy Statement and the Know Your Rights poster.

  • Senior QE Engineer

    2 weeks ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:330977NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior QE Engineer to join our team in Pune, Mahrshtra (IN-MH), India (IN).Senior Quality Engineer I...

  • Senior QE Engineer

    22 hours ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:330974NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior QE Engineer to join our team in Pune, Mahrshtra (IN-MH), India (IN).Senior Quality Engineer I...

  • Senior Software Engineer

    3 weeks ago

    Pune, Maharashtra, India Data Dynamics Full time

    About the Role : We are seeking a Senior Software Engineer to join our team in Pune. As a key member of our engineering group, you will play a crucial role in the entire software development lifecycle, from initial design to hands-on development. You will work with cross-functional teams to create innovative solutions that are scalable and reliable. This...

  • Senior Data Engineer

    5 days ago

    Pune, Maharashtra, India SPIRO Full time

    Position: Senior Data EngineerLocation: PuneExperience: 5+ yearsJob Summary:We are seeking an experienced Senior Data Engineer with a strong background in PySpark, Spark, and Big Data technologies. The ideal candidate will have over five years of hands-on experience designing, building, and optimizing large-scale data processing systems and pipelines.Key...

  • Senior Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Kivor Full time

    Company DescriptionKivor is revolutionizing enterprise application support with Gen AI-powered autonomous agents. Our platform transforms data from logs, tickets, source code, and documentation into an application-specific knowledge graph, enabling faster issue detection and seamless autonomous resolutions. This innovation reduces support costs by over 50%...

  • Senior Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Kivor Full time

    Company Description Kivor is revolutionizing enterprise application support with Gen AI-powered autonomous agents. Our platform transforms data from logs, tickets, source code, and documentation into an application-specific knowledge graph, enabling faster issue detection and seamless autonomous resolutions. This innovation reduces support costs by over 50%...

  • Senior Data Engineer

    3 weeks ago

    Pune, Maharashtra, India One Click AI Full time

    Senior Data Engineer (Snowflake)Employment Type : Full-Time | PayrollExperience : 5+ YearsAbout Agivant :Agivant is a new-age, AI-first digital and cloud engineering services company. We partner with enterprises to deliver scalable, secure, and high-performance data solutions that power business growth. We are now seeking a Senior Data Engineer to join our...

  • Senior Data Engineer

    5 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team. As a senior member of our engineering team, you will be responsible for designing and implementing scalable data processing pipelines using PySpark.Your primary focus will be on developing high-quality code that integrates, cleanses, and processes large...

  • Senior Data Engineer

    1 week ago

    Pune, Maharashtra, India Expleo Full time US$ 90,000 - US$ 1,20,000 per year

    OverviewSenior Data Engineer Years of experience - 4 to 6 yearsAWS Data Engineer with strong python experience. Helping in defining the data architecture within our Snowflake install. Working with Data Analysts to help them upskill and create a gold-tier data structures with best practices. Help technically with data pipeline buildout within Snowflake....

  • Senior Data Engineer

    6 days ago

    Pune, Maharashtra, India InfoBeans Full time

    Greetings from Infobeans ( Title: Senior Data EngineerSkills: Python, Data EngineeringExperience Required: 5- 7 YearsLocation: PuneNotice Period: Immediate Joiners PreferredAbout the RoleWe are seeking a Senior Data Engineer with a passion for building scalable data infrastructure, fluent Python skills, and a sharp analytical mindset. You will design,...