Lead Big Data Engineer

3 days ago

Pune, Maharashtra, India Citi Full time ₹ 1,04,000 - ₹ 1,30,878 per year

Role Overview:
We are looking for a Big Data Engineer with expertise in PySpark and Apache Iceberg to design and implement scalable data pipelines. The candidate will work on processing large datasets, optimizing data workflows, and ensuring data quality and reliability.

Key Responsibilities:

  • Design, develop, and maintain big data pipelines using PySpark and Apache Iceberg.
  • Optimize data workflows for performance and scalability.
  • Implement data transformation, cleansing, and enrichment processes.
  • Collaborate with data scientists and analysts to deliver actionable insights.
  • Ensure data quality, consistency, and security across all pipelines.
  • Monitor and troubleshoot data pipeline issues in production.

Required Skills:

  • Strong experience with PySpark and Apache Iceberg.
  • Proficiency in distributed data processing and big data technologies.
  • Knowledge of data modeling and schema design.
  • Familiarity with cloud platforms like AWS, Azure, or GCP.
  • Experience with version control tools like Git.

Qualifications:

  • years of relevant experience in Apps Development or systems analysis role
  • Extensive experience system analysis and in programming of software applications
  • Experience in managing and implementing successful projects
  • Subject Matter Expert (SME) in at least one area of Applications Development
  • Ability to adjust priorities quickly as circumstances dictate
  • Demonstrated leadership and project management skills
  • Consistently demonstrates clear and concise written and verbal communication

Education:

  • Bachelor's degree/University degree or equivalent experience
  • Master's degree preferred

This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.

Job Family Group:

Technology

Job Family:

Applications Development

Time Type:

Full time

Most Relevant Skills

Please see the requirements listed above.

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi's EEO Policy Statement and the Know Your Rights poster.

  • Lead Big Data Engineer

    3 days ago

    Pune, Maharashtra, India beBeeDataScientist Full time ₹ 10,00,000 - ₹ 12,00,000

    Big Data Architect PositionWe are seeking a skilled Big Data Architect to lead our team's technical efforts.The ideal candidate will have hands-on experience in designing and developing applications using Big Data, DWH, Cloud technologies, ETL, SQL.Expertise in Spark, Hadoop ecosystem components, and cloud platforms (AWS/Azure/GCP)Good knowledge of Shell...

  • Big Data Engineering Lead

    3 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data Engineering Leader">We are looking for a seasoned Senior Data Engineering Leader to oversee the development and implementation of scalable, distributed data platforms that drive business value.">As a key member of our engineering team, you will be responsible for:">">Designing and architecting high-performance data pipelines that integrate with...

  • Big Data Engineer

    2 days ago

    Pune, Maharashtra, India Merito Full time

    Job Description :We are looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining,...

  • Big Data Engineer

    3 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title:Big Data EngineerAbout the RoleWe're seeking an experienced data engineer to lead the development of large-scale data processing pipelines using Apache Spark and related technologies.The ideal candidate will have a strong background in data warehousing, ETL concepts, and SQL/NoSQL operations.Familiarity with UNIX shell scripting for scheduling and...

  • Big Data Engineer

    6 days ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time

    Big Data Engineer (PySpark) Location: Pune/Nagpur (WFO) Experience: 8 - 12 Years Employment Type: Full-time Job Overview We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...

  • Big Data Engineer

    3 days ago

    Pune, Maharashtra, India WorkiFicient Full time

    To be successful in this role, you should possess : - Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data toolsand frameworks required to solve Big Data problems at scale.- Design and implement...

  • Big Data Engineer

    3 days ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Big Data Engineer (PySpark)Location:Pune/Nagpur (WFO)Experience:8 - 12 YearsEmployment Type:Full-timeJob OverviewWe are looking for an experiencedBig Data Engineerwith strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance...

  • Senior Big Data Engineer

    5 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 30,00,000

    Our company is seeking a highly skilled Data Architect to design, develop and implement scalable data solutions using cutting-edge technologies.Key Responsibilities:Data Ingestion and Processing: Develop and maintain high-quality code, adhering to industry standards and best practices, for efficient data ingestion from disparate sources onto our Big Data...

  • Big Data Engineering Leadership

    5 days ago

    Pune, Maharashtra, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe are seeking ambitious professionals for the role of Lead Apache Hadoop Engineer in our Technology, Data, and Innovation (TDI) team.Key ResponsibilitiesDevelop robust architectures and designs for big data platforms and applications within the Apache Hadoop ecosystem.Implement and deploy big data platforms and solutions on-premises and in...

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India LION AND ELEPHANTS CONSULTANCY PRIVATE LIMITED Full time

    Job Description : We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data environment at an enterprise scale application. You will work...