Bigdata Platform Lead Engineer
7 days ago
The Lead Data Analytics analyst is responsible for managing, maintaining, and optimizing on-premises and cloud-based big data infrastructure. The individual should have expertise in administering big data platforms, ensuring high availability, performance, and security of data systems, and supporting data engineering and analytics teams. This role requires hands-on experience with both on-premises big data technologies (e.g., Hadoop, Spark) and cloud-based platforms (e.g., AWS, Azure, GCP).
Responsibilities:
- Lead day to day operation and support for Cloudera Hadoop ecosystem components (HDFS, YARN, Hive, Impala, Spark, HBase, etc)
- Troubleshoot issues related to data ingestion, job failures, performance degradation and service unavailability.
- Monitor cluster health using Cloudera Manager and respond to alerts, logs, and metrics.
- Collaborate with engineering teams to analyze root causes and implement preventive measures.
- Collaborate patching, service restarts, failovers and rolling restarts for cluster maintenance.
- Assist in user onboarding, access control and issues in accessing the cluster services.
- Contribute to documentation for knowledge base.
- Work on data recovery, replication, and backup support tasks.
- Responsible for moving all legacy workloads to cloud platform
- Ability to research and assess open-source technologies, public cloud tech stack (AWS/GCP) components to recommend and integrate into the design and implementation
- Be the technical expert and mentor other team members on Big Data and Cloud Tech stacks
- Define needs around maintainability, testability, performance, security, quality and usability for data platform
- Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes
- Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance
- Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.
- Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function.
- Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.
- Supervise day-to-day staff management issues, including resource management, work allocation, mentoring/coaching and other duties and functions as assigned
- Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
Qualifications:
- 10+ years of total IT experience
- 5+ years of experience in supporting Hadoop (Cloudera)/big data technologies
- 5+ years of experience in public cloud infrastructure (AWS or GCP)
- Experience with Kubernetes and cloud-native technologies.
- Experience with all aspects of DevOps (source control, continuous integration, deployments, etc.)
- Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr).
- Knowledge of troubleshooting techniques for Hive, Spark, YARN, Kafka, and HDFS.
- Advanced Linux system administration and scripting skills (Shell, Python)
- Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Spark with Java or Scala or Python
- Expert level building pipelines using Apache Spark Familiarity with core provider services from AWS, Azure or GCP, preferably having supported deployments on one or more of these platforms
- Hands-on experience with Python/Pyspark/Scala is required.
- System level understanding - Data structures, algorithms, distributed storage & compute
- Can-do attitude on solving complex business problems, good interpersonal and teamwork skills
- Possess team management experience and have led a team of data/platform engineers and analysts.
- Experience in Snowflake or Delta lake is a plus.
Education:
- Bachelor's degree/University degree or equivalent experience
Job Family Group:
Technology
Job Family:
Data Science
Time Type:
Full time
Most Relevant Skills
Please see the requirements listed above.
Other Relevant Skills
For complementary skills, please see above and/or contact the recruiter.
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review
Accessibility at Citi.
View Citi's EEO Policy Statement and the Know Your Rights poster.
-
Bigdata an Cloudera Engineer
2 weeks ago
Bengaluru, Karnataka, India Fractal Full time ₹ 15,00,000 - ₹ 25,00,000 per yearAbout the Company:Fractal is a leading AI and analytics company that helps businesses leverage data to drive growth and innovation. Our mission is to empower organizations with advanced analytics and AI solutions, fostering a culture of collaboration and continuous improvement.About the Role:We are seeking highly skilled Big Data Engineers/Senior Engineers...
-
Bigdata + Cloudera Engineer
7 days ago
Bengaluru, Karnataka, India Fractal Full time ₹ 20,00,000 - ₹ 25,00,000 per yearAbout the Company:Fractal is a leading AI and analytics company that helps businesses leverage data to drive growth and innovation. Our mission is to empower organizations with advanced analytics and AI solutions, fostering a culture of collaboration and continuous improvement.About the Role:We are seeking highly skilled Big Data Engineers/Senior Engineers...
-
AWS BigData
2 weeks ago
Bengaluru, Karnataka, India Elfonze Technologies Full time ₹ 12,00,000 - ₹ 24,00,000 per yearJob InformationDate Opened10/13/2025Job TypeFull timeIndustryIT ServicesCityBangalore NorthState/ProvinceKarnatakaCountryIndiaZip/Postal Code560001Job DescriptionAs a Software Development Lead, you will be responsible for developing and configuring software systems, either end-to-end or for specific stages of the product lifecycle.Your typical day will...
-
Lead Platform Engineer
2 weeks ago
Bengaluru, Karnataka, India Reqres Full time ₹ 12,00,000 - ₹ 36,00,000 per yearDescription : Urgent Hiring Lead Platform engineer || Bangalore Profile - Lead Platform engineer Experience - 8years Location : Bangalore Role Overview : We are seeking a seasoned Lead Platform Engineer with a strong background in platform development and a proven track record of leading technology design and teams. The ideal candidate...
-
Lead Platform Engineer
2 days ago
Bengaluru, Karnataka, India Torry Harris Integration Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per yearJob Title: Lead Platform EngineerJob DescriptionWe are seeking a Lead Platform Engineer to oversee the design, development, and implementation of our platform infrastructure. The ideal candidate will be responsible for enhancing the scalability, reliability, and efficiency of our services and applications. You will collaborate closely with cross-functional...
-
Bigdata Developer
1 day ago
Bengaluru, Karnataka, India IQVIA Full time ₹ 15,00,000 - ₹ 25,00,000 per yearBigdata Developer Job Overview Modifies applications programs from detailed specifications. Codes, tests, debugs, documents, and maintains those programs. Essential Functions • Conducts and coordinates the analysis, design and implementation of computer based information systems to meet user requirements. • Plans and directs preliminary...
-
Lead Software Engineer
2 days ago
Bengaluru, Karnataka, India Convate Consultancy Services Full time ₹ 20,00,000 - ₹ 25,00,000 per yearDear Candidate,We have a Job Opening for Lead Software Engineer - Bigdata in Banking Based Product Company at Bangalore.Requirement Details:Location: BangaloreDesignation: Specialist Software Engineer - GenesysExperience: 8 to 10 YearsExpected Notice Period: ImmediateProfile Required:At least 3 to 7 Years of experience on big data platform and at least 2...
-
Lead Platform Automation Engineer
3 days ago
Bengaluru, Karnataka, India InfoVision Inc Full time ₹ 12,00,000 - ₹ 36,00,000 per yearSummary:Lead Platform Automation Engineer will be responsible for developing, implementing, and maintaining test automation frameworks. You will contribute to the design of test data management practices and develop tools to support test infrastructure. This role requires working closely with engineering teams and help create scalable, high-performance...
-
Lead Data Engineer
7 days ago
Bengaluru, Karnataka, India Nam Info Full time ₹ 15,00,000 - ₹ 25,00,000 per yearDear,Hope you are doing well.Please find the Job description. Kindly share the resume with expected salary and send it to -Confirm your availability for IN- PERSON or Face-to-Face interview availability as on 1st Oct at Madivala, Bengaluru. YES OR NO?Candidates need to Join 15-20 days max notice. YES OR NO?LOCALS TO BENGALURU ONLY.Role : Lead Data...
-
BigData Engineer
2 weeks ago
Bengaluru, Karnataka, India Fractal Full time ₹ 20,00,000 - ₹ 25,00,000 per yearIt's fun to work in a company where people truly BELIEVE in what they are doingWe're committed to bringing passion and customer focus to the business.About The RoleWe are seeking highly skilled Big Data Engineers/Senior Engineers with a strong background in building and supporting big data solutions. The ideal candidate will have expertise in big data...