See more Collapse

Sr. Spark and Data Lake Administrator

2 months ago

pune, India Jade Global Full time

ob Description

Responsibilities:

 Configure, fine-tune, and optimize Spark and YARN administration for high performance

and resource efficiency.

 Setup, configuring and manage Data Lake Houses using Hudi and/or Delta Lake for

efficient data storage and processing.

 Collaborate with cross-functional teams to design and maintain scalable and reliable

data lake architectures.

 Monitor and troubleshoot Spark and Data Lake environments to ensure continuous

operation and performance.

 Provide technical guidance, training and support to team members and stakeholders on

Spark and Data Lake administration best practices.

Requirements:

 Proven experience in configuring and tuning Spark and YARN administration in large-

scale environments. Kubernetes experience a plus.

 Solid understanding of Data Lake architectures and Big Table technologies, data

modeling, and management principles. Experience setting up, configuring and working

with Hudi and/or Delta Lake.

 Experience in Python and Spark development is .

 Strong problem-solving skills and the ability to troubleshoot complex technical issues.

 Excellent communication and collaboration skills.

We have other current jobs related to this field that you can find below

  • Sr. Spark and Data Lake Administrator

    2 weeks ago

    Pune, Maharashtra, India Jade Global Full time

    ob Description Responsibilities:  Configure, fine-tune, and optimize Spark and YARN administration for high performance and resource efficiency.  Setup, configuring and manage Data Lake Houses using Hudi and/or Delta Lake for efficient data storage and processing.  Collaborate with cross-functional teams to design and maintain scalable and reliable...

  • Data Lake Architect

    3 weeks ago

    Pune, Maharashtra, India NewVision Softcom & Consultancy Full time

    DataPunePosted On - 19 Feb 2024 - Required Experience - 10 - 14 Years Basic SectionGrade Role Architect Employment Type Full Time Employee Category OrganisationalGroup Company NewVision Company Name New Vision Softcom & Consultancy Pvt. Ltd Function Business Units (BU) Department/Practice Data Department/Practice Data...

  • Sr. Data Engineer

    2 weeks ago

    Pune, Maharashtra, India MICHELIN Full time

    Sr. Data Engineer A Data Engineer is a data professionals with extensive knowledge and expertise in "big data" technologies and frameworks that works with Python/Scala/Java. They are software engineers who design, build, integrate data from various resources, and manage big data. Experienced in writing spark code with python or Scala, Sql queries, Linux/Unix...

  • Hiring For Apache spark developer

    4 days ago

    Pune, India timesjobs Full time

    Hiring For Apache spark developerApache spark developerDesired ProfileSkills : Apache spark developer, Data Lake solution using Scala/Python with large and multi-format data, coding experience using Spark (Scala/Python), SQLNotice Period: 0- 60 daysEducation: any graduates(full-time)Location: PuneEmail: career@krazymantra.com

  • Sr. Data Engineer

    2 weeks ago

    Pune, Maharashtra, India MICHELIN Full time

    Sr. Data Engineer A Data Engineer is a data professionals with extensive knowledge and expertise in "big data" technologies and frameworks that works with Python/Scala/Java. They are software engineers who design, build, integrate data from various resources, and manage big data. Experienced in writing spark code with python or Scala, Sql queries,...

  • Sr. Azure Data Engineer

    1 month ago

    Pune, India Fractal Full time

    It's fun to work in a company where people truly BELIEVE in what they are doing!JOB DESCRIPTION: If you are an extraordinary developer and who loves to push the boundaries to solve complex business problems using creative solutions, then we wish to talk with you. As a Data Engineer - Azure, you will work in the Technology team that helps deliver our Data...

  • Sr. Azure Data Engineer

    4 weeks ago

    Pune, India Fractal Full time

    It's fun to work in a company where people truly BELIEVE in what they are doing!JOB DESCRIPTION: If you are an extraordinary developer and who loves to push the boundaries to solve complex business problems using creative solutions, then we wish to talk with you. As a Data Engineer - Azure, you will work in the Technology team that helps deliver our Data...

  • Senior Azure Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Yash Technologies Full time

    Sr.Software Engineer - Azure Experience: - 7 years of Experience What you should have? Azure Data Factory, Azure Dev Ops CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, Py Spark, Git Hub.Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL.Well versed with Azure Services Must have experience and knowledge on ADF, ADLS,...

  • Data Lake Admin

    2 weeks ago

    Pune, India Vertiv Full time

    Position Summary:This candidate will be a key contributor to the Global Data Management & Analytics team by implementing innovative big data solutions to solve business challenges. Business challenges this individual will contribute to include data integrations, relational data source ingestion/mapping/transformation, cloud application integration, and...

  • Big Data Administrator

    3 weeks ago

    Pune, India TransUnion Full time

    TransUnion's Job Applicant Privacy Notice **What We'll Bring**: We’re looking for a Big Data Administrator to join our growing Big Data Administration team. You will be based both from our offices in Leeds and working remotely as part of our ‘flex together’ approach. In this fast-paced role you’ll work with Data Engineering and Data Science teams to...

  • Data Lake Admin

    2 weeks ago

    Pune, Maharashtra, India Vertiv Full time

    Position Summary:This candidate will be a key contributor to the Global Data Management & Analytics team by implementing innovative big data solutions to solve business challenges. Business challenges this individual will contribute to include data integrations, relational data source ingestion/mapping/transformation, cloud application integration, and...

  • Data Lake Admin

    2 weeks ago

    Pune, Maharashtra, India Vertiv Group Corp Full time

    Position Summary:He will have hands on experience in big data architecture and data warehousing principles as well as hands on experience in full information lifecycle for large scale big data environments - from data origination to data consumption.Key Duties Administer our data lake organizing access controls processes Support the ETL development team...

  • Data Lake Admin

    2 weeks ago

    Pune, Maharashtra, India Vertiv Group Corp Full time

    Position Summary: He will have hands on experience in big data architecture and data warehousing principles as well as hands on experience in full information lifecycle for large scale big data environments - from data origination to data consumption. Key Duties: Administer our data lake organizing access controls processes Support the ETL development team...

  • Data Lake Admin

    3 weeks ago

    Pune, Maharashtra, India Vertiv Group Corp Full time

    **Position Summary**: He will have hands on experience in big data architecture and data warehousing principles as well as hands on experience in full information lifecycle for large scale big data environments - from data origination to data consumption. Key Duties - Administer our data lake organizing access controls processes - Support the ETL...

  • Data Lake Admin

    3 months ago

    Pune, India Vertiv Full time

    Position Summary : This candidate will be a key contributor to the Global Data Management & Analytics team by implementing innovative big data solutions to solve business challenges. Business challenges this individual will contribute to include data integrations, relational data source ingestion/mapping/transformation, cloud application integration, and...

  • Data Lake Admin

    4 weeks ago

    pune, India Vertiv Full time

    Position Summary : This candidate will be a key contributor to the Global Data Management & Analytics team by implementing innovative big data solutions to solve business challenges. Business challenges this individual will contribute to include data integrations, relational data source ingestion/mapping/transformation, cloud application integration,...

  • Data Lake Admin

    2 weeks ago

    Pune, Maharashtra, India Vertiv Full time

    Position Summary : This candidate will be a key contributor to the Global Data Management & Analytics team by implementing innovative big data solutions to solve business challenges. Business challenges this individual will contribute to include data integrations, relational data source ingestion/mapping/transformation, cloud application integration,...

  • Frequence - Data Engineer - SQL/Scala/Spark

    4 weeks ago

    Pune, India Frequence Full time

    Job Description : - You are a 5-8 Years of experienced professional having in-depth knowledge of SQL, Scala, Spark, AWS - EMR, AWS Athena, AWS S3, AWS Glue, and PHP (all skills are required).- Knowledge of scripting languages including Python, Java, C++, and Linux are add-ons.- In-depth Knowledge of NoSQL and RDBMS databases.- Familiarity with Airflow and...

  • Frequence - Data Engineer - SQL/Scala/Spark

    3 months ago

    Pune, India Frequence Full time

    Job Description : - You are a 5-8 Years of experienced professional having in-depth knowledge of SQL, Scala, Spark, AWS - EMR, AWS Athena, AWS S3, AWS Glue, and PHP (all skills are required).- Knowledge of scripting languages including Python, Java, C++, and Linux are add-ons.- In-depth Knowledge of NoSQL and RDBMS databases.- Familiarity with Airflow and...

  • Data Lake Admin

    4 weeks ago

    Pune, India Vertiv Full time

    Position Summary : This candidate will be a key contributor to the Global Data Management & Analytics team by implementing innovative big data solutions to solve business challenges. Business challenges this individual will contribute to include data integrations, relational data source ingestion/mapping/transformation, cloud application integration,...