Senior Big Data Expert

2 days ago

Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

About this role

We're seeking an experienced Data Engineer to lead the design and development of scalable data pipelines using Spark.

You'll work with clients to deliver a top-notch product experience through best-in-class PIM implementation and rich product information across channels and digital touchpoints.

Key responsibilities include:

  • Designing and developing efficient pipelines to process and transform large datasets using Spark
  • Crafting a scalable and robust framework for generating PDF reports using Python and Spark
  • Utilizing Snowflake Spark SQL to perform aggregations on high volumes of data
  • Developing Stored Procedures Views Indexes Triggers and Functions in Snowflake Database to maintain data and share with downstream applications via APIs
  • Leveraging Azure Databricks and Datalake for data processing and storage
  • Developing APIs using Python's Flask framework to support front-end applications

You'll collaborate with Architects and Business stakeholders to understand reporting requirements and improve existing reporting pipelines and infrastructure.

Requirements include:

  • Proven experience as a Data Engineer with a strong understanding of data pipelines and ETL processes
  • Proficiency in Python with experience in data manipulation libraries such as Pandas and Numpy
  • Experience with SQL Snowflake Spark for data querying and aggregations
  • Familiarity with Azure cloud services such as Data Factory Databricks and Datalake

  • Big Data Expert

    4 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 80,00,000 - ₹ 1,20,00,000

    Big Data DeveloperThis role is ideal for a skilled and ambitious professional with expertise in big data technologies. As a Big Data Developer, you will play a key part in designing, developing, and maintaining large-scale data processing systems using Hadoop, Spark, and NoSQL databases.Key Responsibilities:Develop scalable data pipelines using Apache Kafka,...

  • AWS Big Data Engineer

    4 days ago

    Pune, Maharashtra, India beBeeBigData Full time ₹ 15,00,000 - ₹ 28,00,000

    AWS Big Data ExpertWe are looking for an experienced AWS big data expert to join our team. As a key member of our technology group, you will be responsible for designing, implementing, and maintaining large-scale big data systems on the AWS platform.

  • Big Data

    5 days ago

    Pune, Maharashtra, India Atgeir Solutions Full time US$ 1,50,000 - US$ 2,00,000 per year

    We're Hiring – 𝐁𝐢𝐠 𝐃𝐚𝐭𝐚 / 𝐂𝐥𝐨𝐮𝐝 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭Location: 𝐆𝐥𝐨𝐛𝐚𝐥 𝐃𝐞𝐥𝐢𝐯𝐞𝐫𝐲 𝐂𝐞𝐧𝐭𝐞𝐫, 𝐏𝐮𝐧𝐞Type: 𝐅𝐮𝐥𝐥-𝐓𝐢𝐦𝐞Experience: 𝟏𝟎–𝟏𝟓 𝐘𝐞𝐚𝐫𝐬Job ID:...

  • Big Data Cluster Manager

    7 days ago

    Pune, Maharashtra, India beBeeExpertise Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Big Data Cluster ManagerOverview: We are seeking a seasoned expert to manage and optimize our big data clusters.

  • Big Data Architectural Expert

    5 days ago

    Pune, Maharashtra, India beBeeEngineer Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Take Your Career to the Next Level as a Lead Big Data Engineer!Job Description:Are you passionate about harnessing the power of big data to drive business growth and innovation? We're seeking an exceptional Lead Big Data Engineer to join our team and help us build the next generation of big data platforms.The ideal candidate will have extensive experience in...

  • Big Data Specialist

    2 hours ago

    Pune, Maharashtra, India beBeeBigData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Expert - Big Data SpecialistDescription:We are seeking a highly skilled professional to join our team. The ideal candidate will have extensive experience with big data technologies and a strong background in developing and optimizing data integration frameworks and applications.Main Responsibilities:Data Solutions: Design and implement robust...

  • Senior Big Data Solutions Architect

    5 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data Solutions ArchitectWe are seeking a seasoned Big Data Solutions Architect to lead our efforts in designing and implementing cutting-edge data solutions. The ideal candidate will have a strong background in distributed computing principles, Apache Spark, and Python programming.Key Responsibilities:Design and implement scalable big data architectures...

  • Big Data Developer

    3 weeks ago

    Pune, Maharashtra, India Impetus Full time

    Job Descriptions for Big data or Cloud Engineer Position Summary: We are looking for candidates with hands on experience in Big Data or Cloud Technologies. Locations: Pune, Gurgaon, Indore, Noida, Hyderabad, Bangalore Must have technical Skills 2-4 Years of experience Expertize and hands-on experience on Python – Must Have Expertize knowledge on...

  • Big Data Developer

    3 weeks ago

    Pune, Maharashtra, India Impetus Full time

    Job Descriptions for Big data or Cloud Engineer Position Summary: We are looking for candidates with hands on experience in Big Data or Cloud Technologies. Locations: Pune, Gurgaon, Indore, Noida, Hyderabad, Bangalore Must have technical Skills 2-4 Years of experience Expertize and hands-on experience on Python – Must Have Expertize knowledge on...

  • Big Data Expert

    6 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Unlock Big Data InsightsImagine a world where data is not just a buzzword, but a playground where you get to build innovative solutions that make a real impact.Key Responsibilities:Become the driving force behind building and maintaining a scalable data ecosystem using Apache Spark.Collaborate with data engineers to design and implement ETL workflows for...