Lead Big Data Engineer Manager

5 days ago

Pune, Maharashtra, India CIRCANA Full time

What will you be doing?

  • As a part of this team, youll work alongside the brightest minds. As a Big Data Lead evaluate domain, financial and technical feasibility of solution ideas with help of all key stakeholders.
  • Design, develop, and maintain highly scalable data processing applications.

Job Responsibilities:

  • Write efficient, reusable and well documented code
  • Deliver big data projects using Spark, Scala , Python, SQL
  • Maintain and tune existing Spark applications to the fullest.
  • Find opportunities for optimizing existing spark applications.
  • Work closely with QA, Operations and various teams to deliver error free software on time.
  • Actively lead / participate daily agile / scrum meetings.
  • Take responsibility for Apache Spark development and implementation.

Requirements:

  • Engineering graduates in computer science backgrounds preferred with 8+ years of software development experience with Hadoop framework components(HDFS, Spark, Spark, Scala, PySpark)
  • Excellent at verbal, written and presentation skills
  • Ability to present and defend a solution with technical facts & business proficiency
  • Understanding of data-warehousing and data-modeling techniques
  • Strong data engineering skills
  • Knowledge of Core Java, Linux, SQL, and any scripting language
  • At least 6+ years of experience using Python / Scala, Spark, SQL
  • Knowledge of shell scripting is a plus
  • Knowledge of Core and Advance Java is a plus.
  • Experience in developing and tuning spark applications
  • Excellent understanding of spark architecture, data frames and tuning spark
  • Strong knowledge of database concepts, systems architecture, and data structures is a must Process oriented with strong analytical and problem solving skills
  • Experience in writing Python standalone applications dealing with PySpark API

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Vodafone Full time

    Design, develop, and maintain scalable data pipelines for processing structured and unstructured data. Build and optimise data storage solutions including distributed file systems, NoSQL databases, and data warehouses. Utilise big data frameworks such as Apache Hadoop, Spark, Flink, and Kafka to develop data processing workflows. Ensure high performance and...

  • Big Data Architect(Lead)

    4 hours ago

    Pune, Maharashtra, India CRUTZ LEELA ENTERPRISES Full time

    Big Data Architect- LeadExperience: 10-12 yearsKey Responsibilities:Data Architecture Design: Designing and implementing scalable and resilient dataarchitectures for both batch and streaming data processing.Data Modeling: Developing data models and database structures to ensure efficientdata storage and retrieval.Data Security and Governance: Ensuring data...

  • Big Data Lead

    11 hours ago

    Pune, Maharashtra, India DemandMatrix Full time

    Company Description At DemandMatrix, our vision is to disrupt the $100 billion sales and marketing intelligence industry by using domain knowledge, machine learning and AI. Fortune 100 companies like Microsoft, Google, Adobe, Amazon, IBM trust us to identify their next customer. Job Description What will you do?To help us go to the next level we are looking...

  • Manager-Data Engineering-Big Data Engineering

    6 days ago

    Pune, Maharashtra, India EXL Full time

    DescriptionAbout the RoleWe are seeking a highly skilled Lead Data Engineer to drive end-to-end data engineering initiatives, lead cross-functional teams, and deliver scalable, cloud-based data solutions. The ideal candidate will bring deep technical expertise, strong leadership, and the ability to collaborate effectively with global stakeholders in a...

  • Big Data Developer

    1 week ago

    Pune, Maharashtra, India airtel Full time

    Job Title:Big Data Developer (Manager)Location: PuneExperience: 3-5 yearsJob Description:Looking for an experienced Big Data Developer to develop, maintain, and optimize our big data solutions. The ideal candidate will have strong expertise in Java, Spark, API development, and a deep understanding of data engineering and big data tools.Key...

  • Manager-Data Engineering-Big Data Engineering

    3 days ago

    Pune, Maharashtra, India EXL Full time

    About The RoleWe are seeking a highly skilled Lead Data Engineer to drive end-to-end data engineering initiatives, lead cross-functional teams, and deliver scalable, cloud-based data solutions. The ideal candidate will bring deep technical expertise, strong leadership, and the ability to collaborate effectively with global stakeholders in a fast-paced...

  • Lead Azure Big Data Engineer

    8 hours ago

    Pune, Maharashtra, India VidPro Consultancy Services Full time

    Senior Azure Data EngineerLocation: Hyderabad, Bangalore and PuneExperience: 3-7 YearsWork Mode: HybridMandatory Skills: Python, Spark, SQL, Hadoop, Hive, Spark, Azure Databricks, Azure DataFactory, Azure Synapse, DLTs, Unity Catalog, Workflows, Infrastructure as Code (IaC), Orchestration-Airflow, ETL & Data Pipelines.Role Overview:We are looking for a...

  • Big Data Engineer

    10 hours ago

    Pune, Maharashtra, India Digital Placement Full time

    Primary – Scala with Spark Hdfs  ,Spark with scala , Scala  (Functional Programming ), Hbase , Sql , Sqoop ,Shell Script Optional :     Apache  Kafka , Akka Http , TeamCity , U-deploy ,  Autosys , SBT Good to have  :     Pivotal Gemfire 7+ years of strong experience as big data developer. Hands on knowledge of Shell Scripting. Familiar with...

  • Big Data Specialist

    3 days ago

    Pune, Maharashtra, India Emergys Full time

    Work Location:Pune(Work-from-Office)Notice Period:Immediate to 30 DaysExperience:8+ YearsMust Have Skills:Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.Data Modeling & Integration – Strong ability to build efficient data models and integrate data from multiple sources.Big...

  • Big Data Developer Lead

    3 days ago

    Pune, Maharashtra, India People Prime Worldwide Full time

    Our is Client is a largest Top 5 Software giant in India, with over 11.3 USD billion dollars revenue, Global work force 2,40,000 employees, It delivers end-to-end technology, consulting, and business process services to clients across the globe, Presence: 60+ countries and Publicly traded company NSE & BSE (India), NYSE (USA).· Job Title: Big Data Developer...