Big Data Lead

17 hours ago

Pune, Maharashtra, India DemandMatrix Full time
Company Description

At DemandMatrix, our vision is to disrupt the $100 billion sales and marketing intelligence industry by using domain knowledge, machine learning and AI. Fortune 100 companies like Microsoft, Google, Adobe, Amazon, IBM trust us to identify their next customer.

Job Description

What will you do?

  • To help us go to the next level we are looking to onboard a hands-on SME in leveraging big data tech to solve the most complex data issues. You will spend almost half of time with hands-on coding.
  • It involves large scale text data processing, event driven data pipelines, in-memory computations, optimization considering CPU core to network IO to disk IO.
  • You will be using cloud native services in AWS and GCP.

Who Are You? 

  • Solid grounding in computer engineering, Unix, data structures and algorithms would enable you to meet this challenge.
  • Designed and built multiple big data modules and data pipelines to process large volume. 
  • Genuinely excited about technology and worked on projects from scratch. 

Must have:

  • 7+ years  of hands-on experience in Software Development with a focus on big data and large data pipelines.
  • Minimum 3 years of experience to build services and pipelines using Python.
  • Expertise with a variety of data processing systems, including streaming, event, and batch (Spark, Hadoop/MapReduce)
  • Understanding of at least one NoSQL stores like MongoDB, Elasticsearch, HBase
  • Understanding of how data models, sharding and data location strategies for distributed data stores in large scale high-throughput and high-availability environments and their effect in non-structured text data processing
  • Experience with running scalable & high available systems with AWS or GCP.

Good to have:

  • Experience with Docker / Kubernetes
  • Exposure with CI/CD
  • Knowledge of Crawling/Scraping
Additional Information
  • Entire Work From Home
  • Birthday Leave
  • Remote Work 

  • Big Data Architect(Lead)

    10 hours ago

    Pune, Maharashtra, India CRUTZ LEELA ENTERPRISES Full time

    Big Data Architect- LeadExperience: 10-12 yearsKey Responsibilities:Data Architecture Design: Designing and implementing scalable and resilient dataarchitectures for both batch and streaming data processing.Data Modeling: Developing data models and database structures to ensure efficientdata storage and retrieval.Data Security and Governance: Ensuring data...

  • Big Data Developer

    5 days ago

    Pune, Maharashtra, India Talent Worx Full time

    We're hiring for one of the world's leading professional services firms, renowned for its commitment to innovation, excellence, and global impact. With a presence in over 150 countries, this organization provides services across consulting, audit, tax, risk advisory, and financial advisory — helping Fortune 500 companies and governments navigate complex...

  • Big Data Developer

    4 days ago

    Pune, Maharashtra, India Talent Worx Full time

    We're hiring for one of the world's leading professional services firms, renowned for its commitment to innovation, excellence, and global impact. With a presence in over 150 countries, this organization provides services across consulting, audit, tax, risk advisory, and financial advisory — helping Fortune 500 companies and governments navigate complex...

  • Big Data Developer

    1 week ago

    Pune, Maharashtra, India airtel Full time

    Job Title:Big Data Developer (Manager)Location: PuneExperience: 3-5 yearsJob Description:Looking for an experienced Big Data Developer to develop, maintain, and optimize our big data solutions. The ideal candidate will have strong expertise in Java, Spark, API development, and a deep understanding of data engineering and big data tools.Key...

  • Big Data Developer Lead

    4 days ago

    Pune, Maharashtra, India People Prime Worldwide Full time

    Our is Client is a largest Top 5 Software giant in India, with over 11.3 USD billion dollars revenue, Global work force 2,40,000 employees, It delivers end-to-end technology, consulting, and business process services to clients across the globe, Presence: 60+ countries and Publicly traded company NSE & BSE (India), NYSE (USA).· Job Title: Big Data Developer...

  • Lead Big Data Engineer Manager

    6 days ago

    Pune, Maharashtra, India CIRCANA Full time

    What will you be doing?As a part of this team, youll work alongside the brightest minds. As a Big Data Lead evaluate domain, financial and technical feasibility of solution ideas with help of all key stakeholders.Design, develop, and maintain highly scalable data processing applications.Job Responsibilities:Write efficient, reusable and well documented...

  • Big Data Solution Architect

    2 weeks ago

    Pune, Maharashtra, India Emergys Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Work Location:PuneNotice Period:Immediate to 30 DaysExperience:8+ YearsMust Have Skills:Hadoop Ecosystem Expertise – Strong hands-on experience with core Hadoop components and related big data technologies.DevOps & Cloud (Preferably Azure) – Experience using DevOps tools and CI/CD processes on cloud platforms, with a preference for Microsoft...

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Vodafone Full time

    Design, develop, and maintain scalable data pipelines for processing structured and unstructured data. Build and optimise data storage solutions including distributed file systems, NoSQL databases, and data warehouses. Utilise big data frameworks such as Apache Hadoop, Spark, Flink, and Kafka to develop data processing workflows. Ensure high performance and...

  • Big Data Specialist

    4 days ago

    Pune, Maharashtra, India Emergys Full time

    Work Location:Pune(Work-from-Office)Notice Period:Immediate to 30 DaysExperience:8+ YearsMust Have Skills:Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.Data Modeling & Integration – Strong ability to build efficient data models and integrate data from multiple sources.Big...

  • Big Data Developer

    3 weeks ago

    Pune, Maharashtra, India, Maharashtra UST Full time

    We are seeking a Big Data Engineer with strong hands-on experience in Spark and AWS technologies. The ideal candidate should demonstrate a deep understanding of big data concepts, programming fundamentals, and the ability to solve complex problems related to scalability, failure handling, and optimization. Key Responsibilities:Design, develop, and optimize...