Lead Data Engineer

1 day ago

Bengaluru, Karnataka, India 5c292c78-aa36-42b5-b9a3-4c26cc408f73 Full time ₹ 19,00,000 - ₹ 20,00,000 per year

Job Designation: Senior Data Ops Engineer – Team Lead

Job Location: Bengaluru

Key Responsibilities:

  • Leadership & Team Management
  • Lead and mentor a team of DataOps engineers in designing and maintaining robust data pipelines.
  • Plan, assign, and review team tasks to ensure timely and quality delivery.
  • Collaborate with data engineers, data scientists, and business teams to prioritize data needs and ensure alignment with organizational goals.
  • Drive best practices in coding standards, documentation, and deployment automation.
  • Technical Delivery
  • Design and implement scalable ETL/ELT pipelines using Pentaho, StreamSets, and Python-based frameworks.
  • Manage real-time and batch data ingestion using Kafka for streaming and MySQL/Snowflake for storage and transformation.
  • Implement and maintain data quality checks, validation, and reconciliation frameworks.
  • Ensure pipeline observability, error handling, and alerting mechanisms for proactive issue resolution.
  • Optimize Snowflake and MySQL queries for performance and cost efficiency.
  • Lead migration or modernization initiatives (e.g., on-prem to Snowflake/cloud).
  • Governance & Operations
  • Maintain data security, access control, and compliance with enterprise standards.
  • Define and track DataOps KPIs such as pipeline success rates, latency, and data quality metrics.
  • Partner with Infrastructure and DevOps teams for seamless environment management and scalability.

Technical Skills Required:

Databases:

  • Strong expertise in MySQL (query optimization, stored procedures, schema design).
  • Advanced knowledge of Snowflake (data modelling, performance tuning, cost optimization).

ETL & Data Pipeline Tools:

  • Hands-on experience with Pentaho Data Integration (Kettle) and/or StreamSets for ETL/ELT automation.

Streaming:

  • In-depth understanding of Apache Kafka (topic configuration, producer/consumer setup, schema registry, stream processing).

Programming:

  • Proficient in Python for data automation, transformation scripts, and integration with APIs.

Monitoring & Observability:

  • Familiarity with Grafana, Prometheus, or similar tools for performance and error tracking.

Cloud:

  • Exposure to AWS/Azure/GCP data stack (S3, Lambda, Glue, Dataflow, etc.).

Job Type: Full-time

Pay: ₹1,900, ₹2,000,000.00 per year

Benefits:

  • Commuter assistance
  • Health insurance
  • Provident Fund

Work Location: In person

  • Technical Lead- Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Astar Data Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Kindly find the Job Description Below.Job Title:Technical Lead- Data EngineerLocation: BangaloreYears of Experience: 8+ years of experienceSigmoidworks with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software...

  • Data & AI Engineer Lead

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Framework Design & Architecture Architect a metadata-driven, Python/Spark-based framework for automated data validation across high-volume production datasets. Define DQ rule templates for completeness, integrity, conformity, accuracy, and timeliness. Establish data quality thresholds, escalation protocols, and exception workflows. Automation & Integration...

  • Data Entry Operator

    1 week ago

    Bengaluru, Karnataka, India Design Engineer Full time ₹ 2,64,000 per year

    We are seeking a dedicated and detail-oriented Data Entry Operator (DEO) to support R&D projects. The role involves accurate data entry, documentation, and maintenance of research records in secure systems. The DEO will assist scientists, engineers, and administrative staff by ensuring timely and error-free handling of project information.Key...

  • Data Engineer Senior Consultant

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 1,20,000 - ₹ 3,00,000 per year

    Req ID: 341131NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer Senior Consultant to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Location: | Experience:...

  • Lead Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Infogain Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    The Azure Data Engineer (Lead) is a management-level role responsible for leading a team of data engineers to design, develop, and maintain data processing systems on the Azure platform. They work with data architects, data scientists, and other stakeholders to ensure data solutions meet business requirements.Core SkillsMastery of Azure data services such as...

  • Sr Data Engineer

    2 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Req ID: 340286NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sr Data Engineer (Life and Annuity) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Duties: At...

  • Lead Data Engineer

    1 day ago

    Bengaluru, Karnataka, India Infiniti Research Full time

    Lead Engineer MarTech/Digital Engineering (Customer Data Management)We are seeking a Lead Data Engineer to serve as the technical lead for the team responsible for managing and engineering customer data across Catalyst Brands.The ideal candidate will have a strong background in data engineering and software development, with hands-on experience building...

  • Lead Data Engineer

    1 week ago

    Bengaluru, Karnataka, India FYERS Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    We are a fast-scaling, data-first organization building intelligent data infrastructure to empower real-time, data-driven decisions across the business. We're seeking a Senior Data Engineer / Team Lead to architect scalable data platforms and lead a team of high-performing data engineers. This is a hands-on technical leadership role where you will not only...

  • Data Engineer

    7 days ago

    Bengaluru, Karnataka, India NTT DATA North America Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Hands-on experience in building and managing data...

  • Lead Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Cimpress Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Lead Data Engineer -What You Will Do:As a PR3 Lead Data Engineer, you will be instrumental in driving our data strategy, ensuring data quality, and leading the technical execution of a small, impactful team. Your responsibilities will include:Team Leadership:Establish the strategic vision for the evolution of our data products and our technology solutions,...