Lead Data Engineer, Bangalore

9 hours ago

Bengaluru, Karnataka, India ITC Infotech Full time ₹ 12,00,000 - ₹ 25,00,000 per year

Location: Bangalore, Immediate joiners only (who can join within 3rd week of November)

Experience Level: 8+ years

Employment Type: Full-time

Reporting To: Data Engineering Manager / Head of Data Platform

Job Description:

We are seeking a highly skilled and experienced Lead Data Engineer to architect, design, and implement scalable data pipelines and solutions using Azure Data Factory, Databricks, PySpark, and SQL/SparkSQL. The ideal candidate will have mandatory hands-on experience with Databricks and strong exposure to Azure Data Factory, along with a deep understanding of distributed data processing and cloud-native data engineering practices.

Key Responsibilities:

  • Design and develop scalable data pipelines using Azure Data Factory and Databricks.
  • Lead the implementation of ETL/ELT workflows using PySpark and SparkSQL.
  • Architect and optimize data lake and data warehouse solutions on Azure.
  • Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
  • Ensure data quality, governance, and security across all data platforms.
  • Implement CI/CD pipelines for data engineering workflows.
  • Monitor and troubleshoot performance issues in Spark jobs and ADF pipelines.
  • Mentor junior data engineers and contribute to best practices and standards.

Mandatory Skills & Experience:

  • Databricks (Mandatory):

  • Extensive experience in building and managing notebooks, clusters, jobs, and workflows.

  • Proficient in PySpark for distributed data processing.

  • Experience with Delta Lake, Unity Catalog, and Databricks SQL.

  • Azure Data Factory (ADF):

  • Strong exposure to building and orchestrating pipelines.

  • Integration with various Azure services (Blob Storage, Synapse, Key Vault, etc.).

  • PySpark:

  • Advanced programming skills in PySpark for data transformation and aggregation.

  • Performance tuning of Spark jobs.

  • SQL & SparkSQL:

  • Strong command over writing optimized SQL queries.

  • Experience with SparkSQL for querying large datasets.

Preferred Skills:

  • Experience with Azure Synapse Analytics, Azure Functions, and Azure DevOps.
  • Familiarity with data modeling, data warehousing, and dimensional modeling.
  • Knowledge of data governance frameworks and metadata management.
  • Exposure to streaming data using Spark Structured Streaming or Azure Event Hub.

Qualifications:

  • Bachelors or Masters degree in Computer Science, Engineering, or related field.
  • 8+ years of experience in data engineering roles.
  • Certifications in Azure or Databricks are a plus.

Regards

Manosh Happy

  • Data & AI Engineer Lead

    2 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Framework Design & Architecture Architect a metadata-driven, Python/Spark-based framework for automated data validation across high-volume production datasets. Define DQ rule templates for completeness, integrity, conformity, accuracy, and timeliness. Establish data quality thresholds, escalation protocols, and exception workflows. Automation & Integration...

  • Lead Data Scientist

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are currently seeking a Lead Data Scientist - Computer Vision & Generative AI to join our team in Bangalore, Karntaka (IN-KA), India (IN).We are seeking a highly skilled and experienced Lead Data Scientist with 7+ years of expertise in Machine Learning, Deep Learning (especially Computer Vision), and Generative AI. The ideal candidate will lead the design...

  • Sr. Data Engineer Data Science

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Req ID: 342869NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sr. Data Engineer Data Science to join our team in Bangalore, Karnātaka (IN-KA), India (IN). ...

  • Engineering Lead – Data and AI Platform- Bangalore

    2 weeks ago

    Bengaluru, Karnataka, India HYrEzy Tech Solutions Full time ₹ 70,00,000 - ₹ 80,00,000 per year

    Engineering Lead – Data and AI PlatformLocation: Bangalore (preferred) — candidates open to relocation invitedExperience: 8 –10 years (3 – 4 years in engineering leadership roles)Compensation: ₹70 – 80 LPA + ESOP (based on fit)About company:-We are a fast-growing startup building an AI-first omnichannel franchisee platform, empowering global...

  • Data Engineer

    3 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Req ID: 345819NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer / Developer (Pipelines & Analytics) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Role:...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 4,00,000 - ₹ 8,00,000 per year

    Req ID: 344005NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - ETL Developer to join our team in Bangalore, Karnātaka (IN-KA), India (IN). "Job Duties: Design,...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Req ID: 343998NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - Sr ETL Developer (Architect) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). "Job...

  • Data Engineering

    23 minutes ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 18,00,000 per year

    Req ID: 345813NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineering & Analytics Architect to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Role: Own the...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...

  • Lead Data Engineer

    3 days ago

    Bengaluru, Karnataka, India Capgemini Engineering Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Job DescriptionData engineers are responsible for building reliable and scalable data infrastructure that enables organizations to derive meaningful insights, make data-driven decisions, and unlock the value of their data assets.Job Description - Grade SpecificThe role involves leading and managing a team of data engineers, defining and executing the data...