Lead Data Engineer, Bangalore

22 hours ago

Bengaluru, Karnataka, India ITC Infotech Full time ₹ 12,00,000 - ₹ 25,00,000 per year

Location: Bangalore, Immediate joiners only (who can join within 3rd week of November)

Experience Level: 8+ years

Employment Type: Full-time

Reporting To: Data Engineering Manager / Head of Data Platform

Job Description:

We are seeking a highly skilled and experienced Lead Data Engineer to architect, design, and implement scalable data pipelines and solutions using Azure Data Factory, Databricks, PySpark, and SQL/SparkSQL. The ideal candidate will have mandatory hands-on experience with Databricks and strong exposure to Azure Data Factory, along with a deep understanding of distributed data processing and cloud-native data engineering practices.

Key Responsibilities:

  • Design and develop scalable data pipelines using Azure Data Factory and Databricks.
  • Lead the implementation of ETL/ELT workflows using PySpark and SparkSQL.
  • Architect and optimize data lake and data warehouse solutions on Azure.
  • Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
  • Ensure data quality, governance, and security across all data platforms.
  • Implement CI/CD pipelines for data engineering workflows.
  • Monitor and troubleshoot performance issues in Spark jobs and ADF pipelines.
  • Mentor junior data engineers and contribute to best practices and standards.

Mandatory Skills & Experience:

  • Databricks (Mandatory):

  • Extensive experience in building and managing notebooks, clusters, jobs, and workflows.

  • Proficient in PySpark for distributed data processing.

  • Experience with Delta Lake, Unity Catalog, and Databricks SQL.

  • Azure Data Factory (ADF):

  • Strong exposure to building and orchestrating pipelines.

  • Integration with various Azure services (Blob Storage, Synapse, Key Vault, etc.).

  • PySpark:

  • Advanced programming skills in PySpark for data transformation and aggregation.

  • Performance tuning of Spark jobs.

  • SQL & SparkSQL:

  • Strong command over writing optimized SQL queries.

  • Experience with SparkSQL for querying large datasets.

Preferred Skills:

  • Experience with Azure Synapse Analytics, Azure Functions, and Azure DevOps.
  • Familiarity with data modeling, data warehousing, and dimensional modeling.
  • Knowledge of data governance frameworks and metadata management.
  • Exposure to streaming data using Spark Structured Streaming or Azure Event Hub.

Qualifications:

  • Bachelors or Masters degree in Computer Science, Engineering, or related field.
  • 8+ years of experience in data engineering roles.
  • Certifications in Azure or Databricks are a plus.

Regards

Manosh Happy

  • Technical Lead- Data Engineer

    2 days ago

    Bengaluru, Karnataka, India Astar Data Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Kindly find the Job Description Below.Job Title:Technical Lead- Data EngineerLocation: BangaloreYears of Experience: 8+ years of experienceSigmoidworks with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software...

  • Lead Data Scientist

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are currently seeking a Lead Data Scientist - Computer Vision & Generative AI to join our team in Bangalore, Karntaka (IN-KA), India (IN).We are seeking a highly skilled and experienced Lead Data Scientist with 7+ years of expertise in Machine Learning, Deep Learning (especially Computer Vision), and Generative AI. The ideal candidate will lead the design...

  • Data & AI Engineer Lead

    3 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Framework Design & Architecture Architect a metadata-driven, Python/Spark-based framework for automated data validation across high-volume production datasets. Define DQ rule templates for completeness, integrity, conformity, accuracy, and timeliness. Establish data quality thresholds, escalation protocols, and exception workflows. Automation & Integration...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Req ID:321800We are currently seeking a Data Engineer (Talend &Pyspark) to join our team in Bangalore, Karntaka (IN-KA), India (IN)."Job Duties: Key Responsibilities: Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical...

  • Sr. Data Engineer Data Science

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Req ID: 342869NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sr. Data Engineer Data Science to join our team in Bangalore, Karnātaka (IN-KA), India (IN). ...

  • Lead Data Scientist

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Lead Data Scientist - Computer Vision & Generative AI to join our team in Bangalore, Karnātaka (IN-KA), India (IN). We are seeking...

  • Lead Data Engineer

    12 hours ago

    Bengaluru, Karnataka, India MM Staffing & Career Consultants Full time ₹ 25,00,000 - ₹ 28,00,000 per year

    Job Description:About CompanyIt is a global consulting company offering solutions, services and platforms on Open Source, Cloud and Automation technologies. Since 2000, It has been delivering enterprise class solutions, which are built using leading platforms & can easily be integrated with existing systems to achieve unparalleled results.Job...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 4,00,000 - ₹ 8,00,000 per year

    Req ID: 344005NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - ETL Developer to join our team in Bangalore, Karnātaka (IN-KA), India (IN). "Job Duties: Design,...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Req ID: 343998NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - Sr ETL Developer (Architect) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). "Job...

  • Sr. Data Science

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are currently seeking a Sr. Data Science & AIML, GenAI Lead/Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN).Job Duties: Job Title: Data Science & AIML, GenAI Lead/EngineerKey Responsibilities: Develop and implement traditional machine learning algorithms. Deploy at least one model in a production environment. Write and maintain Python...