Lead Data Engineering Solutions

5 days ago

Pune, Maharashtra, India beBeeDataEngineering Full time
Senior Data Engineering Expert

At the forefront of innovation, we seek a visionary Senior Data Engineering Expert to spearhead our data-driven solutions. With 5-8 years of experience in crafting efficient data pipelines and analytics workflows, you will be responsible for designing and developing scalable architectures that drive business growth.

Key Responsibilities:

  • Design, develop, and optimize ETL/ELT pipelines using Apache Spark, PySpark, Databricks, and Snowflake.
  • Implement real-time and batch data processing workflows in cloud environments (AWS, Azure, GCP).
  • Develop high-performance, scalable data pipelines for structured, semi-structured, and unstructured data.
  • Work with Delta Lake and Lakehouse architectures to improve data reliability and efficiency.
  • Optimize Snowflake and Databricks performance, including query tuning, caching, partitioning, and cost optimization.
  • Implement data governance, security, and compliance best practices.
  • Build and maintain data models, transformations, and data marts for analytics and reporting.
  • Collaborate with data scientists, analysts, and business teams to define data engineering requirements.
  • Automate infrastructure and deployments using Terraform, Airflow, or dbt.
  • Monitor and troubleshoot data pipeline failures, performance issues, and bottlenecks.
  • Develop and enforce data quality and observability frameworks using Great Expectations, Monte Carlo, or similar tools.

Requirements:

  • Bachelor's or Master's Degree in Computer Science or Data Science.
  • 5-8 years of experience in data engineering, big data processing, and cloud-based data platforms.
  • Hands-on expertise in Apache Spark, PySpark, and distributed computing frameworks.
  • Strong experience with Snowflake (Warehouses, Streams, Tasks, Snowpipe, Query Optimization).
  • Experience in Databricks (Delta Lake, MLflow, SQL Analytics, Photon Engine).
  • Proficiency in SQL, Python, or Scala for data transformation and analytics.
  • Experience working with data lake architectures and storage formats (Parquet, Avro, ORC, Iceberg).
  • Hands-on experience with cloud data services (AWS Redshift, Azure Synapse, Google BigQuery).
  • Experience in workflow orchestration tools like Apache Airflow, Prefect, or Dagster.
  • Strong understanding of data governance, access control, and encryption strategies.
  • Experience with CI/CD for data pipelines using GitOps, Terraform, dbt, or similar technologies.

Preferred Qualifications:

  • Knowledge of streaming data processing (Apache Kafka, Flink, Kinesis, Pub/Sub).
  • Experience in BI and analytics tools (Tableau, Power BI, Looker).
  • Familiarity with data observability tools (Monte Carlo, Great Expectations).
  • Experience with machine learning feature engineering pipelines in Databricks.
  • Contributions to open-source data engineering projects.

  • Lead Data Engineering Solutions

    1 day ago

    Pune, Maharashtra, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 25,00,000

    **Job Opportunity: Data Engineering Expert**We're seeking an experienced professional to lead our data engineering initiatives and design, develop, and implement large-scale solutions using Azure cloud services.Collaborate with stakeholders to gather requirements and provide innovative solutionsDevelop and implement data pipelines using Azure Data Factory...

  • Data Engineering Lead

    2 days ago

    Pune, Maharashtra, India Ascentt Full time US$ 1,25,000 - US$ 1,75,000 per year

    Job Title: Data Engineering LeadJob Summary:We are seeking an experienced Data Engineering Lead to oversee and guide our organization's data engineering function. The ideal candidate will combine deep technical expertise in big data and data engineering with strong leadership skills to drive innovation, best practices, and excellence across our data...

  • Lead Data Engineer

    5 days ago

    Pune, Maharashtra, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering LeaderA seasoned Data Engineer to spearhead our data strategy and drive business outcomes.Job SummaryWe are seeking an experienced Data Engineer to lead our team in driving the development, implementation, and maintenance of data engineering solutions. The successful candidate will have a strong background in data engineering principles,...

  • Delivery Solution Architect – IIoT, Cloud Data

    2 weeks ago

    Pune, Maharashtra, India Capgemini Engineering Full time

    Role – Delivery Solution Architect – IIoT, Cloud Data & AI Analytics Job description We are seeking a highly skilled and experienced Delivery Solution Architect with deep expertise in Industrial IoT (IIoT), Cloud Data Platforms, and AI/Analytics to lead the design and delivery of smart factory solutions under the Industry 4.0 paradigm. This role will...

  • Lead Data Engineer

    4 days ago

    Pune, Maharashtra, India Mastercard Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Title:Lead Data Engineer Overview:OverviewWe are the global technology company behind the world's fastest payments processing network. We are a vehicle for commerce, a connection to financial systems for the previously excluded, a technology innovation lab, and the home of Priceless. We ensure every employee has the opportunity to be a part of something...

  • 3 Days Left Delivery Solution Architect – IIoT, Cloud Data

    4 weeks ago

    Pune, Maharashtra, India Capgemini Engineering Full time

    Role – Delivery Solution Architect – IIoT, Cloud Data & AI AnalyticsJob descriptionWe are seeking a highly skilled and experienced Delivery Solution Architect with deep expertise in Industrial IoT (IIoT), Cloud Data Platforms, and AI/Analytics to lead the design and delivery of smart factory solutions under the Industry 4.0 paradigm. This role will be...

  • Lead Data Engineer

    3 days ago

    Pune, Maharashtra, India Mastercard Full time US$ 1,25,000 - US$ 1,75,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...

  • Lead Data Engineer

    5 days ago

    Pune, Maharashtra, India Mastercard Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...

  • Data Engineering Lead

    2 days ago

    Pune, Maharashtra, India beBeeDataEngineering Full time US$ 1,30,000 - US$ 1,70,000

    Job Title: Data Engineering LeadWe are seeking a highly skilled and experienced data engineering lead to design, implement, and manage large-scale data solutions using Azure technologies. The ideal candidate will have a proven track record of designing, implementing, and managing complex system integration processes focused primarily on the use of Azure SQL,...

  • Lead Data Engineer

    4 weeks ago

    Pune, Maharashtra, India VY SYSTEMS PRIVATE LIMITED Full time

    Immediate to 15 days and are highly encouraged to applyExperiences : 9+ YearsLocation : PuneRelocation candidates are also welcome.Job Summary :We are looking for a highly experienced and technically proficient Lead Data Engineer with 9+ years of experience to design, build, and optimize data pipelines and platforms in a cloud-native environment. This role...