Scalable Data Architect

23 hours ago

Pune, Maharashtra, India beBeeData Full time ₹ 18,23,239 - ₹ 24,30,987
Senior Data Engineer - Azure Databricks

We are seeking a seasoned professional to lead the development of scalable data pipelines and streaming architectures using Azure-native technologies.

Main Responsibilities :

  • Design, develop, and optimize data pipelines using Databricks (PySpark) for batch and real-time data processing.
  • Implement CDC (Change Data Capture) and Delta Live Tables/Autoloader to support near-real-time ingestion.
  • Integrate various structured and semi-structured data sources using ADF, ADLS, and Kafka (Confluent).
  • Develop CI/CD pipelines for data engineering workflows using GitHub Actions or Azure DevOps.
  • Write efficient and reusable SQL and Python code for data transformations and validations.
  • Ensure data quality, lineage, governance, and security across all ingestion and transformation layers.
  • Collaborate closely with business analysts, data scientists, and data stewards to support use cases in risk, finance, compliance, and operations.
  • Participate in code reviews, architectural discussions, and documentation efforts.

Required Skills & Qualifications :

  • Strong proficiency in SQL, Python, and PySpark.
  • Proven experience with Azure Databricks, including notebooks, jobs, clusters, and Delta Lake.
  • Experience with Azure Data Lake Storage (ADLS Gen2) and Azure Data Factory (ADF).
  • Hands-on with Confluent Kafka for streaming data integration.
  • Strong understanding of Autoloader, CDC mechanisms, and Delta Lake-based architecture.
  • Experience implementing CI/CD pipelines using GitHub and/or Azure DevOps.
  • Knowledge of data modeling, data warehousing, and data security best practices.
  • Exposure to regulatory and risk data use cases in the banking/financial sector is a strong plus.

Preferred Qualifications :

  • Azure certifications (e.g., Azure Data Engineer Associate).
  • Experience with tools such as Delta Live Tables, Unity Catalog, and Lakehouse architecture.
  • Familiarity with business glossaries, data lineage tools, and data governance frameworks.
  • Understanding of financial data including GL, loan, customer, transaction, or market risk domains.

  • Architecting Scalable Data Models for Cloud Platforms

    2 weeks ago

    Pune, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Enterprise Data Model ArchitectDescription:As a seasoned data professional, we are seeking an accomplished Enterprise Data Model Architect to design and implement scalable and modular data models for our cloud-based platform. This role requires expertise in designing enterprise-level data models, incorporating region and country-specific...

  • Building Scalable Data Solutions

    2 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data ArchitectWe are seeking a highly skilled Senior Data Architect to design and implement scalable data solutions.Develops and implements comprehensive data architectures, ensuring scalability and maintainability.Designs and optimizes large-scale data pipelines with strong communication skills.Collaborates with cross-functional teams to deliver...

  • Expert Leader

    5 days ago

    Pune, Maharashtra, India beBeeDataSystems Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Lead Advanced Data SystemsThe Team Lead for Advanced Data Systems Delivery will lead a team responsible for delivering high-performance, scalable data solutions.Lead the end-to-end delivery of advanced data systems for mission-critical Tier 1 applications.Ensure solutions meet NFRs (e.g., scalability, performance, availability, security) and SLAs.This role...

  • Designing Scalable Data Solutions

    4 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 25,00,000 - ₹ 30,00,000

    Job Title:Data Solutions ArchitectWe are seeking a highly skilled data solutions architect to join our team. The ideal candidate will have extensive experience in designing, building, and maintaining robust and scalable data solutions.Collaborate with Data and ETL teams to review code, understand current architecture, and improve it based on the latest best...

  • Scalable Data Solutions Developer

    2 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 2,59,20,000

    Data Engineering RoleWe are seeking an accomplished Data Engineer to develop, deploy and maintain scalable data pipelines using Python and GCP.Key Responsibilities:Data Pipeline DevelopmentCreate high-throughput, low-latency pipelines to process massive datasets efficiently.Ensure pipelines can handle 5–10 TB of structured and unstructured data at...

  • Scalable Data Architect

    6 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Azure Databricks Engineer RoleWe are seeking a skilled and experienced professional to join our organization as an Azure Databricks Engineer. This role requires deep hands-on expertise in building scalable data pipelines and streaming architectures using Azure-native technologies.Key Responsibilities:Design, develop, and optimize data pipelines using...

  • Data Architect

    7 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Senior Data Solutions ArchitectWe are seeking a highly skilled and experienced Senior Data Solutions Architect to design, build, and maintain robust data pipelines using Azure cloud services.The ideal candidate will have 9-12 years of experience in designing, building, and maintaining scalable data solutions for large-scale organizations.Key...

  • Senior Snowflake Data Engineer

    1 week ago

    Pune, Maharashtra, India beBeeDataWarehouseArchitect Full time ₹ 2,14,88,000 - ₹ 2,41,33,000

    About the JobWe are seeking a highly skilled Data Warehouse Architect to lead our team in designing, implementing, and optimizing scalable data solutions using Snowflake.The ideal candidate will have hands-on experience with Snowflake Data Cloud and a deep understanding of data warehousing concepts, including structured, semi-structured, and unstructured...

  • Data Architect

    4 weeks ago

    Pune, Maharashtra, India Peoplefy Full time

    Hi,Greetings from Peoplefy Infosolutions We are hiring for one of our reputed MNC client based in Pune.We are looking for candidates with 10+ years of experience who is currently working as a Data Architect.Job Description:We are seeking a highly skilled and experienced Cloud Data Architect to design, implement, and manage scalable, secure, and efficient...

  • Chief Data Architect

    5 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About the RoleWe are seeking a seasoned Data Architect to design, develop, and implement scalable data warehouses for enterprise-scale analytics. The ideal candidate will have extensive experience in data modeling, SQL, and Google Cloud Platform (GCP) services.