Principal Big Data Engineer

4 days ago

Mumbai, Maharashtra, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000
Job Overview

The DevOps Architect plays a pivotal role in our data and analytics team, overseeing the administration, configuration, and optimization of the Databricks Lakehouse Platform.

This key position ensures platform stability, security, and performance, enabling data engineering, data science, and machine learning initiatives to thrive.

Key Responsibilities:
  1. Provision and configure Databricks workspaces, clusters, pools, and jobs across various environments.
  2. Create catalogs, schemas, access controls, and lineage configurations.
  3. Implement identity and access management using account groups, workspace-level permissions, and data-level governance.
  4. Monitor platform health, cluster utilization, job performance, and cost using Databricks admin tools and observability dashboards.
  5. Automate workspace onboarding, schema creation, user/group assignments, and external location setup using Terraform, APIs, or CLI.
  6. Integrate with Azure services like ADLS Gen2, Azure Key Vault, Azure Data Factory, and Azure Synapse.
  7. Support model serving, feature store, and MLflow lifecycle management for Data Science/ML teams.
  8. Manage secrets, tokens, and credentials securely using Databricks Secrets and integration with Azure Key Vault.
  9. Define and enforce tagging policies, data masking, and row-level access control using Unity Catalog and Attribute-Based Access Control (ABAC).
  10. Ensure compliance with enterprise policies, security standards, and audit requirements.
  11. Coordinate with Ops Architect, Cloud DevOps teams for network, authentication issues.
  12. Troubleshoot workspace, job, cluster, or permission issues for end users and data consumers.
Qualifications
  1. Databricks Certified Associate Platform Administrator or other relevant Databricks certifications.
  2. Experience with Apache Spark and data engineering concepts.
  3. Knowledge of monitoring tools (e.g., Splunk, Grafana, Cloud-native monitoring).
  4. Familiarity with data warehousing and data lake concepts.
  5. Experience with other big data technologies (e.g., Hadoop, Kafka).
  6. Previous experience leading or mentoring junior administrators.

  • Big Data Engineer

    3 days ago

    Mumbai, Maharashtra, India Progriz Coe Full time US$ 1,50,000 - US$ 2,00,000 per year

    ,,Job Title: Training & Mentorship Specialist Big Data & Cloud TechnologiesLocation: Onsite Lower Parel MumbaiEmployment Type: Full-time Role Overview:We are seeking a highly skilled Big Data & Cloud Technologies Training & Mentorship Specialist to design, deliver, and mentor teams on cutting-edge data engineering, analytics, and cloud practices. The ideal...

  • Big Data Engineer

    7 days ago

    Mumbai, Maharashtra, India beBeeSparkDeveloper Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Title: Big Data Engineer  

  • Big Data Engineer

    4 days ago

    Navi Mumbai, Maharashtra, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 25,00,000

    Enterprise Data EngineerWe are seeking a skilled and experienced Enterprise Data Engineer to design, develop, and implement efficient data models, data pipelines, and data integration solutions.This role involves collaboration with cross-functional teams to drive business growth through data-driven insights. The ideal candidate will possess strong analytical...

  • Big Data Engineer

    2 days ago

    Mumbai, Maharashtra, India beBeePySpark Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Big Data Engineer - Spark DevelopmentJob DescriptionWe are seeking a skilled Big Data Engineer with expertise in PySpark to join our team. The ideal candidate will have hands-on experience with distributed computing, data processing, and big data technologies.Key Responsibilities:

  • Senior Big Data Engineer

    4 days ago

    Mumbai, Maharashtra, India beBeeBigDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Senior Big Data EngineerJob Summary:Data Engineers design, develop and maintain large-scale data processing systems. In this role, you will work with our team to build scalable data pipelines using PySpark and ensure the smooth operation of our data infrastructure.About the Role:The ideal candidate will have strong experience in building data...

  • Big Data Developer

    4 weeks ago

    Mumbai, Maharashtra, India HirePower Staffing Solution Full time

    Position Overview: We are seeking a skilled Big Data Developer to join our growing delivery team, with a dual focus on hands-on project support and mentoring junior engineers. This role is ideal for a developer who not only thrives in a technical, fast-paced environment but is also passionate about coaching and developing the next generation of talent. You...

  • Data Engineer

    2 days ago

    Mumbai, Maharashtra, India The BIG Advantage Full time

    About client :This opening is for leading NBFC company operating pan india location headquartered in Mumbai.Professional Qualification :- Bachelor's or master's degree in computer science, Information Systems, or a related field.- Experience with cloud-based analytics platforms such as Databricks, Azure Synapse analytics, Other data specific cloud services-...

  • Cloud Engineer for Big Data Solutions

    2 days ago

    Mumbai, Maharashtra, India beBeeEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    We are seeking an experienced Cloud Engineer with 5-10 years of experience in GCP services and Big Data Analytics solutions.The ideal candidate will have hands-on experience in designing, building, and optimizing data pipelines using BigQuery, Dataflow, DataProc, and Cloud Composer.Responsibilities include implementing Big Data Analytics solutions leveraging...

  • Big Data Architect

    3 days ago

    Mumbai, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 28,00,000

    Job Description:As a seasoned data architect, you will play a pivotal role in designing and implementing large-scale data processing systems that drive business growth and innovation.The ideal candidate will possess a deep understanding of big data technologies and best practices, as well as the ability to work with large amounts of data from various...

  • Senior Big Data Professional

    2 days ago

    Mumbai, Maharashtra, India beBeeEngineer Full time US$ 1,50,000 - US$ 2,00,000

    Big Data Engineer OpportunitiesWe are seeking an experienced Senior Big Data Engineer to join our team.As a key member of our data engineering team, you will be responsible for designing, developing, and deploying large-scale data processing systems using Apache Spark, Scala, and Python.Key Responsibilities:Lead the design and development of big data...