Databricks Architect

Databricks Data Engineer

Experience: 3-6 years
Location: India
Compensation: Up to 30 LPA

Key Responsibilities

• Design, develop, and maintain data pipelines and ETL workflows using Databricks and Apache Spark
• Implement Delta Lake and Medallion architecture (Bronze/Silver/Gold layers) for structured data processing
• Integrate data from multiple sources into cloud-based data lakes and warehouses
• Collaborate with analytics teams to enable self-service BI and reporting
• Optimize Spark jobs for performance and cost efficiency
• Ensure data quality, governance, and security in pipelines

Required Skills

• Strong programming in Python or Scala
• Hands-on expertise with Databricks and Apache Spark
• Experience with ETL/ELT pipelines, Delta Lake, and ADF (Azure Data Factory)
• Exposure to AWS, Azure, or GCP cloud services
• Knowledge of SQL and data warehousing concepts
• Bonus: Databricks Certification (Associate level preferred)

Tech Stack

• Languages: Python, Scala, SQL
• Frameworks/Platforms: Apache Spark, Databricks, Delta Lake
• ETL & Orchestration: Azure Data Factory, Airflow
• Cloud: Azure (preferred), AWS, GCP
• DevOps Tools: Git, CI/CD pipelines
• Databases: SQL Server, Snowflake, Redshift, Synapse

Databricks Lead Engineer

Experience: 610 years
Location: India
Compensation: Up to 43 LPA

Key Responsibilities

• Lead design and implementation of scalable, high-performance data solutions on Databricks
• Architect and optimize data pipelines and workflows leveraging Spark and Delta Lake
• Mentor and guide junior data engineers in best practices and coding standards
• Collaborate with data scientists, analysts, and business teams to deliver solutions
• Manage and optimize Databricks environments including clusters, security, and notebooks
• Implement CI/CD pipelines and infrastructure as code (Terraform) for data projects
• Drive cost optimization, performance tuning, and automation initiatives

Required Skills

• Strong proficiency in Python/Scala, Spark, and SQL
• Extensive experience with Databricks, Delta Lake, and Medallion architecture
• Knowledge of ADF, Iceberg, and orchestration frameworks
• Hands-on expertise with cloud platforms (Azure preferred, AWS/GCP)
• Familiarity with MLOps, ML model integration, and deployment on Databricks
• Databricks Professional certification preferred

Tech Stack

• Languages: Python, Scala, SQL, Java (optional)
• Frameworks/Platforms: Apache Spark, Databricks, Delta Lake, Iceberg
• ETL & Orchestration: Azure Data Factory, Airflow, Synapse Pipelines
• Cloud: Azure (primary), AWS, GCP
• DevOps/IaC: GitHub Actions, Jenkins, Terraform, Docker
• Databases: Snowflake, Synapse, BigQuery, Redshift
• ML/AI Tools: MLflow, Databricks ML, TensorFlow (basic integration)

Databricks Architect

Experience: 10–14 years
Location: India
Compensation: Up to 70 LPA

Key Responsibilities

• Own and define the enterprise-level data architecture strategy using Databricks
• Design, build, and scale end-to-end data platforms integrating structured/unstructured data
• Implement and enforce Medallion architecture standards across projects
• Guide teams in data lakehouse design, governance, and security frameworks
• Provide leadership on AI/ML integration, model deployment, and advanced analytics
• Manage complex multi-cloud and hybrid deployments on Databricks
• Establish best practices, reusable frameworks, and automation strategies for data engineering

Required Skills

• Expert-level knowledge of Databricks, Apache Spark, Delta Lake, Iceberg
• Strong programming and solutioning in Python, Scala, SQL
• Proven expertise in Azure Data Factory, pipelines, orchestration, and governance
• Experience with multi-cloud environments (Azure/AWS/GCP)
• Proficiency with Terraform, CI/CD, containerization (Docker, Kubernetes)
• Deep understanding of machine learning, deep learning, and MLOps frameworks
• Excellent leadership, architectural thinking, and problem-solving skills
• Databricks Architect or Professional certification highly preferred

Tech Stack

• Languages: Python, Scala, SQL, R (for advanced analytics)
• Frameworks/Platforms: Apache Spark, Databricks, Delta Lake, Iceberg, Unity Catalog
• ETL & Orchestration: Azure Data Factory, Airflow, Synapse, Dataflow
• Cloud: Multi-cloud (Azure, AWS, GCP)
• DevOps/IaC: Terraform, Jenkins, GitHub Actions, Docker, Kubernetes
• Databases/Warehouses: Snowflake, Synapse, BigQuery, Redshift, Lakehouse architectures
• ML/AI Tools: MLflow, Databricks ML, TensorFlow, PyTorch, Hugging Face
• Security/Governance: Unity Catalog, RBAC, data encryption