Databricks Platform Engineer
14 hours ago
Experience:3 to 6 years Notice Period:Immediate to 15 days Location:Bangalore Work from officeJOB DESCRIPTIONJob Description: Databricks Platform Engineer / Administrator (AWS) Department: Data Engineering / Cloud Platform / DevOpsAbout the Role We are looking for a skilled and proactiveDatabricks Platform Engineer / Administratorto manage, optimize, and govern our Databricks environment on AWS. The ideal candidate will be responsible for overseeing platform operations, optimizing performance and cost, and ensuring secure and efficient use of Databricks and integrated AWS services. This role sits at the intersection of platform engineering, data infrastructure, and cost governance, requiring both technical depth and cross-functional collaboration.Key Responsibilities Platform Administration Deploy, configure, and manageDatabricks workspaceson AWS. Set up and maintainAWS infrastructure(VPCs, IAM roles, S3, security groups) to support Databricks operations. Manage user access, job permissions, cluster configurations, and workspace-level controls. Cost Optimization Implement and enforcecluster policiesto control resource consumption and prevent cost overruns. Promote efficient compute usage by enablingjob clusters,auto-termination, andspot instancestrategies. Monitor and analyze cost usingAWS Cost Explorer,Databricks REST APIs, and internal dashboards. Optimize storage costs throughS3 lifecycle policies,Delta Lake optimization, andtable version management. Performance Tuning Analyze and tuneSpark job performance, including partitioning, caching, and join strategies. Provide guidance on usingPhoton runtime,Delta Live Tables, andSQL optimization best practices. Collaborate with data engineers and analytics teams to improve job execution efficiency and pipeline throughput. Monitoring & Observability Set up and managelogging, monitoring, and alertingvia CloudWatch, Databricks audit logs, and external tools. Track metrics such as cluster utilization, job failures, job durations, and resource bottlenecks. Develop dashboards for real-time visibility into platform usage, performance, and cost. Security & Governance Enforce access control throughUnity Catalog,IAM roles, andcluster policies. Ensure compliance with enterprise security policies, data governance standards, and audit requirements. Prevent data duplication and misuse through strong access boundaries and usage monitoring. Automation & Documentation Automate provisioning and configuration usingTerraform,Databricks CLI, andREST APIs. Maintain detailed documentation of cluster policies, platform standards, optimization playbooks, and cost governance guidelines. Drive platform standardization and advocate for best practices across teams.Qualifications Required 3–6+ years of experience managingDatabricks on AWSor similar Spark-based platforms. Strong hands-on experience withAWS services(S3, IAM, VPC, EC2, CloudWatch). Proficiency inSpark performance tuningandDatabricks architecture. Solid understanding ofcost optimization strategiesin cloud-based data platforms. Experience withDatabricks cluster policies,job scheduling, andDelta Lake. Familiarity withPython, SQL, Terraform, andDatabricks REST APIs. Preferred Experience withUnity Catalog,Delta Live Tables, orMLflow. Knowledge ofAWS cost governance tools(Cost Explorer, Budgets, Trusted Advisor). Prior work in FinOps, DataOps, or Cloud Center of Excellence (CCoE) roles. Certifications such as: Databricks Certified Data Engineer / Admin AWS Certified Solutions Architect / DevOps Engineer
-
2892- Databricks Engineer
2 days ago
New Delhi, India EXL Full timeAbout the RoleWe are seeking a highly skilled Databricks Engineer with 5–7 years of experience in data engineering and analytics. The ideal candidate will have strong expertise in designing, developing, and optimizing large-scale data pipelines and solutions using Databricks, Spark, and cloud platforms. This role requires a combination of strong technical...
-
Databricks Data Engineer
2 days ago
New Delhi, India Tata Consultancy Services Full timeAbout the RoleThe Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.Experience- 3 to 6 yearsLocation - Kolkata /Chennai/TrivandrumRequired Technical Skill Set ------ Databricks, AWS, Python & PySparkGood to have- Experience in AWS platform (S3, EC2,...
-
Lead Data Platform Optimization
2 days ago
New Delhi, India FinOpsly Full timeFinOpsly is the AI brain behind cost decisions across cloud, data, and AI — built to think, plan, and act. Purpose-built for today’s complex enterprise environments, we unify FinOps across AWS, Azure, Snowflake, Databricks, and modern AI workloads — enabling organizations to move from cost reporting to cost action.We are building a differentiated...
-
Databricks Engineer
13 hours ago
New Delhi, India Persistent Systems Full timeAbout Position:We are seeking data bricks engineer with hands on experience in Power BI, python, spark and data bricks.Role: Databricks Engineer Location: Pune & Mumbai Experience: 8 to 12 Years Job Type: Full Time EmploymentWhat You'll Do:3+ years of experience in data engineering or analytics roles. Strong hands-on experience withDatabricksandApache Spark...
-
Databricks Engineer
4 days ago
New Delhi, India Tata Consultancy Services Full timeRole: Databricks Engineer Location: Hyd, Noida, Chennai, Mumbai Experience: 5 to 10 Years Job Description: Working on a scrum team as a full stack engineer. Design and develop reusable software modules that meet customer requirements while upholding high standards of reliability, security, maintainability, and performance. Assist in defining product...
-
Architect : Big Data
2 weeks ago
Delhi, India Databricks Full timeAs a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...
-
Delhi, India Databricks Full timeAs a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...
-
Databricks Engineer
15 hours ago
New Delhi, India KPG99 INC Full timeDatabricks EngineerLocation: REMOTEDuration: 12 months with extensionsOpenings: 3REQUIRED SKILLS AND EXPERIENCE- 3–5 years of experience in data engineering roles- Strong hands-on experience with Databricks for data processing and pipeline development.- Proficiency in SQL for data querying, transformation, and troubleshooting.- Solid programming skills in...
-
Databricks Admin
2 days ago
New Delhi, India EverestDX Inc Full timeAbout the Company:Everest DX – We are a Digital Platform Services company, headquartered in Stamford. Our Platform/Solution includes Orchestration, Intelligent operations with BOTs’, AI-powered analytics for Enterprise IT. Our vision is to enable Digital Transformation for enterprises to deliver seamless customer experience, business efficiency and...
-
Databricks Admin
4 days ago
New Delhi, India EverestDX Inc Full timeAbout the Company: Everest DX – We are a Digital Platform Services company, headquartered in Stamford. Our Platform/Solution includes Orchestration, Intelligent operations with BOTs’, AI-powered analytics for Enterprise IT. Our vision is to enable Digital Transformation for enterprises to deliver seamless customer experience, business efficiency and...
