Senior Databricks Engineer
7 days ago
About the Role We are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data ecosystems to power business insights and AI/ML initiatives. Key Responsibilities Design & - Development: - Build and maintain scalable ETL/ELT data pipelines using Apache Spark and Databricks. - Develop Delta Lake architectures and optimize data storage for performance and cost. - Automate data ingestion, transformation, and orchestration processes across multiple data sources. - Build different stages of medallion architecture such as Bronze, Silver and Gold layer Data Architecture & Modeling: - Collaborate with data architects to design data lakehouse and warehouse solutions. - Implement data quality frameworks, versioning, and governance standards (e.g., Unity Catalog). - Optimize Spark jobs, cluster configurations, and query performance. - Cloud & Infrastructure: - Work with Azure Databricks or AWS to configure and manage cloud-based data platforms. - Develop and manage CI/CD pipelines for Databricks notebooks, workflows, and jobs. - Implement security best practices for data access, encryption, and compliance. Collaboration & Leadership: - Partner with data scientists, analysts, and product teams to enable data-driven decisions. - Provide technical mentorship to junior engineers. - Contribute to strategic planning and architecture reviews. Required Qualifications - Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or related field. - 6+ years of experience in data engineering or software development. - 4+ years hands-on experience with Databricks and Apache Spark. - Strong proficiency in Python, SQL, and PySpark. - Experience with Delta Lake, Data Lakehouse, and Data Warehousing concepts. - Solid understanding of data modeling, ETL design, medallion architecture and distributed data processing. - Familiarity with CI/CD tools (Azure DevOps, GitHub Actions, Jenkins, etc.). - Cloud platform expertise in Azure, AWS, or GCP. Preferred Qualifications - Experience with machine learning pipelines in Databricks (MLflow). - Knowledge of data governance and compliance frameworks. - Experience integrating Databricks with BI tools like Power BI, Tableau, or Looker. - Certifications such as: - Databricks Certified Data Engineer Associate or Professional - Azure Data Engineer Associate - AWS Big Data Specialty Soft Skills - Strong problem-solving and analytical thinking. - Excellent communication and collaboration abilities. - Proactive mindset with ownership of deliverables. - Passion for learning emerging data technologies. Why Join Us - Work with cutting-edge data technologies on large-scale projects. - Opportunity to shape the organization’s data strategy and architecture. - Collaborative and innovative culture focused on continuous growth.
-
Senior Databricks Engineer
5 days ago
Pune, India Newscape Consulting Full timeAbout the RoleWe are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data...
-
Senior Databricks Engineer
6 days ago
pune, India Newscape Consulting Full timeAbout the Role We are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data...
-
Senior Databricks Engineer
4 days ago
Pune, India Newscape Consulting Full timeAbout the RoleWe are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data...
-
Senior Databricks Engineer
5 days ago
Pune, India Newscape Consulting Full timeAbout the Role We are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data...
-
Senior Databricks Engineer
9 hours ago
Pune, India Whatjobs IN C2 Full timeAbout the Role We are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data...
-
Senior Databricks Engineer
7 days ago
Pune, India Newscape Consulting Full timeAbout the Role We are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data...
-
Senior Databricks Engineer
7 days ago
pune, India Newscape Consulting Full timeAbout the RoleWe are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data...
-
Senior Databricks Engineer
7 days ago
Pune, India Newscape Consulting Full timeAbout the RoleWe are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data...
-
Senior Databricks Engineer
5 days ago
Pune, India Newscape Consulting Full timeAbout the Role We are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and...
-
Senior Azure Databricks Engineer
1 week ago
Pune, India TN BUSINESS SERVICES PRIVATE LIMITED Full timeAbout the job : We seeking a talented and motivated senior Azure Data Engineer. The role will be responsible for developing solutions for risk, regulatory, and compliance projects on AzureRole : Senior Azure Databricks EngineerExp : 8 to 12 yrsLocation : Pune/IndoreNotice Period : Immediate to 30 DaysMust-have skills : Azure Databricks heavyYou will play a...
