Spendcube - Databricks Lead

4 weeks ago


india Marlabs LLC Full time

Description:

Job Title: Technical Lead - Azure Databricks

Location: Bangalore, India

Position Overview:

We are looking for an experienced Technical Lead with 8 to 10 years of hands-on experience working on Azure Databricks and Azure native services. The Technical Lead will play a critical role in managing and driving day-to-day data engineering work and leads the implementation of medallian architecture. Reports status, risks, and issues to technical project manager. Responsible for Development with adherence various best practices and coding guidelines.

Key Responsibilities:

Lead a team of data engineers in the design, development, and implementation of cloud-based data solutions using Azure Databricks and Azure native services. Collaborate with cross-functional teams to define data requirements, develop technical specifications, and architect scalable and efficient data pipelines. Design and implement data models, ETL processes, and data integration solutions to support business objectives and ensure data quality and integrity. Optimize data architecture for performance, scalability, and cost-effectiveness, leveraging cloud-native technologies and best practices. Provide technical leadership and mentorship to team members, guiding them in the adoption of best practices, tools, and methodologies. Stay current with emerging technologies and trends in data engineering and cloud computing and evaluate opportunities for innovation and optimization. Establish and maintain documentation for data architecture, including data models, data lineage, and system configurations.

Qualifications:

Bachelor’s or master’s degree in computer science, Engineering, or a related field. 8 to 10 years of experience working as a Data Engineer or a similar role, with a focus on cloud-based data platforms and services, particularly Azure Databricks and Azure native services. Extensive hands-on experience designing and implementing data solutions using Azure cloud services, including Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Azure Cosmos DB, and Azure Data Lake Storage. Strong proficiency in SQL, Python, and Spark, with advanced knowledge of data modelling techniques, dimensional modelling, and data warehousing concepts. In-depth understanding of data governance, security, and compliance frameworks, with experience implementing security controls and encryption mechanisms in cloud environments. Excellent leadership and communication skills, with the ability to effectively communicate technical concepts and strategies to stakeholders at all levels. Proven ability to lead cross-functional teams, drive consensus, and achieve project goals in a dynamic and fast-paced environment. Relevant certifications such as Microsoft Certified: Azure Solutions Architect Expert or Microsoft Certified: Azure Data Engineer Associate are highly desirable.

  • india Impetus Full time

    Hands on experience in deploying Production grade Databricks cluster Hands on experience in Shell scripting/Perl/Python/Hive/SQL Sound UNIX abilities involving user management, security & permissions Sound knowledge about the Hardware (CPU, memory, disk, network, Firewalls etc.) Sound knowledge about the clustered deployment Architecture Exposure to cloud...


  • india Infogain Full time

    Position: Lead - Azure Databricks Overall IT experience: 9 to 13 years Primary Skills: Azure Databricks, Hadoop, Hive, Pig, Impala, ETL, Data warehouse Secondary Skills: U-SQL, Spark, Scala, Python Work Location: Bengaluru, Pune, Mumbai, Noida, Gurugram Mode of work: Hybrid at Infogain office locations mentioned above Notice Period: 0 to 30 Days...

  • Data Engineering Lead

    2 weeks ago


    india QBrainX Full time

    Job Title : Data Engineering Lead (With Architecture Experience) Exp level : 10+ years Location : Coimbatore Position Type : Full-Time No of opening : 2- Immediate Joiner (OR) Who can join us within 15 to 20 days is preferred.Key Responsibilities : Architect and Design Data Solutions : - Collaborate with cross-functional teams to understand business...

  • Manager -Databricks

    2 months ago


    india Tredence Full time

    About Tredence Tredence is a global analytics services and solutions company. We are one of the fastest growing private companies in the country for three straight years according to the Inc. 5000 and we continue to set ourselves apart from our competitors by attracting the greatest talent in the data analytics and data science space. Our...

  • Azure Data Lead

    4 weeks ago


    Anywhere in India/Multiple Locations, IN Etaash Consulting Full time

    Years of experience : 7 to 15 Years Role : Sr. Tech LeadJob Description :- Experience in Perform Design, Development & Deployment using Azure Services (Databricks, PySpark, SQL, Data Factory,)- Develop and maintain scalable data pipelines and build new Data Source integrations to support increasing data volume and complexity.- Experience in creating...

  • Azure Data Lead

    3 weeks ago


    Anywhere in India/Multiple Locations Etaash Consulting Full time

    Years of experience : 7 to 15 Years Role : Sr. Tech Lead Job Description : - Experience in Perform Design, Development & Deployment using Azure Services (Databricks, PySpark, SQL, Data Factory,)- Develop and maintain scalable data pipelines and build new Data Source integrations to support increasing data volume and complexity.- Experience in...


  • india LTIMindtree Full time

    Job Title: Specialist-Data Engineering Experience Range: 5.5 to 10 years Hiring Locations: Bangalore, Hyderabad, Chennai, Kolkata, Noida, Pune, Mumbai Target Notice Period: Immediate Joiner Key Skills to Look for: Databricks, Data Lake, Pyspark with Unity catalog Databricks -Module Lead -Job Description The individual will be working in Unity Catalog...

  • Lead Consultant

    1 week ago


    india Genpact Full time

    Responsibilities • Responsible for development based on client prioritization and approval • Support client technical architects in conducting PoCs , including cost evaluations, of various AWS services and other tools • Develop data ingress frameworks using the AWS tech stack like AWS Glue, Athena, Databricks, Python, Pyspark, SQL • Design and...

  • Senior Engineer

    6 days ago


    india Bristlecone Full time

    ABOUT US : Bristlecone is the leading provider of AI-powered application transformation services for the connected supply chain. We empower our customers with speed, visibility, automation, and resiliency – to thrive on change. Our transformative solutions in Digital Logistics, Cognitive Manufacturing, Autonomous Planning, Smart Procurement and...

  • Senior Engineer

    6 days ago


    India Bristlecone Full time

    ABOUT US:Bristlecone is the leading provider of AI-powered application transformation services for the connected supply chain. We empower our customers with speed, visibility, automation, and resiliency – to thrive on change.Our transformative solutions in Digital Logistics, Cognitive Manufacturing, Autonomous Planning, Smart Procurement and Digitalization...

  • Databricks Engineer

    2 weeks ago


    india Futureleap search partners LLP Full time

    Main Skills - Azure Databricks, PySpark, SQL and Azure Cloud. Role : Senior Databricks Engineer / Databricks Technical Lead/ Data ArchitectExperience : 8-15 yearsLocation : Bangalore, Chennai, Delhi, PuneAbout Company : We focus on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science...

  • Data Engineer

    2 months ago


    India VSquare Systems Pvt. Ltd. Full time

    Job Title: Data Engineer (Azure Databricks) Location: Bangalore / HybridJob Type: Full-timeJob Description:Experienced Data engineer & Solutions designer with a deep understanding of data solution design and implementation, seeking a challenging role to leverage my expertise in data architecture, cloud migrations, and solution design, aiming to drive...

  • Data Engineer

    2 months ago


    India VSquare Systems Pvt. Ltd. Full time

    Job Title: Data Engineer (Azure Databricks) Location: Bangalore / Hybrid Job Type: Full-time Job Description: Experienced Data engineer & Solutions designer with a deep understanding of data solution design and implementation, seeking a challenging role to leverage my expertise in data architecture, cloud migrations, and solution design, aiming to drive...


  • india Marlabs LLC Full time

    Description: Job Title: Data Architect - Azure Databricks Location: Bangalore, India Position Overview: We are looking for a Data Architect with 10 to 12 years of experience specializing in Azure Databricks and Azure native services. The ideal candidate will have a proven track record of designing and implementing...


  • india KARYA CONSULTANTS PRIVATE LIMITED Full time

    Responsibilities :- Closely work with Architect and lead to design solutions to meet functional and non-functional- requirements.- Participate to understand architecture and solution design- artifacts.- Evangelize re-use through the implementation of shared- assets.- Proactively implement engineering methodologies, standards, and leading practices.- Provide...

  • Lead MLOps Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations Teksands.ai Full time

    Job Description : We are looking for Lead ML Ops Engineers with experience on Azure services like Databricks, ML, Dev Ops and who can support Development, Deployment, Model Management and Monitoring for different clients.Desired Skills and Qualifications : - 8+ of experience in understanding AI/ML projects and exposure to supporting the Model execution on...

  • Lead MLOps Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations, IN Teksands.ai Full time

    Job Description :We are looking for Lead ML Ops Engineers with experience on Azure services like Databricks, ML, Dev Ops and who can support Development, Deployment, Model Management and Monitoring for different clients.Desired Skills and Qualifications :- 8+ of experience in understanding AI/ML projects and exposure to supporting the Model execution on...


  • india Tredence Full time

    About Tredence Tredence is a global analytics services and solutions company. We are one of the fastest growing private companies in the country for three straight years according to the Inc. 5000 and we continue to set ourselves apart from our competitors by attracting the greatest talent in the data analytics and data science space. Our...


  • india Symphoni HR Full time

    Job Description : The ideal candidate for this role would be a seasoned IT professional with extensive experience in data and analytics program leadership and technical delivery. They should have a track record of successfully leading large-scale legacy EDW/data platform modernization and migration projects to cloud platforms like Snowflake, Databricks,...


  • India Marlabs Full time

    **Date**:Apr 30, 2024**Location**:Any Marlabs Office Location, IN**Company**:Marlabs Innovations Pvt Ltd**Description**: **_Job Title: Data Architect - Azure Databricks_** **_Location: Bangalore, India_** - Position Overview:- Key Responsibilities: - Lead the design and implementation of our cloud-based data architecture using Azure Databricks and Azure...