Azure Databricks Engineer

4 weeks ago


Bangalore, Karnataka, India ARR Recruitment Solutions Full time

Azure Databricks Engineer


About the Role :

- We are seeking a highly skilled Azure Databricks Engineer to join our team and play a vital role in developing and maintaining our big data processing pipelines.

- This role is ideal for a candidate with extensive experience in Azure data services and a strong understanding of Apache Spark for large-scale data processing.

Responsibilities :

- Design, develop, and implement efficient and scalable data processing pipelines using Apache Spark on Azure Databricks.

- Collaborate with data engineers, data scientists, and other stakeholders to understand data requirements and translate them into technical solutions.

- Leverage Azure Data Factory to orchestrate data movement and integration between various Azure data services (Data Lake, Synapse, SQL Database, etc. ).

- Utilize SQL and Python/PySpark for data manipulation, transformation, and analysis.

- Implement unit and integration tests to ensure the quality and reliability of data pipelines.

- Configure and manage Azure Databricks clusters for optimal performance and resource utilization.

- Integrate data pipelines with Azure DevOps for continuous integration and continuous delivery (CI/CD).

- Monitor and troubleshoot data pipelines to identify and resolve issues proactively.

- Stay up-to-date with the latest advancements in Azure Databricks, Apache Spark, and related technologies.

- Participate in code reviews and knowledge sharing sessions with the 5-10 years of experience in data engineering with a focus on big data processing.

- In-depth knowledge of Azure data services, including Azure Databricks, Data Factory, Data Lake, and Synapse.

- Proven experience in developing and deploying large-scale data processing pipelines using Apache Spark.

- Proficiency in SQL and Python/PySpark for data manipulation and analysis.

- Experience with CI/CD methodologies and Azure DevOps is a plus.

- Familiarity with data governance tools like Azure Purview or Collibra Unity Catalog (a plus).

- Excellent problem-solving and analytical skills.

- Strong communication and collaboration skills.

- Ability to work independently and as part of a team.

(ref:hirist.tech)
  • Data Engineer

    1 week ago


    Bangalore, Karnataka, India WIZSTAFFING PRIVATE LIMITED Full time

    Job Description :We at Captain Fresh, are building smart supply chain to deliver the highest quality seafood and meat for the Indian consumer.Our innovations in process management and workforce orchestration along with strong industry credentials are enabling us to deliver the fastest harvest-to-fork in the industry. Our endeavor is to leverage experience...

  • Data Modeler

    1 month ago


    Bangalore, Karnataka, India Whizz HR Full time

    We are seeking an experienced and skilled Data Modeler to join our team on a remote basis. The ideal candidate will have extensive expertise in data modeling and a strong background in utilizing Azure Data Lake and Databricks. This is an active position with a high priority, offering a competitive rate and the potential for a long-term engagement.Key...

  • Python Developer

    1 month ago


    Bangalore, Karnataka, India IT Full time

    Job Overview :We are looking for a seasoned Python Developer with over 10 years of experience in IT, with a focus on data engineering. The ideal candidate will have extensive knowledge of Python, Pyspark, Azure Databricks, ADF, SQL, and data warehouse concepts. This role requires strong technical skills, project management capabilities, and excellent...

  • Senior Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India GLAXOSMITHKLINE ASIA PVT. LTD Full time

    A new world-leading consumer health company. Shaped by all who join us. Together, we're improving everyday health for billions of people. By growing and innovating our global portfolio of category-leading brands including Sensodyne, Panadol, Advil, Voltaren, Theraflu, Otrivin, and Centrum through a unique combination of deep human understanding and...

  • Data Engineer

    2 weeks ago


    Bangalore, Karnataka, India Varite India Full time

    Duties :- Experience working in Azure/Databricks.- Expert in Python to create application and ETL to extract data from SharePoint, Salesforce, SQL, etc.- Experience working with APIs and integrating data from various sources into Python-based applications.- Experience in SQL along with writing complex queries.- Exposure to C# and Azure function along with...

  • Software Engineer

    1 month ago


    Bangalore, Karnataka, India IISC Full time

    - At IUDX, we are building an advanced state of the art HPC analytics processing engine. - The role involves developing a full stack analytic system including workflow/process graph compilations, workflow orchestration, distributed process execution, distributed file storage and system level performance optimization with TF/Torch/Cuda and other computing...


  • Bangalore, Karnataka, India Dimiour Full time

    Role Description : - Work within a remotely distributed DevSecOps team to plan and build a DevSecOps Platform as a Product for the self-service automation of Azure Cloud-Native infrastructure build & deployments.Key Responsibilities and Expectations :- Build automation of Infrastructure and workloads for build, test, and deploy for optimal performance and...


  • Bangalore, Karnataka, India Transformhub Full time

    Required Experience & Skills :- 5+ years of previous experience as DevOps/ DevSecOps- Proficient knowledge working with AWS, Azure security- Hands-on experience with ECS, EKS, API gateway, RDS, S3, CloudWatch, CloudFormation, IAC, Code Pipeline, Code Build, Bitbucket- Proficient Unix, PowerShell knowledge- Degree in Computer Science, IT or other equivalent...


  • Bangalore, Karnataka, India Dimiour Full time

    Shift : US Shift (7.30PM IST - 4.30AM IST)Job Description :- Resolve incidents, and problems affecting the operations/availability of high-profile production systems in Azure cloud.- Migrating of servers from on-prem to Azure environment- Migrating Active Directory Domain from on-prem to Azure creating Azure AD Sync for synchronizing AD users with O365.-...


  • Bangalore, Karnataka, India WIZSTAFFING PRIVATE LIMITED Full time

    Job Description :We at Captain Fresh, are building smart supply chain to deliver the highest quality seafood and meat for the Indian consumer. Our innovations in process management and workforce orchestration along with strong industry credentials are enabling us to deliver the fastest harvest-to-fork in the industry.Our endeavor is to leverage experience...

  • Machine Learning

    1 month ago


    Bangalore, Karnataka, India Trueblue HR Consultants Full time

    About the job :Designation : Sr Machine Learning and Python Expert-AI : Sr Machine Learning and Python Expert are responsible for delivering high quality modern data solutions through collaboration with our team and our customers.Projects may range from short term client assessments through multiyear delivery engagements with large, blended...

  • Azure Synapse Analyst

    1 month ago


    Bangalore/Hyderabad, Karnataka, India Swift Strategic Staff Solutions INC Full time

    Notice : Immediate Joiner.Skill : Azure Data Factory, Azure cloud, Data Lake, Azure Synapse and ETL.Job Description:Roles and Responsibilities :- The person in this position will work in a team environment to support applications using Microsoft.- Data Access Technologies.- The role of a Support Engineer within the DS group focuses on delivering high quality...

  • CI/CD Engineer

    3 weeks ago


    Bangalore, Karnataka, India ProPMO Services Private Limited Full time

    Role : CI/CD EngineerExperience : 7+ years Relevant: 6 years must Location : Bangalore - Work From Office mandatoryEmployment type: Permanent role Automotive Engineering Service CompanyJob Description :- Strong 7+ years of IT experience in DevOps CI/CD - Review and modify CI/CD principles - Maintain CI/CD tools/platforms - Develop and maintain pipeline...


  • Bangalore, Karnataka, India Protoporos Staffing Services Pvt Ltd Full time

    About :Opportunity for a role of Engineering Manager with a Enterprise B2B SaaS product firm providing Services/products to Fortune 100 organizations.The ideal candidate must be from a B2B SaaS product organization only. Title : Staff Platform Engineer/Site Reliability Engineer. Mandatory Skills : B2B SaaS Product Development, Java, AWS, 2 years of...


  • Bangalore, Karnataka, India Infiniti Research Full time

    About Quantzig : . Quantzig is a global analytics and advisory firm with offices in the US, UK, Canada, China, and India. we have assisted our clients across the globe with end-to-end advanced analytics, visual storyboarding, Machine Learning, and data engineering solutions implementation for prudent decision making. We are a rapidly growing organization...

  • Engineering Manager

    1 month ago


    Bangalore, Karnataka, India Connectio IT Pvt Ltd Full time

    Job Description :- Lead and mentor a team of software engineers, fostering a culture of innovation, collaboration, and continuous improvement. -Provide guidance and support to ensure team members meet their performance metrics and career development goals.- Oversee the planning, execution, and delivery of engineering projects, ensuring they are completed on...


  • Bangalore, Karnataka, India Protoporos Staffing Services Pvt Ltd Full time

    Opportunity with a leading B2B SaaS product client specializing in cutting-edge data integration solutions. Position Overview: We are seeking a highly skilled and experienced Staff Site Reliability Engineer to join our team. As a Staff SRE, you will play a critical role in ensuring the reliability, scalability, and performance of our data integration...


  • Bangalore, Karnataka, India Innova ESI Full time

    About the Role :InnovaESI is seeking a talented and experienced Ansible Automation Engineer to join our growing IT team. In this role, you will play a key role in automating IT infrastructure and operations tasks to improve efficiency, reliability, and cost savings. You will collaborate closely with cross-functional teams to understand their needs and...

  • Data Engineer

    1 month ago


    Bangalore, Karnataka, India D Square Consulting Service Full time

    Role : Sr. Data Engineer DOMOExperience : 5+ YearsLocation : Bangalore- Candidates who can join within 30 days, may only applyWorking Mode : Work From OfficeJob Highlights :Must-Have skills :- 5+ years' experience in ELT systems (Snowflake and DOMO)- Must have experience in digital marketing projects.- Strong knowledge of digital marketing toolsPreferred...


  • Bangalore, Karnataka, India Spigot Software Pvt Ltd Full time

    Seeking a skilled Microsoft Entra ID Engineer to join our dynamic team. The ideal candidate will have a strong background in designing, implementing, and managing Entra ID solutions and on-premises Active Directory Domain Services. The candidate should be well-versed in Microsoft cloud technologies, particularly Azure services, and possess excellent...