Azure Databricks Engineer
4 weeks ago
Azure Databricks Engineer
About the Role :
- We are seeking a highly skilled Azure Databricks Engineer to join our team and play a vital role in developing and maintaining our big data processing pipelines.
- This role is ideal for a candidate with extensive experience in Azure data services and a strong understanding of Apache Spark for large-scale data processing.
Responsibilities :
- Design, develop, and implement efficient and scalable data processing pipelines using Apache Spark on Azure Databricks.
- Collaborate with data engineers, data scientists, and other stakeholders to understand data requirements and translate them into technical solutions.
- Leverage Azure Data Factory to orchestrate data movement and integration between various Azure data services (Data Lake, Synapse, SQL Database, etc. ).
- Utilize SQL and Python/PySpark for data manipulation, transformation, and analysis.
- Implement unit and integration tests to ensure the quality and reliability of data pipelines.
- Configure and manage Azure Databricks clusters for optimal performance and resource utilization.
- Integrate data pipelines with Azure DevOps for continuous integration and continuous delivery (CI/CD).
- Monitor and troubleshoot data pipelines to identify and resolve issues proactively.
- Stay up-to-date with the latest advancements in Azure Databricks, Apache Spark, and related technologies.
- Participate in code reviews and knowledge sharing sessions with the 5-10 years of experience in data engineering with a focus on big data processing.
- In-depth knowledge of Azure data services, including Azure Databricks, Data Factory, Data Lake, and Synapse.
- Proven experience in developing and deploying large-scale data processing pipelines using Apache Spark.
- Proficiency in SQL and Python/PySpark for data manipulation and analysis.
- Experience with CI/CD methodologies and Azure DevOps is a plus.
- Familiarity with data governance tools like Azure Purview or Collibra Unity Catalog (a plus).
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Ability to work independently and as part of a team.
-
Data Engineer
1 week ago
Bangalore, Karnataka, India WIZSTAFFING PRIVATE LIMITED Full timeJob Description :We at Captain Fresh, are building smart supply chain to deliver the highest quality seafood and meat for the Indian consumer.Our innovations in process management and workforce orchestration along with strong industry credentials are enabling us to deliver the fastest harvest-to-fork in the industry. Our endeavor is to leverage experience...
-
Data Modeler
1 month ago
Bangalore, Karnataka, India Whizz HR Full timeWe are seeking an experienced and skilled Data Modeler to join our team on a remote basis. The ideal candidate will have extensive expertise in data modeling and a strong background in utilizing Azure Data Lake and Databricks. This is an active position with a high priority, offering a competitive rate and the potential for a long-term engagement.Key...
-
Python Developer
1 month ago
Bangalore, Karnataka, India IT Full timeJob Overview :We are looking for a seasoned Python Developer with over 10 years of experience in IT, with a focus on data engineering. The ideal candidate will have extensive knowledge of Python, Pyspark, Azure Databricks, ADF, SQL, and data warehouse concepts. This role requires strong technical skills, project management capabilities, and excellent...
-
Senior Data Engineer
3 weeks ago
Bangalore, Karnataka, India GLAXOSMITHKLINE ASIA PVT. LTD Full timeA new world-leading consumer health company. Shaped by all who join us. Together, we're improving everyday health for billions of people. By growing and innovating our global portfolio of category-leading brands including Sensodyne, Panadol, Advil, Voltaren, Theraflu, Otrivin, and Centrum through a unique combination of deep human understanding and...
-
Data Engineer
2 weeks ago
Bangalore, Karnataka, India Varite India Full timeDuties :- Experience working in Azure/Databricks.- Expert in Python to create application and ETL to extract data from SharePoint, Salesforce, SQL, etc.- Experience working with APIs and integrating data from various sources into Python-based applications.- Experience in SQL along with writing complex queries.- Exposure to C# and Azure function along with...
-
Software Engineer
1 month ago
Bangalore, Karnataka, India IISC Full time- At IUDX, we are building an advanced state of the art HPC analytics processing engine. - The role involves developing a full stack analytic system including workflow/process graph compilations, workflow orchestration, distributed process execution, distributed file storage and system level performance optimization with TF/Torch/Cuda and other computing...
-
Dimiour - DevSecOps Engineer - Azure Cloud
3 weeks ago
Bangalore, Karnataka, India Dimiour Full timeRole Description : - Work within a remotely distributed DevSecOps team to plan and build a DevSecOps Platform as a Product for the self-service automation of Azure Cloud-Native infrastructure build & deployments.Key Responsibilities and Expectations :- Build automation of Infrastructure and workloads for build, test, and deploy for optimal performance and...
-
Senior DevSecOps Engineer
1 month ago
Bangalore, Karnataka, India Transformhub Full timeRequired Experience & Skills :- 5+ years of previous experience as DevOps/ DevSecOps- Proficient knowledge working with AWS, Azure security- Hands-on experience with ECS, EKS, API gateway, RDS, S3, CloudWatch, CloudFormation, IAC, Code Pipeline, Code Build, Bitbucket- Proficient Unix, PowerShell knowledge- Degree in Computer Science, IT or other equivalent...
-
Bangalore, Karnataka, India Dimiour Full timeShift : US Shift (7.30PM IST - 4.30AM IST)Job Description :- Resolve incidents, and problems affecting the operations/availability of high-profile production systems in Azure cloud.- Migrating of servers from on-prem to Azure environment- Migrating Active Directory Domain from on-prem to Azure creating Azure AD Sync for synchronizing AD users with O365.-...
-
Machine Learning Engineer
1 week ago
Bangalore, Karnataka, India WIZSTAFFING PRIVATE LIMITED Full timeJob Description :We at Captain Fresh, are building smart supply chain to deliver the highest quality seafood and meat for the Indian consumer. Our innovations in process management and workforce orchestration along with strong industry credentials are enabling us to deliver the fastest harvest-to-fork in the industry.Our endeavor is to leverage experience...
-
Machine Learning
1 month ago
Bangalore, Karnataka, India Trueblue HR Consultants Full timeAbout the job :Designation : Sr Machine Learning and Python Expert-AI : Sr Machine Learning and Python Expert are responsible for delivering high quality modern data solutions through collaboration with our team and our customers.Projects may range from short term client assessments through multiyear delivery engagements with large, blended...
-
Azure Synapse Analyst
1 month ago
Bangalore/Hyderabad, Karnataka, India Swift Strategic Staff Solutions INC Full timeNotice : Immediate Joiner.Skill : Azure Data Factory, Azure cloud, Data Lake, Azure Synapse and ETL.Job Description:Roles and Responsibilities :- The person in this position will work in a team environment to support applications using Microsoft.- Data Access Technologies.- The role of a Support Engineer within the DS group focuses on delivering high quality...
-
CI/CD Engineer
3 weeks ago
Bangalore, Karnataka, India ProPMO Services Private Limited Full timeRole : CI/CD EngineerExperience : 7+ years Relevant: 6 years must Location : Bangalore - Work From Office mandatoryEmployment type: Permanent role Automotive Engineering Service CompanyJob Description :- Strong 7+ years of IT experience in DevOps CI/CD - Review and modify CI/CD principles - Maintain CI/CD tools/platforms - Develop and maintain pipeline...
-
Software Engineering Manager
2 weeks ago
Bangalore, Karnataka, India Protoporos Staffing Services Pvt Ltd Full timeAbout :Opportunity for a role of Engineering Manager with a Enterprise B2B SaaS product firm providing Services/products to Fortune 100 organizations.The ideal candidate must be from a B2B SaaS product organization only. Title : Staff Platform Engineer/Site Reliability Engineer. Mandatory Skills : B2B SaaS Product Development, Java, AWS, 2 years of...
-
Machine Learning Engineer
3 weeks ago
Bangalore, Karnataka, India Infiniti Research Full timeAbout Quantzig : . Quantzig is a global analytics and advisory firm with offices in the US, UK, Canada, China, and India. we have assisted our clients across the globe with end-to-end advanced analytics, visual storyboarding, Machine Learning, and data engineering solutions implementation for prudent decision making. We are a rapidly growing organization...
-
Engineering Manager
1 month ago
Bangalore, Karnataka, India Connectio IT Pvt Ltd Full timeJob Description :- Lead and mentor a team of software engineers, fostering a culture of innovation, collaboration, and continuous improvement. -Provide guidance and support to ensure team members meet their performance metrics and career development goals.- Oversee the planning, execution, and delivery of engineering projects, ensuring they are completed on...
-
Staff Site Reliability Engineer
2 weeks ago
Bangalore, Karnataka, India Protoporos Staffing Services Pvt Ltd Full timeOpportunity with a leading B2B SaaS product client specializing in cutting-edge data integration solutions. Position Overview: We are seeking a highly skilled and experienced Staff Site Reliability Engineer to join our team. As a Staff SRE, you will play a critical role in ensuring the reliability, scalability, and performance of our data integration...
-
Ansible Automation Engineer
3 weeks ago
Bangalore, Karnataka, India Innova ESI Full timeAbout the Role :InnovaESI is seeking a talented and experienced Ansible Automation Engineer to join our growing IT team. In this role, you will play a key role in automating IT infrastructure and operations tasks to improve efficiency, reliability, and cost savings. You will collaborate closely with cross-functional teams to understand their needs and...
-
Data Engineer
1 month ago
Bangalore, Karnataka, India D Square Consulting Service Full timeRole : Sr. Data Engineer DOMOExperience : 5+ YearsLocation : Bangalore- Candidates who can join within 30 days, may only applyWorking Mode : Work From OfficeJob Highlights :Must-Have skills :- 5+ years' experience in ELT systems (Snowflake and DOMO)- Must have experience in digital marketing projects.- Strong knowledge of digital marketing toolsPreferred...
-
Active Directory Specialist
3 weeks ago
Bangalore, Karnataka, India Spigot Software Pvt Ltd Full timeSeeking a skilled Microsoft Entra ID Engineer to join our dynamic team. The ideal candidate will have a strong background in designing, implementing, and managing Entra ID solutions and on-premises Active Directory Domain Services. The candidate should be well-versed in Microsoft cloud technologies, particularly Azure services, and possess excellent...