Azure Data Engineer

1 month ago

Bengaluru, India Gainwell Technologies LLC Full time

Summary

As an Azure Data Engineer at Gainwell, you will be a part of an innovative healthcare technology company with a mission to serve the healthcare needs of our communities.

Primary skills:

Candidates should possess skills in

  • ADF pipeline set up and management
  • SQL programming (MS SQL Server – SSMS)
  • Issue Resolution & Troubleshooting in Azure environments
  • Testing and validations
  • Automation activities
  • Set up ETL activities in ADF framework
  • Identify process efficiencies opportunities

Secondary skills:

  • Analyze logs and set up monitoring mechanisms
  • Security & Access Management
  • Problem solving and root cause analysis

Candidate should have expertise in:

  • Azure Data Factory, Azure Data Lake, and Azure SQL Database
  • Azure DevOps (for managing CI/CD processes, including pipeline setup, tracking, and performance monitoring)
  • Working knowledge of data integration patterns and ETL processes

Key Responsibilities:

Data Pipeline Management:

  • Build and execute ADF pipelines for continuous integration/continuous deployment (CI/CD) processes.
  • Set up and configure multi-stage pipelines and various environment configurations for development, testing, and production.
  • Track pipeline performance and ensure the health of applications, taking corrective actions when needed.
  • Design processes for monitoring pipeline activities and system health.
  • Integrate data from multiple sources, including SQL databases and third-party APIs.
  • Develop data integrations between source and destination systems to support operational and reporting needs.
  • Build infrastructure for data extraction, transformation, and loading (ETL) using MS SQL and Azure technologies.
  • Identify opportunities for automating tasks within the ADF framework and execute enhancements to improve efficiency.
  • Set up Azure Monitor to track performance metrics, logs, and alerts for Azure resources.

SQL Operations:

  • Run SQL programs within Azure environments, optimizing for performance and data accuracy.
  • Work with data in relational databases and assist with query optimization and troubleshooting.

Issue Resolution & Troubleshooting:

  • Perform root cause analysis and debugging to address issues related to application performance.
  • Coordinate with the Azure IT team to troubleshoot memory leaks, inefficient memory usage, and other performance bottlenecks in applications.
  • Provide rapid response to resolve critical system errors and ensure minimal downtime.

Logs & Monitoring:

  • Create and manage rules and alerts for system monitoring.
  • Analyze logs from various Azure resources to detect and resolve performance issues.
  • Set up comprehensive logging and monitoring solutions for CI/CD processes to ensure system reliability.

Testing & Validation:

  • Conduct thorough testing and validation of pipelines and infrastructure before production deployment.
  • Collaborate with the IT and development teams to perform functional, performance, and regression testing.

Security & Access Management:

  • Coordinate with IT teams to manage access control, granting or revoking access as needed and raising tickets for user management.
  • Maintain documentation of current pipelines, refresh processes, and security configurations.
  • Coordinate and set up various security levels in reports and applications to ensure data privacy and compliance with security protocols.
  • Assist in the review and audit of Azure infrastructure architecture, providing insights for improvements and optimizations.
  • Work on creating a unified strategy for Azure cloud transformation, covering governance, automation, security, and operational aspects.

Other Responsibilities:

  • Assist in the creation and execution of monthly refresh plans to keep data infrastructure updated.
  • Maintain up-to-date documentation for all data pipelines, processes, and operational tasks.

What We’re Looking For:

The ideal candidate should have the following qualifications:

  • Bachelor's degree in Computer Science, Information Technology, or a related field.
  • Azure certifications such as Azure Data Engineer Associate (or equivalent certifications).
  • Working knowledge of Azure DevOps for managing CI/CD processes, including pipeline setup, tracking, and performance monitoring.
  • Proficiency in Power BI for reporting and data visualization.
  • Adequate experience in Azure Data Factory, Azure Data Lake, and Azure SQL Database.
  • Strong working knowledge of data integration patterns and ETL processes for seamless data transfer and transformations.
  • Familiarity with Azure Log Analytics to query and analyze logs from various Azure resources, providing insight into performance and system health.
  • Good understanding of Azure environments, infrastructure, and resource management.

Soft Skills:

  • Good verbal and written communication skills to effectively collaborate with cross-functional teams and business stakeholders.
  • Attention to detail and a proactive approach to identifying and addressing potential issues.
  • Self-driven with the ability to work independently and manage priorities efficiently.

Other Requirements:

  • Ability to work on evening shift to maximize overlap with US teams, ensuring efficient communication and collaboration.
  • Eye for details in monitoring, troubleshooting, and pipeline execution.

 

  • Azure data engineer

    1 month ago

    Bengaluru, India Recro Full time

    Job Title: Azure Data EngineerLocation: RemoteExperience Required: 3+ YearsJob Overview:We seek a skilled Azure Data Engineer with 3+ years of experience, specializing in SQL, Data Modeling, and Azure Data Engineering tools. You will be responsible for building and enhancing data models, managing data pipelines with Azure Data Factory, and working with...

  • Azure Data Engineer

    6 months ago

    Bengaluru, India Torry Harris Full time

    Are you an experienced data engineer looking for a new challenge? If so, Torry Harris is seeking a talented Azure Data Engineer to join our team and help drive our data engineering efforts to the next level.As an Azure Data Engineer at Torry Harris, you will be responsible for designing, implementing, and maintaining cloud-based data solutions using Azure...

  • Azure Data Engineer

    3 months ago

    Bengaluru, India Recro Full time

    Job Title:Azure Data EngineerLocation:RemoteExperience Required:3+ YearsJob Overview:We seek a skilled Azure Data Engineer with 3+ years of experience, specializing in SQL, Data Modeling, and Azure Data Engineering tools. You will be responsible for building and enhancing data models, managing data pipelines with Azure Data Factory, and working with Azure...

  • Azure Data Engineer

    3 months ago

    Bengaluru, India Recro Full time

    Job Title: Azure Data Engineer Location: Remote Experience Required: 3+ Years Job Overview:We seek a skilled Azure Data Engineer with 3+ years of experience, specializing in SQL, Data Modeling, and Azure Data Engineering tools. You will be responsible for building and enhancing data models, managing data pipelines with Azure Data Factory, and working with...

  • Azure Data Engineer

    3 months ago

    Bengaluru, India Recro Full time

    Job Title: Azure Data Engineer Location: Remote Experience Required: 3+ Years Job Overview: We seek a skilled Azure Data Engineer with 3+ years of experience, specializing in SQL, Data Modeling, and Azure Data Engineering tools. You will be responsible for building and enhancing data models, managing data pipelines with Azure Data Factory, and working...

  • Azure data engineer

    4 days ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TCS!!!**TCS is Hiring for Azure Databricks Data Engineer **Walkin Interview for Azure Databricks Data Engineer in BangaloreWalkin Interview Date: 14th Dec2024(Saturday)Role: Azure Databricks Data EngineerDesired Experience: 5- 10 YearsJob Location: BangaloreJob Description:· Implementation, and operations of OLTP, OLAP, DW technologies such...

  • Azure Data Engineer

    6 days ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TCS!!!**TCS is Hiring for Azure Databricks Data Engineer **Walkin Interview for Azure Databricks Data Engineer in BangaloreWalkin Interview Date: 14th Dec2024(Saturday)Role: Azure Databricks Data EngineerDesired Experience: 5- 10 YearsJob Location: BangaloreJob Description:· Implementation, and operations of OLTP, OLAP, DW technologies such...

  • Azure data engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Job Description**mandatorySNRequired InformationDetails1Role**Azure Data Bricks/Data/Data Factory Enginner2Required Technical Skill Set**Azure Data Bricks/ADF EnginnerDesired Experience Range**8-11 yearsLocation of RequirementBangaloreDesired Competencies (Technical/Behavioral Competency)Must-Have**Build and maintain the technology platform for both on...

  • Data engineer

    4 weeks ago

    Bengaluru, India Jobtrix | Powered By Kyla Full time

    Hiring for Fortune companiesJob Title: Azure Data EngineerExperience: 7+ yearsLocation: Bangalore / Chennai / HyderabadDesign and implement data pipelines using the Azure Data Engineering Stack (Azure Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, Function Apps).Lead and mentor a team of data engineers, ensuring best practices in Azure...

  • Data Engineer

    4 weeks ago

    Bengaluru, India Jobtrix | Powered by Kyla Full time

    Hiring for Fortune companiesJob Title: Azure Data EngineerExperience: 7+ yearsLocation: Bangalore / Chennai / HyderabadDesign and implement data pipelines using the Azure Data Engineering Stack (Azure Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, Function Apps).Lead and mentor a team of data engineers, ensuring best practices in Azure...

  • Azure Data Engineer

    1 month ago

    Bengaluru, India Infogain Full time

    Greetings from Infogain!We are having Immediate requirement for Azure Data Engineer in Infogain India Pvt LtdWe’re looking for a skilled Azure Data Engineer to join our dynamic team! If you thrive in a collaborative environment and have a knack for transforming raw data into actionable insights, we want to hear from you!Mode of Hiring- Full TimeSkills :...

  • Data Engineer

    4 weeks ago

    Bengaluru, India Jobtrix | Powered by Kyla Full time

    Hiring for Fortune companiesJob Title: Azure Data EngineerExperience: 7+ yearsLocation: Bangalore / Chennai / HyderabadDesign and implement data pipelines using the Azure Data Engineering Stack (Azure Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, Function Apps).Lead and mentor a team of data engineers, ensuring best practices in Azure...

  • Data Engineer

    4 weeks ago

    Bengaluru, India Jobtrix | Powered by Kyla Full time

    Hiring for Fortune companiesJob Title: Azure Data EngineerExperience: 7+ yearsLocation: Bangalore / Chennai / HyderabadDesign and implement data pipelines using the Azure Data Engineering Stack (Azure Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, Function Apps).Lead and mentor a team of data engineers, ensuring best practices in Azure...

  • Data Engineer

    4 weeks ago

    Bengaluru, India Jobtrix | Powered by Kyla Full time

    Hiring for Fortune companies Job Title: Azure Data Engineer Experience: 7+ years Location: Bangalore / Chennai / Hyderabad Design and implement data pipelines using the Azure Data Engineering Stack (Azure Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, Function Apps). Lead and mentor a team of data engineers, ensuring best practices...

  • Azure Data Engineer

    6 days ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TCS!!! **TCS is Hiring for Azure Databricks Data Engineer ** Walkin Interview for Azure Databricks Data Engineer in Bangalore Walkin Interview Date: 14th Dec2024(Saturday) Role: Azure Databricks Data Engineer Desired Experience: 5- 10 Years Job Location: Bangalore Job Description: · Implementation, and operations of OLTP,...

  • Azure Data Engineer

    6 days ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TCS!!!**TCS is Hiring for Azure Databricks Data Engineer **Walkin Interview for Azure Databricks Data Engineer in BangaloreWalkin Interview Date: 14th Dec2024(Saturday)Role: Azure Databricks Data Engineer Desired Experience: 5- 10 YearsJob Location: BangaloreJob Description:· Implementation, and operations of OLTP, OLAP, DW technologies such...

  • Azure Data Engineer

    4 weeks ago

    Bengaluru, India Networth Corp Full time

    Responsibilities:Working along with architects for planning, designing, integrating, and implementing complex solutions in Azure Cloud computing servicesEnd to end development lifecycle of complex large-scale and multi-user applicationsImplement Azure Cloud data warehouses, Azure and No-SQL databases and hybrid data scenariosBig Data ETL considering...

  • Data Engineering Cloud Specialist

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data Engineering Cloud SpecialistAbout the RoleWe are seeking an experienced Data Engineer to join our team at NTT DATA. This is a fantastic opportunity to work on diverse data engineering activities on Azure Data platform and contribute to building a robust data pipeline connecting various enterprise data sources.About YouTo be successful in this...

  • Azure data engineer

    1 month ago

    Bengaluru, India Infogain Full time

    Greetings from Infogain!We are having Immediate requirement for Azure Data Engineer in Infogain India Pvt LtdWe’re looking for a skilled Azure Data Engineer to join our dynamic team! If you thrive in a collaborative environment and have a knack for transforming raw data into actionable insights, we want to hear from you!Mode of Hiring- Full TimeSkills :...

  • Azure data engineer

    3 days ago

    Bengaluru, India NeerInfo Solutions Full time

    Hiring for Fortune IT CompanyRole: Azure Data EngineerExperience: 5 to 12 YearsLocation: Pan IndiaNotice Period: Immediate to 45 days to currently servingJD:Proficiency in Azure services: Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake.Strong programming skills in Python, Scala, SQL, or Java.Familiarity with big data tools like...