Azure Data Pipeline Specialist

5 days ago

Hyderabad, India Wall Street Consulting Services LLC Full time

Job Summary: We are seeking a highly skilled Senior Azure Data Engineer to design, develop, and optimize data pipelines and solutions within the Microsoft Azure ecosystem. The ideal candidate will have strong experience with Azure Data Factory (ADF), SQL, JSON data processing, and Python-based ETL pipelines, along with an understanding of Property & Casualty (P&C) insurance data models and business processes. Key Responsibilities:· Design, build, and maintain data ingestion, transformation, and integration pipelines using Azure Data Factory (ADF), Databricks, and Azure Synapse Analytics.· Develop scalable ETL/ELT processes to extract and transform structured and semi-structured data (JSON, CSV, Parquet) from diverse data sources.· Write complex SQL queries, stored procedures, and performance tuning scripts for data validation, cleansing, and reporting.· Utilize Python to automate data workflows, implement data quality checks, and optimize data processing tasks.· Collaborate with business analysts, data architects, and insurance domain experts to translate functional requirements into efficient data solutions.· Implement data governance, lineage, and security best practices across Azure environments.· Support data migration and modernization initiatives for legacy insurance data systems into Azure.· Leverage knowledge of P&C Insurance data domains such as policy, claims, underwriting, and reinsurance to design business-ready data models.· Participate in Agile/Scrum ceremonies, ensuring timely delivery of project milestones. Required Skills & Qualifications:· 8+ years of hands-on experience in Data Engineering and Analytics.· 5+ years of experience working with Microsoft Azure Data Services (ADF, Synapse, Data Lake, Databricks).· Proficiency in SQL development and optimization across large datasets.· Strong experience with Python scripting for ETL, API integration, and data processing.· Experience working with JSON and semi-structured data within Azure and cloud environments.· Proven experience in P&C Insurance domain (policy, claims, premium, underwriting data).· Strong understanding of data architecture, pipelines, and data warehousing concepts.· Familiarity with Azure DevOps, Git, and CI/CD pipelines for data deployments.· Excellent problem-solving, analytical, and communication skills. Preferred Qualifications:· Bachelor’s or Master’s degree in Computer Science, Information Systems, or related field.· Experience with Power BI or other BI tools for data visualization.· Exposure to Azure Purview, DataBricks Delta Lake, or Fabric is a plus.· Prior experience working in an Agile environment with insurance or financial services clients.

  • Azure Data Engineer

    1 week ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Data Pipeline Specialist

    2 weeks ago

    hyderabad, India beBeeDataEngineer Full time

    Job OverviewAs a Data Engineer, you will be responsible for designing and developing robust data pipelines and platforms that enable efficient storage, processing, and consumption of data across the enterprise.The role requires high-quality, timely, and governed data delivery to data scientists, analysts, and business users. You will play a crucial role in...

  • Data Pipeline Architect

    2 days ago

    Hyderabad, India Tata Consultancy Services Full time

    Greetings from TCS!!!Come and join us for an exciting career with TCS!!!Date: 20th December 2025Venue: Hyderabad - Synergy park, Non SEZRole: Azure dataDesired Experience Range: 5-10 yearsLocation: HyderabadMust Have Experiences and Skills:3+ years of relevant experience in Pyspark and Azure Databricks.Proficiency in integrating, transforming, and...

  • Azure Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    ---|--- Azure Infrastructure | VM setup & scaling, managed disks, load balancer, Application Gateway, storage accounts (blob/file), monitoring Networking in Azure | VNet, NSG, subnet configuration, VNet peering, route tables, hybrid network setup Azure DevOps & CI/CD | Classic/YAML pipelines, multi-stage deployment flows, variable groups, rollback/retry...

  • Senior Data Pipeline Specialist

    2 weeks ago

    hyderabad, India beBeeData Full time

    Job Title: Senior Data Pipeline SpecialistCompany Overview: We're seeking a seasoned expert in data engineering to join our team.Job Description: As a Senior Data Pipeline Specialist, you'll design, build, and maintain large-scale data pipelines using tools like Cloud Composer, Dataflow, BigQuery, and Cloud Storage. Your primary focus will be on developing...

  • Data Pipeline Specialist

    5 days ago

    Hyderabad, India ANSR Full time

    ANSR is hiring for one of its client.About Visy:Visy is a family-owned Australian business and a global pioneer in sustainable packaging, recycling and logistics. They operate across 150 sites globally, including operations in Asia, Europe, and the USA, supported by a dedicated workforce of over 7,000 employees. It is Australia and New Zealand’s largest...

  • Senior Azure Databricks Data Engineer

    2 weeks ago

    hyderabad, India beBeeDataEngineer Full time

    Are you an experienced Azure Databricks specialist seeking a new challenge? This role offers the perfect opportunity to utilize your skills in data engineering, architecture, and management.About the RoleData Ingestion Pipeline Management: Design, develop, and maintain efficient data ingestion pipelines using Azure Databricks for both batch and real-time...

  • Principal Data Pipeline Architect

    2 weeks ago

    Hyderabad, India ValueMomentum Full time

    Role: Lead Data EngineerPrimary skills: Databricks, PySpark, SQLSecondary skills: Advanced SQL, Azure Data Factory, and Azure Data lake.Experience: 7 to 15 YearAbout the Job We are seeking a Tech Lead –Databricks Data Engineer with experience in designing and developing data pipelines using Azure Databricks, Data Factory, and Datalake.The role involves...

  • Principal Data Pipeline Engineer

    5 days ago

    Hyderabad, India Tech Mahindra Full time

    We are seeking a highly skilled and motivated Senior Data Engineer/s to architect and implement scalable ETL and data storage solutions using Microsoft Fabric and the broader Azure technology stack. This role will be pivotal in building a metadata driven data lake that ingests data from over 100 structured and semi structured sources, enabling rich insights...

  • Senior Data Pipelining Quality Assurance Specialist

    1 week ago

    hyderabad, India beBeeDataPipelining Full time

    Job Title: Quality Assurance Specialist - Data PipeliningJob DescriptionWe are seeking a skilled professional to fill the role of Quality Assurance Specialist - Data Pipelining.Required Skills and QualificationsETL/Data Pipeline Testing: Perform end-to-end testing of ETL pipelines across cloud/on-prem environments, including validation of source-to-target...