Data Pipeline Specialist

11 hours ago

Hyderabad, Telangana, India beBeeProfessional Full time ₹ 18,00,000 - ₹ 24,00,000

Job Overview

The Software Professional will design, build, and maintain complex data pipelines, datasets, dataflows, integration runtimes, and monitor pipeline executions using Azure Data Factory and Azure Synapse.

Key Responsibilities

  • Extract, transform, and load (ETL) data from source systems and process large datasets using Azure Databricks.
  • Develop advanced SQL scripts for data querying and transformation.
  • Create and manage Synapse pipelines to migrate data from Azure Data Lake Gen2 to Azure SQL databases.
  • Manage data migration pipelines to Azure cloud, including database migration from on-premises SQL Server to Azure Dev Environment using Azure Database Migration Service (DMS) and Data Migration Assistant.
  • Utilize Azure Data Catalog for data asset discovery and management.
  • Work on Big Data batch processing, interactive processing, and real-time processing solutions using Azure.

Required Skills and Qualifications

  • 3 to 5 years of experience working with Microsoft Azure Fabric ecosystem.
  • Proficiency in Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure SQL, and ETL concepts.
  • Strong SQL skills for complex data manipulation and transformation.
  • Experience in building and monitoring data pipelines and triggers.
  • Hands-on experience with data migration techniques using Azure DMS and Data Migration Assistant.
  • Familiarity with Azure Data Catalog for metadata management.
  • Experience working with Big Data processing solutions, both batch and real-time.

Certifications

  • Azure Data Engineer Associate or relevant Microsoft Azure certifications are a plus.

  • Data Pipeline Specialist

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataPipeline Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking an accomplished Data Pipeline Specialist to join our team. The successful candidate will be responsible for creating and maintaining data pipelines that drive business growth.Main ResponsibilitiesCreate and implement new data pipelines to meet business needsEssential Skills and Qualifications4-6 years of experience in designing and...

  • Data Pipeline Specialist

    1 week ago

    Hyderabad, Telangana, India beBeeDataPipelineManager Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Job Title: Data Team LeadData Pipeline ManagerWe are seeking a seasoned data pipeline manager with experience in designing and developing robust data pipelines using Azure Databricks, Data Factory, and Datalake.The role involves overseeing the development of complex ETL solutions, managing large volumes of data, and working closely with business teams to...

  • Data Pipeline Specialist

    5 days ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 8,00,000 - ₹ 12,00,000

    Data Engineer PositionWe are seeking a skilled data engineer to join our team. The ideal candidate will be responsible for designing, implementing, and optimizing data pipelines on our platform.Data Pipeline Development: Design, develop, and maintain scalable data pipelines using Azure Synapse, Databricks & Apache Spark (PySpark).Data Integration: Integrate...

  • Data Pipeline Specialist

    2 weeks ago

    Hyderabad, Telangana, India beBeeBusiness Full time ₹ 12,00,000 - ₹ 20,00,000

    Job Title: Business Intelligence ArchitectWe are seeking an experienced Business Intelligence Architect with a strong background in data engineering and a passion for designing, building, and optimizing scalable data pipelines.The ideal candidate will have hands-on experience in Snowflake, AWS Lambda, Glue, DBT, and SQL. They will be responsible for...

  • Data Pipeline Specialist

    7 days ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Data Scientist RoleThe ideal candidate will have a strong background in digital channel activities, data engineering and cloud technologies, particularly in AWS.We are seeking an experienced professional to design, build and maintain robust data pipelines for processing channel activity data, focusing on digital channels and integrating multiple sources of...

  • Data Pipeline Specialist

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Opportunity:We are seeking a highly skilled data professional to design and implement robust data pipelines using AWS, Python, and SQL to support business analytics and decision-making.A strong command of SQL for creating, managing, and optimizing data structures and complex queries is required.Expertise in AWS services such as S3, EC2, RDS, Redshift,...

  • Principal Pipeline Specialist

    1 week ago

    Hyderabad, Telangana, India beBeePipeline Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Pipeline Engineering ExcellenceWe seek a skilled pipeline specialist to lead the design and development of complex pipeline systems. Key responsibilities include:Developing and reviewing technical documents, such as specifications, data sheets, drawings, and reports.Evaluating pipeline designs for compliance with industry standards (e.g., ASME B31.4, B31.8,...

  • Data Pipeline Specialist

    1 week ago

    Hyderabad, Telangana, India beBeeEtalizer Full time ₹ 15,00,000 - ₹ 25,00,000

    ETL DeveloperWe are seeking a highly skilled ETL developer to design, develop and optimize complex ETL workflows to support enterprise data integration and analytics.Key Responsibilities:Design, develop, and maintain ETL mappings, workflows, and sessions using Informatica PowerCenter.Implement and optimize Informatica Cloud (IICS) pipelines for cloud-based...

  • Data Pipeline Expert

    2 weeks ago

    Hyderabad, Telangana, India beBeeetl Full time ₹ 10,00,000 - ₹ 15,00,000

    We are currently seeking a talented ETL Support Specialist to join our team. As an ETL Support Specialist, you will play a crucial role in ensuring the smooth operation of our data pipelines.

  • Data Pipeline Quality Engineer

    1 day ago

    Hyderabad, Telangana, India beBeePerformance Full time US$ 6,00,000 - US$ 10,80,000

    Job Overview:We seek a highly skilled Quality Assurance Specialist with 5-10 years of professional experience to ensure the integrity, reliability, and performance of data pipelines and AI/ML solutions within our SmartFM platform.Key Responsibilities: