Data Pipeline Specialist

13 hours ago

Erode, Tamil Nadu, India beBeeDataEngineering Full time ₹ 10,00,000 - ₹ 20,00,000

Job Description:

Seeking a data engineering professional to build scalable data pipelines and cloud-native data solutions on GCP. Ideal candidate will be proficient in Pub/Sub, Dataflow, Cloud Storage, and BigQuery, with a foundational understanding of AI/ML workflows using Vertex AI.

Key Responsibilities:

  • Design and develop robust data ingestion pipelines using GCP services like Pub/Sub, Dataflow, and Cloud Storage.
  • Architect and manage scalable BigQuery data warehouses to support analytics, reporting, and business intelligence needs.
  • Collaborate with data scientists and ML engineers to support AI/ML workflows using Vertex AI.
  • Ensure data quality, reliability, and performance across all pipeline components.
  • Work closely with cross-functional teams to understand data requirements and deliver efficient solutions.

Required Skills & Qualifications:

  • 3-6 years of experience in data engineering, with strong exposure to GCP.
  • Proficiency in GCP services: Pub/Sub, Dataflow, Cloud Storage, and BigQuery.
  • Solid understanding of data modeling, ETL/ELT processes, and performance optimization.
  • Experience with Python, SQL, and cloud-native development practices.
  • Familiarity with CI/CD pipelines and version control (e.g., Git).
  • Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.

Secondary Skills:

  • Basic understanding of AI/ML workflows and tools within Vertex AI.
  • Ability to discuss model lifecycle, deployment strategies, and integration with data pipelines.
  • Awareness of MLOps principles and cloud-based ML orchestration.

  • Data Pipeline Specialist

    14 hours ago

    Erode, Tamil Nadu, India beBeeDataPipelineSpecialist Full time ₹ 12,00,000 - ₹ 21,50,000

    Job Title: Data Pipeline SpecialistAre you a skilled engineer with expertise in data pipeline architecture and big data technologies? Do you have hands-on experience with Spark, DBT, and cloud services?We are looking for a talented individual to join our team as a Data Pipeline Specialist.Create and maintain optimal data pipeline architecturesDesign and...

  • Data Pipeline Specialist

    1 day ago

    Erode, Tamil Nadu, India beBeeIntegration Full time US$ 90,000 - US$ 1,20,000

    Job Title:Azure Integration EngineerKey Responsibilities:Troubleshoot and resolve issues with Azure Synapse Analytics, Power BI, SQL Server, and Azure Data Factory (ADF)Optimize performance and tune queries for data pipelines and integrationsDesign, develop, and deploy data pipelines using ADFMaintain existing reports and databasesWork & Engagement...

  • Data Pipeline Specialist

    3 days ago

    Erode, Tamil Nadu, India beBeeDataQuality Full time ₹ 12,00,000 - ₹ 17,00,000

    About People Prime Worldwide:We are a global IT services company that specializes in digital engineering and IT services, helping clients modernize their technology infrastructure and accelerate innovation.Our expertise lies in combining deep industry knowledge with agile development practices to deliver scalable and cost-effective digital transformation...

  • Data Pipeline Specialist

    3 days ago

    Erode, Tamil Nadu, India beBeeelasticsearch Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    About this Position:We are looking for a skilled IT professional to maintain the health of our data pipeline.

  • Data Pipeline Quality Assurance Specialist

    1 week ago

    Erode, Tamil Nadu, India beBeeSoftwareTester Full time ₹ 25,00,000 - ₹ 30,00,000

    Job Title: Software Quality Assurance SpecialistJob Description:We are seeking a skilled Software Tester to join our team. As a QA/ETL Tester, you will be responsible for ensuring the quality and integrity of our data pipelines.The ideal candidate will have a strong background in data migration testing, with experience in source-to-target validation, record...

  • Senior Data Pipeline Specialist

    1 week ago

    Erode, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering OpportunityWe are seeking a skilled and detail-oriented data engineer to join our team and contribute to the design, build, and management of scalable data pipelines and enterprise data solutions.

  • Cloud Data Pipeline Specialist

    9 hours ago

    Erode, Tamil Nadu, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    We are seeking an experienced data engineer to lead the design, development and maintenance of large-scale data pipelines on Microsoft Azure.Key Responsibilities:Create scalable data architectures on Azure using technologies such as Azure Data Factory, Azure Databricks and Azure Synapse Analytics.Collaborate with cross-functional teams to identify business...

  • Chief Data Pipeline Architect

    3 days ago

    Erode, Tamil Nadu, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    About the RoleWe are seeking a Senior Data Engineer to design, build, and operate scalable data pipelines.This is a critical role where you will create scalable pipelines that process clinical encounter data, retrieve associated medical documents, and deliver validated information to machine learning systems for inference.Your ResponsibilitiesDesign and...

  • Chief Data Pipeline Architect

    7 days ago

    Erode, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 30,00,000

    Key Roles and ResponsibilitiesWe are seeking a highly skilled professional to join our data engineering team. As a key member of the team, you will play a vital role in designing and implementing efficient data pipelines using modern DevOps tools.Develop and manage continuous integration and delivery pipelines using GitHub Actions or equivalent...

  • Expert Large Scale Data Pipeline Specialist

    7 days ago

    Erode, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Freelance Data Engineer – Python, SQL, AWSAbout the RoleWe are seeking an experienced Freelance Data Engineer with expertise in designing, building, and optimizing large-scale data pipelines using PySpark, AWS, and Databricks. The consultant will collaborate with internal teams and clients to ensure robust, scalable, and high-performance data...