Expert Data Pipeline Architect

2 weeks ago

Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000
Job Title:

A Data Engineer is needed to design, build, and maintain large-scale data pipelines using Google Cloud Platform.

Responsibilities:

  • Develop, construct, test, and maintain data acquisition pipelines for structured and unstructured data in GCP.
  • Create complex datasets based on business requirements.
  • Construct 'big data' pipeline architecture.
  • Identify opportunities for data acquisition via stakeholder collaboration.
  • Translate business needs to technical requirements.
  • Leverage various tools in the GCP Ecosystem, including Python, Data Flow, DataStream, CDC, Cloud Functions, Pub Sub, BigQuery, and Cloud Storage.
  • Use logs & alerts to effectively monitor pipelines.
  • Develop a JSON messaging structure for integrating with applications.
  • Implement DevOps and CI/CD practices (GitHub, Terraform) to ensure reliability and scalability of data pipelines.
  • Manage data partitioning, clustering, and retrieval in Big Query, and secure it using IAM roles & Policy Tags.
  • Design and build an ingestion pipeline using Rest API.
  • Recommend ways to improve data quality, reliability, and efficiency.

Requirements:

  • Requires 5-7 years of experience in a data engineering role, with core GCP experience of 3-4 years.
  • Work hours will be in CST until 2 am IST on most days.
  • 1-2 times a month, additional work hours may be required.

  • Highly Skilled Data Architect

    2 weeks ago

    Palakkad, Kerala, India beBeeData Architect Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Senior Snowflake ConsultantJob Description:We are seeking a highly skilled expert in data architecture with hands-on experience in Matillion ETL and proven expertise in the Banking, Financial Services domain.Design, develop, and optimize scalable data pipelines, models, and analytics solutions on Snowflake.Define best practices around performance...

  • Chief Data Pipeline Architect

    2 weeks ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineer Role OverviewSeeking a skilled data engineer to design, develop, and maintain robust data pipelines. Responsibilities include extracting data from Oracle Symphony via APIs, processing and storing it in the Databricks Lakehouse platform, and integrating it into Oracle EPM.Key Responsibilities:Design and implement end-to-end data pipelines from...

  • Data Architect

    7 days ago

    Palakkad, Kerala, India beBeeDataIntegration Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Job Title: Data ArchitectWe are seeking an experienced data architect to design and develop scalable data pipelines that drive business growth and profitability.The ideal candidate will be responsible for leading cross-functional teams to deliver high-quality data solutions that meet business requirements.Design, build, and maintain ETL/ELT pipelines using...

  • Enterprise Data Architect

    2 weeks ago

    Palakkad, Kerala, India beBeeDataEngineering Full time ₹ 18,24,000 - ₹ 21,36,000

    About the RoleWe are seeking an experienced Senior Data Engineer with a strong background in building and scaling enterprise-grade data platforms.The ideal candidate will have expertise in ETL, Cloud, Data Warehousing, and Big Data ecosystems. They will be responsible for designing and optimizing data pipelines using AWS Glue, Airflow, Databricks,...

  • Data Systems Architect

    2 weeks ago

    Palakkad, Kerala, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title: Data Systems ArchitectWe are seeking an experienced Data Systems Architect to join our team. As a Data Systems Architect, you will be responsible for designing, building, and maintaining efficient data pipelines.

  • AI Pipeline Designer

    1 week ago

    Palakkad, Kerala, India beBeeArtificial Full time US$ 1,20,000 - US$ 1,80,000

    Design and Refine AI PipelinesAs a seasoned professional in the field of healthcare AI, you will play a critical role in designing, building, and refining multi-agent, LLM-driven AI pipelines.This individual will develop unique evaluation frameworks and tooling, shaping the entire engineering team's approach. With rapid design, build, and refinement...

  • Chief Data Architect

    1 week ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data EngineerCoforge is seeking a seasoned data expert to craft robust data pipelines, steering business strategies forward.Key Responsibilities:Designing and refining intricate ETL workflows for processing large datasets.Ensuring data accuracy, consistency, and system performance.Collaborating with architects, analysts, and stakeholders to...

  • Data Architect

    2 weeks ago

    Palakkad, Kerala, India beBeeDatasphere Full time ₹ 18,00,000 - ₹ 24,00,000

    Job OverviewWe seek a skilled professional to design, develop, and implement enterprise-level data management and analytics solutions.This role involves creating scalable, secure data pipelines and models, integrating SAP Datasphere with BW/4HANA, SAC, and other data sources.Key ResponsibilitiesArchitect & Integrate SolutionsDesign data pipelines and...

  • Senior Data Pipeline Specialist

    2 weeks ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Cloud Data Engineer PositionLemongrass is seeking a skilled Cloud Data Engineer to design, develop, and maintain scalable data pipelines using cloud-native tools. The ideal candidate will have extensive experience with AWS, Azure, and GCP, as well as knowledge of data governance, security, and compliance best practices.">Key Responsibilities:Design, develop,...

  • Data Architect

    2 weeks ago

    Palakkad, Kerala, India Velodata Global Pvt Ltd Full time

    Job Title: Data Architect Location: Kochi / Trivandrum Experience: 12–17 years Notice Period: Immediate to 30 days Salary: Up to 32 LPAKey ResponsibilitiesDesign and implement scalable, secure, and high-performance data architectures.Lead data strategy across Azure (primary) with exposure to AWS/GCP.Build and optimize data pipelines using Azure Data...