Real-Time Data Ingestion Specialist

1 week ago

kochi, India beBeeDataEngineer Full time

Job SummaryWe are seeking a highly skilled Data Engineer to design and build real-time data ingestion and processing pipelines. The ideal candidate will develop scalable solutions leveraging Apache NiFi and Google Cloud Platform services.Key Responsibilities:Design and implement real-time data ingestion pipelines using Apache NiFi for both structured and unstructured datasets.Integrate ingestion and processing pipelines with the Google Cloud Platform ecosystem, including BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer (Airflow).Ensure low-latency and high-throughput data movement across ingestion, processing, and storage layers.Develop and optimize SQL queries for BigQuery and relational databases to support analytics and reporting.Build Python scripts for data transformation, automation, and developing custom NiFi processors.Implement data quality checks, validation rules, deduplication, late-arriving data handling, and robust error-handling frameworks.Collaborate with data analysts, data scientists, and platform engineering teams to deliver analytics-ready datasets.Apply data security, GCP IAM policies, VPC Service Controls, and encryption standards to maintain compliance.Monitor, troubleshoot, and optimize pipeline performance to ensure stability and reliability.Document data flows, architecture diagrams, transformation logic, and operational best practices.Required Skills & QualificationsStrong hands-on experience with Apache NiFi for real-time ingestion, routing, transformation, and orchestration.Deep knowledge of key Google Cloud Platform services, including:Pub/Sub (event ingestion & streaming)BigQuery (warehouse, SQL optimization)Dataflow / Apache Beam (data processing pipelines)Dataproc (Spark/Hadoop processing)Cloud Storage (data lake)Composer / Airflow (workflow orchestration)Advanced SQL skills for querying, optimization, and data modeling.Proficiency in Python for automation, ETL tasks, and scripting custom processors.Strong understanding of real-time streaming concepts such as windowing, watermarks, deduplication, and event-time processing.Knowledge of data architecture principles, including ETL/ELT design, data lakes, and warehouse best practices.Experience with CI/CD, Git-based version control, and collaborative development workflows.Familiarity with security and governance practices in cloud-based data environments.

  • Cloud Data Engineering Specialist

    2 weeks ago

    kochi, India beBeeData Full time

    Job Title:Cloud Data Engineering SpecialistJob OverviewThreatXIntel is a startup specializing in cybersecurity, offering tailored and cost-effective solutions for businesses and organizations of all sizes. Our expertise spans cloud security, web and mobile security testing, cloud security assessments, DevSecOps, and more.We are seeking an experienced Cloud...

  • Data Acquisition Specialist

    2 weeks ago

    kochi, India beBeeDataAcquisition Full time

    Job Opportunity: Data EngineerWe are seeking a highly skilled Data Acquisition Specialist to join our data engineering team. The ideal candidate will have extensive hands-on experience in building and optimizing data ingestion and transformation pipelines using Azure Data Platform, Databricks, Snowflake, and Kafka.The successful candidate will be responsible...

  • Part-Time Business Intelligence Specialist

    5 days ago

    kochi, India beBeeDataAnalyst Full time

    Role OverviewWe are seeking a skilled Data Analyst to join our team on a part-time basis.Job DescriptionThe ideal candidate will support multiple ongoing client projects by building dashboards, automating data flows, cleaning and modelling real-world datasets, and ensuring that reporting is accurate, reliable, and scalable.This role involves developing data...

  • Title processsing specialist

    3 weeks ago

    Kochi, India Stride Real Estate Full time

    We are looking for an experienced Title Processing Specialist to join our Title Company team. The ideal candidate must have 2–3 years of hands-on experience in U.S. Title Processing and be confident handling end-to-end title and closing workflows.Key Responsibilities • Review and process title orders with high accuracy • Examine property records,...

  • Title processsing specialist

    3 weeks ago

    Kochi, India Stride Real Estate Full time

    We are looking for an experienced Title Processing Specialist to join our Title Company team. The ideal candidate must have 2–3 years of hands-on experience in U.S. Title Processing and be confident handling end-to-end title and closing workflows.Key Responsibilities • Review and process title orders with high accuracy • Examine property records,...

  • Senior Backend Developer

    2 weeks ago

    kochi, India beBeeBackend Full time

    Design and Develop Backend APIsAbout the RoleThis role involves creating scalable backend solutions to power real-time communications analytics and automation. You will work on designing and shipping backend APIs, background jobs, and secure integrations with external services.Key ResponsibilitiesCreate robust and efficient backend APIsDevelop background...

  • Data Annotation Specialist

    5 days ago

    kochi, India beBeeDataAnnotation Full time

    Job Title: Data Annotation SpecialistThe role of a Data Annotation Specialist involves reviewing, tagging, and labeling online content to support the development of artificial intelligence (AI) and machine learning (ML) projects.Responsibilities include:Key ResponsibilitiesReviewing diverse online content including sensitive materialMaintaining top-notch...

  • High-Performance Data Engineer

    2 weeks ago

    kochi, India beBeeData Full time

    Data Support SpecialistWe are seeking a highly skilled Data Support Specialist to serve as the escalation point for data ingestion, pipeline, and consumption issues.Key Responsibilities:Support escalation point for issues with data ingestion, consumption, pipelines.Monitoring: Proactively monitor data pipeline health to ensure stability and uptime.Root cause...

  • Data Architectural Specialist

    1 week ago

    kochi, India beBeeDataArchitecture Full time

    Senior Data Architecture SpecialistWe are seeking a highly skilled professional to design and develop scalable data pipelines using PySpark, Snowflake, and AWS Glue.The ideal candidate should have strong experience in Python, data modeling, ETL frameworks, and cloud-based data platforms. They will be responsible for building and optimizing data models in...

  • Chief Data Integration Specialist

    2 weeks ago

    kochi, India beBeeData Full time

    **Job Title:** Chief Data Integration Specialist**Job Description:** We are seeking a highly skilled professional to join our team as a Chief Data Integration Specialist. In this role, you will be responsible for designing and implementing scalable and flexible integration solutions across multiple sites and enterprise systems.**Key Responsibilities:Design...