Freelance Apache NiFi + GCP Data Engineer

1 week ago

Bikaner, India ThreatXIntel Full time

Company DescriptionThreatXIntel is a startup cybersecurity company specializing in protecting businesses and organizations from cyber threats. We offer services including cloud security, web and mobile security testing, DevSecOps, and cloud security assessments to deliver customized, affordable solutions tailored to client needs. Committed to proactive security, we continuously monitor and test digital environments to mitigate vulnerabilities before they can be exploited. Our mission is to empower businesses of all sizes with high-quality cybersecurity services to protect their digital assets and enable business growth.Role DescriptionWe are seeking an experienced Freelance Data Engineer with strong expertise in Apache NiFi and the Google Cloud Platform (GCP) ecosystem. The consultant will design and implement real-time data ingestion pipelines, integrate with GCP services, and ensure secure, scalable, and high-performance data delivery for analytics and downstream platforms.Key ResponsibilitiesDesign and build real-time ingestion pipelines using Apache NiFi for structured and unstructured datasets.Integrate NiFi pipelines with the GCP ecosystem, including BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer (Airflow).Implement low-latency, high-throughput data movement across ingestion, storage, and processing layers.Develop and optimize SQL queries for BigQuery and relational systems.Build Python scripts for data transformations, automation tasks, and custom NiFi processors.Apply data quality, validation, deduplication, and error-handling frameworks within ingestion flows.Work with analysts, data scientists, and platform teams to deliver clean, reliable, analytics-ready datasets.Enforce GCP IAM, VPC Service Controls, encryption policies, and secure-by-design practices.Monitor, troubleshoot, and tune NiFi processor performance and GCP data pipelines.Document data flows, architecture standards, and operational best practices.Required Skills & QualificationsStrong hands-on experience with Apache NiFi for real-time ingestion, routing, transformation, and flow orchestration.Deep expertise across the GCP ecosystem:Pub/Sub (event ingestion & streaming)BigQuery (data warehousing & SQL optimization)Dataflow / Apache Beam (stream & batch processing)Cloud Storage (data lake)Dataproc (Spark/Hadoop processing)Composer/Airflow (workflow orchestration)Strong proficiency in SQL (analytic functions, optimization, partitioning, clustering).Strong Python development for automation and data processing.Solid understanding of real-time streaming concepts: windowing, late data handling, deduplication, event ordering.Knowledge of data architecture patterns (ETL/ELT, data lakes, data warehouses).Experience with CI/CD, Git-based workflows, and infrastructure-as-code (nice to have).Understanding of cloud security, IAM roles, governance controls, and compliance in GCP environments.

  • Freelance Data Engineer

    1 day ago

    bikaner, India beBeeData Full time

    Job Title: Freelance Data Engineer - Real-Time Pipeline SpecialistWe are seeking a skilled Freelance Data Engineer to design and build high-performance real-time data ingestion and processing pipelines using Apache NiFi and the Google Cloud Platform (GCP) ecosystem.About the RoleThis role requires hands-on technical execution, collaboration with...

  • Cloud Engineer

    3 days ago

    bikaner, India beBeeData Full time

    Job OverviewWe are seeking a skilled Freelance Data Engineer to develop and implement real-time data pipelines using Apache NiFi for structured and unstructured data.The ideal candidate has expertise in the Google Cloud Platform (GCP) ecosystem, including BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer/Airflow.Create efficient data...

  • Data Engineer

    4 weeks ago

    Bikaner, India Aceolution Full time

    Job Title: Data Engineer – Python Expert(Freelance Role)Location: Remote / HybridEmployment Type: Contract/ FreelanceRole Summary We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) development. As a senior Individual Contributor (IC), you will be the team's expert on...

  • Chief Data Engineering Strategist

    5 days ago

    bikaner, India beBeeDataEngineer Full time

    Unlock the Power of Data EngineeringBecoming a leading expert in data engineering is an exceptional opportunity to shape the future of our organization. As a skilled data engineer, you will play a pivotal role in driving innovation within our cutting-edge microservices architecture.Your key responsibilities will include:Designing and Implementing Data...

  • Data Engineering Specialist

    5 days ago

    bikaner, India beBeeDataEngineer Full time

    Job TitlePlatform Engineer – PythonKey Responsibilities:Design and develop scalable data pipelines and platform services using Python, Apache Spark, and Databricks.Develop and manage notebook-based workflows (Jupyter, Databricks, etc.) for data processing, analysis, and automation.Build and optimize data integration workflows and reusable components for...

  • Senior Data Engineer

    3 days ago

    bikaner, India beBeeDataScience Full time

    Data Science SpecialistBayone Solutions is seeking a highly experienced Data Scientist to drive innovation within our microservices architecture.We are looking for someone who can lead the development, optimization, and maintenance of data pipelines and back-end services, ensuring seamless integration within a microservices environment.Design, develop, and...

  • Freelance Machine Learning Engineer for Innovative Projects

    5 days ago

    bikaner, India beBeeMachineLearning Full time

    Machine Learning Engineer for Innovative ProjectsJob Description:We are seeking a seasoned Freelance Machine Learning Engineer to collaborate with our development team on cutting-edge projects. The ideal candidate should possess at least 8 years of experience in machine learning and AI.Key Responsibilities:Design, develop, and deploy high-performing ML...

  • Senior Data Systems Architect

    3 days ago

    bikaner, India beBeeDataEngineer Full time

    Designing Scalable SystemsAs a lead data engineer, you will be responsible for designing and building distributed systems that are fault-tolerant and maintainable.You will collaborate with stakeholders to understand business requirements and work with cross-functional teams to ensure seamless integration of new data pipelines, services, and systems.Technical...

  • Freelance Python Full Stack Engineer

    4 days ago

    Bikaner, India Leading MNC Full time

    Looking for a Freelance Python Full Stack Engineer to join a team of rockstar developers. The candidate should have a minimum of 5+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will...

  • Cloud Data Engineering Opportunities

    1 day ago

    bikaner, India beBeeDataEngineering Full time

    Cloud Data Architecture SpecialistWe are seeking a skilled Cloud Data Architecture Specialist to design and implement scalable data pipelines using modern cloud-native tools and frameworks.As a key member of our team, you will be responsible for developing ETL/ELT workflows for structured, semi-structured, and unstructured data. You will also work with...