Freelance Apache NiFi + GCP Data Engineer

12 hours ago

Nashik, India ThreatXIntel Full time

Company Description ThreatXIntel is a startup cybersecurity company specializing in protecting businesses and organizations from cyber threats. We offer services including cloud security, web and mobile security testing, DevSecOps, and cloud security assessments to deliver customized, affordable solutions tailored to client needs. Committed to proactive security, we continuously monitor and test digital environments to mitigate vulnerabilities before they can be exploited. Our mission is to empower businesses of all sizes with high-quality cybersecurity services to protect their digital assets and enable business growth. Role Description We are seeking an experienced Freelance Data Engineer with strong expertise in Apache NiFi and the Google Cloud Platform (GCP) ecosystem. The consultant will design and implement real-time data ingestion pipelines, integrate with GCP services, and ensure secure, scalable, and high-performance data delivery for analytics and downstream platforms. Key Responsibilities - Design and build real-time ingestion pipelines using Apache NiFi for structured and unstructured datasets. - Integrate NiFi pipelines with the GCP ecosystem, including BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer (Airflow). - Implement low-latency, high-throughput data movement across ingestion, storage, and processing layers. - Develop and optimize SQL queries for BigQuery and relational systems. - Build Python scripts for data transformations, automation tasks, and custom NiFi processors. - Apply data quality, validation, deduplication, and error-handling frameworks within ingestion flows. - Work with analysts, data scientists, and platform teams to deliver clean, reliable, analytics-ready datasets. - Enforce GCP IAM, VPC Service Controls, encryption policies, and secure-by-design practices. - Monitor, troubleshoot, and tune NiFi processor performance and GCP data pipelines. - Document data flows, architecture standards, and operational best practices. Required Skills & Qualifications - Strong hands-on experience with Apache NiFi for real-time ingestion, routing, transformation, and flow orchestration. - Deep expertise across the GCP ecosystem: - Pub/Sub (event ingestion & streaming) - BigQuery (data warehousing & SQL optimization) - Dataflow / Apache Beam (stream & batch processing) - Cloud Storage (data lake) - Dataproc (Spark/Hadoop processing) - Composer/Airflow (workflow orchestration) - Strong proficiency in SQL (analytic functions, optimization, partitioning, clustering). - Strong Python development for automation and data processing. - Solid understanding of real-time streaming concepts: windowing, late data handling, deduplication, event ordering. - Knowledge of data architecture patterns (ETL/ELT, data lakes, data warehouses). - Experience with CI/CD, Git-based workflows, and infrastructure-as-code (nice to have). - Understanding of cloud security, IAM roles, governance controls, and compliance in GCP environments.

  • Freelance Data Engineer

    2 weeks ago

    Nashik, India Leading MNC Full time

    Looking for a Freelance Data Engineer to join a team of rockstar developers. The candidate should have a minimum of 8+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report into...

  • Gcp data engineer

    2 weeks ago

    Nashik, India Adastra Full time

    Job Description We are looking for a proactive and solution-oriented GCP Data Engineer to join our team. This role requires hands-on experience in Google Cloud Platform (GCP), especially with Big Query and Airflow, alongside strong SQL and Python skills. The ideal candidate is a self-starter who thrives in solving problems independently, communicates...

  • GCP Data Engineer

    12 hours ago

    Nashik, India RB Consultancy Services Full time

    Job Title: GCP Data Engineer Location: TCS - Chennai, Hyderabad, Bangalore, Pune, Delhi Type: Full-time Experience: 4+ years Primary Skills: - Python with OOPs concept. - Data flow - Cloud Composer - Big query Secondary Skills: - Agile - Spinnaker - Jenkins - Git - GKE GOOD TO HAVE: - Data modelling basics - Tableau basics - SQL Performance tuning...

  • Principal Big Data Architect

    4 days ago

    nashik, India beBeeBigDataEngineer Full time

    Big Data Engineer RoleJob Overview:We are seeking a seasoned Big Data professional with extensive experience in designing, developing, and managing large-scale data processing systems using Hadoop, PySpark, and MySQL. The ideal candidate should have a proven track record of deploying and optimizing complex Big Data architectures within on-premises...

  • Data Engineer

    3 weeks ago

    Nashik, India HISH IT SERVICES Full time

    We have a new urgent GCP Data Engineer opportunity open to support a migration initiative from Teradata to Cerebro (BigQuery). This role requires a hands-on developer who can collaborate closely with our data and reporting teams to ensure smooth repointing and validation of Power BI reports.Request Details:Title: GCP Data EngineerSeniority: Mid to Senior...

  • AWS Data Engineer

    12 hours ago

    Nashik, India ValueMomentum Full time

    Job Title: SSE – Data Engineer Primary Skill: Python, Pyspark Mandatory, StepFunctions, Lamda, Iceberg, Lake Formation and AWS Services and pipelines and experience in data implementation and medallion architecture. Location: Hyderabad/ Pune/ Coimbatore Experience: 4 - 8 years of experience Mode of Work: 5 Days Work from office Job Summary: We are looking...

  • Senior Data Engineer

    1 week ago

    Nashik, India Whatjobs IN C2 Full time

    Job Summary We're looking for a Senior Data Engineer with 5-8 years of experience to build and maintain scalable, production-grade data pipelines. The ideal candidate is a strong software engineer with hands-on experience in Spark (3.X), Scala, SQL, and Python. You'll be responsible for designing and implementing ETL/ELT solutions, collaborating with teams...

  • Sr. Python Developer

    4 days ago

    nashik, India Pert Telecom Solutions Full time

    Job Description: Software Data EngineerAs a Software Engineer specializing in Telecom Products, you will be a vital member of our R&D team, contributing to the development of cutting-edge telecommunications solutions. Your role will involve leveraging your extensive experience to design and implement robust software solutions for scalable telecom products...

  • Platform engineer – python

    1 week ago

    Nashik, India Syntasa Full time

    Platform Engineer – Python / Databricks / Notebooks/ KubernetesJob DescriptionSyntasa is seeking a high-caliber and dedicated Platform Engineer to join its India subsidiary, Syntasa Technologies India Private Limited. This position offers an exciting opportunity to contribute to the growth of a world-class development and support center. The India...

  • Engineering Manager

    2 weeks ago

    Nashik, India HiLabs Full time

    We are looking for an experienced Data Engineering Manager to lead and scale our data engineering function. The ideal candidate will be responsible for designing, building, and maintaining high-quality data pipelines and platforms that support analytics, reporting, and data-driven decision-making across the organization. This role involves leading a team of...