Freelance GCP Data Engineer

2 weeks ago

india, IN ThreatXIntel Full time

Company DescriptionThreatXIntel is a startup specializing in cybersecurity, offering tailored and cost-effective solutions for businesses and organizations of all sizes. Our expertise spans cloud security, web and mobile security testing, cloud security assessments, DevSecOps, and more. Driven by a proactive approach, we monitor and test our clients' digital environments to identify vulnerabilities before they can be exploited. Our mission is to deliver reliable and accessible cybersecurity services, empowering businesses to focus on growth while safeguarding their digital assets. Based on a commitment to quality, our team strives to provide peace of mind through customized protection strategies.Role DescriptionWe are seeking a skilled Freelance GCP Data Engineer with expertise in Apache NiFi, SQL, and API integrations to design, implement, and optimize real-time data pipelines. This role involves working with the Google Cloud Platform (GCP) ecosystem, including BigQuery, Pub/Sub, Dataflow, and Cloud Storage, to ensure high-performance data processing, transformation, and delivery to downstream systems for analytics and reporting.Key ResponsibilitiesDesign and implement real-time data ingestion pipelines using Apache NiFi for structured and unstructured data.Integrate data pipelines with the GCP ecosystem, including BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer/Airflow.Ensure low-latency, high-throughput data delivery to downstream systems for analytics and reporting.Develop and optimize SQL queries for BigQuery and relational databases to ensure data transformation and analysis.Write Python scripts for data transformation, automation, and custom NiFi processors when required.Implement data quality, validation, and error-handling mechanisms within ingestion pipelines to ensure reliability and accuracy.Collaborate with data analysts, data scientists, and platform engineers to deliver consistent and high-quality datasets.Ensure data security, governance, and compliance with GCP IAM, VPC Service Controls, and encryption standards.Monitor, troubleshoot, and optimize data pipeline performance to ensure smooth operations.Document data flows, architecture, and best practices to maintain consistency across teams.Required Skills & QualificationsProven experience with Apache NiFi for real-time ingestion, routing, and transformation of data.Strong expertise in the GCP ecosystem, including:BigQuery (data warehousing)Pub/Sub (event streaming)Dataflow/Apache Beam (data processing)Cloud Storage (data lake)Composer/Airflow (workflow orchestration)Strong SQL skills for querying, modeling, and performance optimization of data.Proficiency in Python for data transformation, automation, and creating custom NiFi processors.Experience with real-time streaming concepts, such as windowing, late-arriving data, and deduplication.Strong understanding of ETL/ELT processes, data lakes, and data warehouses.Familiarity with CI/CD practices and version control using Git.Understanding of data security, governance, and compliance within cloud environments.

  • Freelance – Apache NiFi

    2 weeks ago

    india, IN ThreatXIntel Full time

    Company DescriptionThreatXIntel is a dynamic cybersecurity startup dedicated to protecting businesses and organizations against cyber threats. Specializing in services such as cloud security, DevSecOps, and vulnerability testing, the company offers tailored, cost-effective solutions to meet clients' unique requirements. Focused on startups and small...

  • Freelance Apache NiFi GCP Data Engineer

    2 weeks ago

    india, IN ThreatXIntel Full time

    Company DescriptionThreatXIntel is a startup cybersecurity company specializing in protecting businesses and organizations from cyber threats. We offer services including cloud security, web and mobile security testing, DevSecOps, and cloud security assessments to deliver customized, affordable solutions tailored to client needs. Committed to proactive...

  • GCP Data Engineer

    2 weeks ago

    india, IN Adastra Full time

    Job DescriptionWe are looking for a proactive and solution-oriented GCP Data Engineer to join our team. This role requires hands-on experience in Google Cloud Platform (GCP), especially with BigQuery and Airflow, alongside strong SQL and Python skills. The ideal candidate is a self-starter who thrives in solving problems independently, communicates...

  • Freelance – OpenShift

    2 weeks ago

    india, IN ThreatXIntel Full time

    Company DescriptionThreatXIntel is a cyber security startup committed to protecting businesses and organizations from cyber threats. With a focus on delivering tailored and cost-effective solutions, we offer services such as cloud security, web and mobile security testing, cloud security assessment, and DevSecOps. Through a proactive approach to security, we...

  • GCP data Engineer

    2 weeks ago

    Gurugram, Haryana, India, IN Impetus Full time

    Location: Gurgaon and BangaloreExperience: 8+ years (data engineering / analytics engineering), with previous lead responsibilitiesJob Descriptions for Big data or Cloud EngineerWe are looking for candidates with hands on experience in pyspark with GCP cloud.Qualifications3-10 years of IT experience range is preferred.Able to effectively use GCP managed...

  • GCP Data Engineer

    2 weeks ago

    Bangalore Urban, Karnataka, India, IN Impetus Full time

    Job Title: GCP Data EngineerLocation: BengaluruExperience: 3-8 yearsAbout the Role:Design, build, and maintain large-scale data pipelines on BigQuery and other Google Cloud Platform (GCP) services.Use Python and PySpark/Spark to transform, clean, aggregate and prepare data for analytics/ML.Orchestrate workflows using Cloud Composer (Airflow) to schedule,...

  • Freelance Data Quality Engineer

    2 weeks ago

    india, IN Leading MNC Part time

    Looking for a Freelance Data Quality Engineer to join a team of rockstar developers. The candidate should have a minimum of 8+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Freelance Data Quality Engineer

    2 weeks ago

    india, IN Leading MNC Part time

    Looking for a Freelance Data Quality Engineer to join a team of rockstar developers. The candidate should have a minimum of 8+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Senior Data Engineer

    1 week ago

    india, IN MindBrain Full time

    Job Title: Data Engineer (Databricks + GCP) Experience: 5+ yearsLocation: RemoteOverview: We are looking for a Data Engineer with strong experience in Databricks and Google Cloud Platform (GCP) to design, build, and optimize scalable data pipelines and Delta Lake environments.Key Responsibilities:Develop and maintain ETL pipelines using PySpark and...

  • Lead GCP Data Engineer

    2 weeks ago

    Gurugram, Haryana, India, IN Impetus Full time

    Job Title: Lead Data Engineer – GCP (BigQuery • Composer • Python • PySpark)Location: GurgaonExperience: 8+ years (data engineering / analytics engineering), with previous lead responsibilitiesAbout the Role:You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform. You will manage a team of data...