Freelance GCP Data Engineer

2 weeks ago

dindigul, India ThreatXIntel Full time

Company DescriptionThreatXIntel is a startup specializing in cybersecurity, offering tailored and cost-effective solutions for businesses and organizations of all sizes. Our expertise spans cloud security, web and mobile security testing, cloud security assessments, DevSecOps, and more. Driven by a proactive approach, we monitor and test our clients' digital environments to identify vulnerabilities before they can be exploited. Our mission is to deliver reliable and accessible cybersecurity services, empowering businesses to focus on growth while safeguarding their digital assets. Based on a commitment to quality, our team strives to provide peace of mind through customized protection strategies.Role DescriptionWe are seeking a skilled Freelance GCP Data Engineer with expertise in Apache NiFi, SQL, and API integrations to design, implement, and optimize real-time data pipelines. This role involves working with the Google Cloud Platform (GCP) ecosystem, including BigQuery, Pub/Sub, Dataflow, and Cloud Storage, to ensure high-performance data processing, transformation, and delivery to downstream systems for analytics and reporting.Key ResponsibilitiesDesign and implement real-time data ingestion pipelines using Apache NiFi for structured and unstructured data.Integrate data pipelines with the GCP ecosystem, including BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer/Airflow.Ensure low-latency, high-throughput data delivery to downstream systems for analytics and reporting.Develop and optimize SQL queries for BigQuery and relational databases to ensure data transformation and analysis.Write Python scripts for data transformation, automation, and custom NiFi processors when required.Implement data quality, validation, and error-handling mechanisms within ingestion pipelines to ensure reliability and accuracy.Collaborate with data analysts, data scientists, and platform engineers to deliver consistent and high-quality datasets.Ensure data security, governance, and compliance with GCP IAM, VPC Service Controls, and encryption standards.Monitor, troubleshoot, and optimize data pipeline performance to ensure smooth operations.Document data flows, architecture, and best practices to maintain consistency across teams.Required Skills & QualificationsProven experience with Apache NiFi for real-time ingestion, routing, and transformation of data.Strong expertise in the GCP ecosystem, including:BigQuery (data warehousing)Pub/Sub (event streaming)Dataflow/Apache Beam (data processing)Cloud Storage (data lake)Composer/Airflow (workflow orchestration)Strong SQL skills for querying, modeling, and performance optimization of data.Proficiency in Python for data transformation, automation, and creating custom NiFi processors.Experience with real-time streaming concepts, such as windowing, late-arriving data, and deduplication.Strong understanding of ETL/ELT processes, data lakes, and data warehouses.Familiarity with CI/CD practices and version control using Git.Understanding of data security, governance, and compliance within cloud environments.

  • Freelance Data Architect

    3 weeks ago

    Dindigul, India Leading MNC Full time

    Looking for a Freelance Data Architect to join a team of rockstar developers. The candidate should have a minimum of 12+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report into...

  • Data Solution Architect

    4 weeks ago

    Dindigul, India Whatjobs IN C2 Full time

    Job Title: Data Solutions Architect – Google Cloud Platform (GCP) Location: Noida/ Bangalore Job Type: Full Time Experience: 7+ Years Industry: IT Services & Analytics Job Description We are seeking a highly experienced and strategic Data Solutions Architect – Google Cloud Platform (GCP) to join our technology team. This pivotal role will be responsible...

  • GCP Big Data Engineer

    3 weeks ago

    Dindigul, India HARP Full time

    Job Summary:We are looking for an experienced and motivated GCP Big Data Engineer to join our team in a leadership capacity. The ideal candidate will have 5 years of relevant experience in data engineering, with a strong focus on Google Cloud Platform (GCP), SQL, PySpark, and ETL processes. This role requires strong technical expertise and leadership...

  • GCP Big Data Engineer

    3 weeks ago

    Dindigul, India HARP Full time

    Job Summary:We are looking for an experienced and motivated GCP Big Data Engineer to join our team in a leadership capacity. The ideal candidate will have 5 years of relevant experience in data engineering, with a strong focus on Google Cloud Platform (GCP), SQL, PySpark, and ETL processes. This role requires strong technical expertise and leadership...

  • Lead Large-Scale Data Engineering Projects

    3 days ago

    dindigul, India beBeeDataEngineer Full time

    Senior Data Engineer JobJob Title: Senior Data Engineer – Python Expert (Freelance)Location: Remote / HybridEmployment Type: Contract/ FreelanceRole SummaryWe are seeking a seasoned Senior Data Engineer to design, develop, and own robust data pipelines that power our large language model development. As a senior individual contributor, you will be the...

  • Data Engineer

    7 days ago

    dindigul, India Ignisov Consulting Services Full time

    Role: Data Engineer – HR Data ConversionJob Type: ContractJob Location: RemoteExperience: 5–10 yearsJob Description:Key Responsibilities:Develop and maintain ETL pipelines using Python and AirflowWork with SQL, GCP BigQuery, and cloud-based data workflowsBuild scalable solutions using Spark, Hive, HDFSDesign and optimize data models for HR data...

  • Data Engineer Professional

    1 week ago

    dindigul, India beBeeEngineering Full time

    Data Engineering ProfessionalAbout the RoleOur organization seeks a skilled data professional to handle data engineering tasks. The ideal candidate will have hands-on experience in building scalable data pipelines, designing data warehousing solutions, and working with modern cloud-based frameworks.Key Responsibilities:Design, build, and optimize ETL/ELT...

  • Senior Data Engineering Lead

    2 weeks ago

    dindigul, India beBeeDataEngineering Full time

    Project Manager - Data EngineeringThe role of a Project Manager in data engineering and analytics involves leading and delivering projects.Key Responsibilities:Project Delivery: Plan, execute, and deliver technical projects on time and within scope.Data Pipelines: Strong understanding of data pipelines, ETL/ELT, and analytics workflows.Collaboration:...

  • Leading Data Engineering Expert

    1 week ago

    dindigul, India beBeeDataEngineer Full time

    Data Engineer OpportunityWe are seeking a skilled data engineer to join our team.The ideal candidate will have extensive experience with SQL, Ab-Initio, Teradata, and Google Cloud Platform (GCP), as well as strong expertise in designing, developing, and optimizing large-scale data pipelines. They will be responsible for implementing scalable, reliable, and...

  • Data Quality Professional

    2 weeks ago

    dindigul, India beBeeData Full time

    Freelance Data Quality ExpertAs a Freelance Data Quality Engineer, you will work on multiple projects with experienced developers. With 8+ years of experience in data quality, you'll have the flexibility to work 35 hours a week, 100% remotely.This opportunity offers a long-term engagement for high performers.Must-Have Skills:Python: Develop clean, automated...