Freelance GCP Data Engineer
3 days ago
Company DescriptionThreatXIntel is a startup specializing in cybersecurity, offering tailored and cost-effective solutions for businesses and organizations of all sizes. Our expertise spans cloud security, web and mobile security testing, cloud security assessments, DevSecOps, and more. Driven by a proactive approach, we monitor and test our clients' digital environments to identify vulnerabilities before they can be exploited. Our mission is to deliver reliable and accessible cybersecurity services, empowering businesses to focus on growth while safeguarding their digital assets. Based on a commitment to quality, our team strives to provide peace of mind through customized protection strategies.Role DescriptionWe are seeking a skilled Freelance GCP Data Engineer with expertise in Apache NiFi, SQL, and API integrations to design, implement, and optimize real-time data pipelines. This role involves working with the Google Cloud Platform (GCP) ecosystem, including BigQuery, Pub/Sub, Dataflow, and Cloud Storage, to ensure high-performance data processing, transformation, and delivery to downstream systems for analytics and reporting.Key Responsibilities- Design and implement real-time data ingestion pipelines using Apache NiFi for structured and unstructured data. - Integrate data pipelines with the GCP ecosystem, including BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer/Airflow. - Ensure low-latency, high-throughput data delivery to downstream systems for analytics and reporting. - Develop and optimize SQL queries for BigQuery and relational databases to ensure data transformation and analysis. - Write Python scripts for data transformation, automation, and custom NiFi processors when required. - Implement data quality, validation, and error-handling mechanisms within ingestion pipelines to ensure reliability and accuracy. - Collaborate with data analysts, data scientists, and platform engineers to deliver consistent and high-quality datasets. - Ensure data security, governance, and compliance with GCP IAM, VPC Service Controls, and encryption standards. - Monitor, troubleshoot, and optimize data pipeline performance to ensure smooth operations. - Document data flows, architecture, and best practices to maintain consistency across teams.Required Skills & Qualifications- Proven experience with Apache NiFi for real-time ingestion, routing, and transformation of data. - Strong expertise in the GCP ecosystem, including: - BigQuery (data warehousing) - Pub/Sub (event streaming) - Dataflow/Apache Beam (data processing) - Cloud Storage (data lake) - Composer/Airflow (workflow orchestration) - Strong SQL skills for querying, modeling, and performance optimization of data. - Proficiency in Python for data transformation, automation, and creating custom NiFi processors. - Experience with real-time streaming concepts, such as windowing, late-arriving data, and deduplication. - Strong understanding of ETL/ELT processes, data lakes, and data warehouses. - Familiarity with CI/CD practices and version control using Git. - Understanding of data security, governance, and compliance within cloud environments.
-
Freelance – Apache NiFi
17 hours ago
New Delhi, India ThreatXIntel Full timeCompany Description ThreatXIntel is a dynamic cybersecurity startup dedicated to protecting businesses and organizations against cyber threats. Specializing in services such as cloud security, DevSecOps, and vulnerability testing, the company offers tailored, cost-effective solutions to meet clients' unique requirements. Focused on startups and small...
-
Freelance Data Engineer
3 weeks ago
New Delhi, India Leading MNC Full timeLooking for a Freelance Data Engineer to join a team of rockstar developers. The candidate should have a minimum of 8+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report into...
-
Freelance Data Engineer
3 weeks ago
New Delhi, India Leading MNC Full timeLooking for a Freelance Data Engineer to join a team of rockstar developers. The candidate should have a minimum of 8+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report into...
-
Freelance – Apache NiFi
2 days ago
Delhi, India ThreatXIntel Full timeCompany DescriptionThreatXIntel is a dynamic cybersecurity startup dedicated to protecting businesses and organizations against cyber threats. Specializing in services such as cloud security, DevSecOps, and vulnerability testing, the company offers tailored, cost-effective solutions to meet clients' unique requirements. Focused on startups and small...
-
GCP Data Engineer
3 days ago
New Delhi, India Intellistaff Services Pvt. Ltd Full timeJob description:Data Engineer with Python and GCP Experience Level:5 to 9 Years Loc:ChennaiMust Have Skillset SQL (4+ Years) Python or PySpark (4+ Years) GCP services (3+ Years) BigQuery Dataflow or Dataproc Pub/Sub Scheduled Query Cloud Functions Monitoring Tools Dashboard Apache Kafka Terraform scripting (2+ Year) Airflow/Astronomer/Cloud Composer (2+...
-
GCP Data Engineer
3 days ago
New Delhi, India Intellistaff Services Pvt. Ltd Full timeJob description:Data Engineer with Python and GCP Experience Level:5 to 9 Years Loc:ChennaiMust Have Skillset SQL (4+ Years) Python or PySpark (4+ Years) GCP services (3+ Years) BigQuery Dataflow or Dataproc Pub/Sub Scheduled Query Cloud Functions Monitoring Tools Dashboard Apache Kafka Terraform scripting (2+ Year) Airflow/Astronomer/Cloud Composer (2+...
-
GCP Data Engineer
3 weeks ago
New Delhi, India Adastra Full timeJob Description We are looking for a proactive and solution-oriented GCP Data Engineer to join our team. This role requires hands-on experience in Google Cloud Platform (GCP), especially with BigQuery and Airflow, alongside strong SQL and Python skills. The ideal candidate is a self-starter who thrives in solving problems independently, communicates...
-
Freelance – Apache NiFi
2 days ago
Delhi, India ThreatXIntel Full timeCompany DescriptionThreatXIntel is a dynamic cybersecurity startup dedicated to protecting businesses and organizations against cyber threats. Specializing in services such as cloud security, DevSecOps, and vulnerability testing, the company offers tailored, cost-effective solutions to meet clients' unique requirements. Focused on startups and small...
-
GCP Data engineer
2 weeks ago
New Delhi, India Tata Consultancy Services Full timeWe await your innovation at TCS: Hiring | GCP Data Engineer|Greetings from TCS!! We are Hiring for GCP Data EngineerRequired Experience: 5-10 yearsWork location: Hyderabad, Chennai, Bangalore, DelhiSkills: GCP, Big Query, Dataflow, Data Proc, Cloud SQL, Cloud Spanner, Pub/Sub, ETL, Apache airflow, Python, Pandas and NumphyJD:- Google Data Engineer (Overall...
-
GCP data Engineer
2 weeks ago
New Delhi, India Impetus Full timeLocation: Gurgaon and BangaloreExperience: 8+ years (data engineering / analytics engineering), with previous lead responsibilitiesJob Descriptions for Big data or Cloud EngineerWe are looking for candidates with hands on experience in pyspark with GCP cloud.Qualifications- 3-10 years of IT experience range is preferred. - Able to effectively use GCP managed...
