
Consultant – Cloud Data Engineer – GCP
6 days ago
Key Responsibilities:
- Design, develop, and manage scalable, secure, and efficient data pipelines on Google Cloud Platform (GCP) to process and transform large datasets from multiple sources.
- Implement and optimize GCP data services such as BigQuery, Cloud Storage, Cloud SQL, Dataflow, Dataproc, Pub/Sub, and Cloud Functions.
- Architect and maintain ETL/ELT processes to ensure efficient data ingestion, transformation, and storage using Cloud Data Fusion, Dataflow (Apache Beam), or Composer (Airflow).
- Architect and implement data models on GCP to support efficient data storage and retrieval in BigQuery, Cloud Spanner, or Firestore.
- Collaborate with data architects to design and implement data lakes, data warehouses, and data marts on GCP.
- Build and maintain data integration workflows using tools like Cloud Composer (Apache Airflow), Cloud Functions, or Dataflow.
- Utilize GCP DevOps tools for source control, build automation, release management, and infrastructure as code (IaC) using Terraform or Deployment Manager.
- Implement CI/CD pipelines within Google Cloud Build to automate the build, test, and deployment of data pipelines and infrastructure changes, ensuring rapid and reliable delivery of data solutions.
- Utilize big data technologies such as Dataproc (Apache Spark, Hadoop, etc.) to handle large volumes of data and perform complex analytics tasks.
- Implement real-time data processing solutions using streaming technologies like Pub/Sub and Dataflow (Apache Beam) to enable timely insights and actions.
- Implement data governance policies and security controls within GCP environments to protect sensitive data and ensure compliance with regulatory requirements such as GDPR, HIPAA, or PCI DSS.
- Optimize data pipelines and processing workflows for performance, scalability, and cost-effectiveness on GCP, leveraging BigQuery optimizations, Autoscaling, and cost management techniques.
- Monitor, troubleshoot, and optimize the performance of data pipelines and cloud-based systems to ensure high availability, low latency, and scalability.
- Work closely with cross-functional teams to understand business requirements and translate them into scalable GCP data solutions.
- Ensure data security and compliance with industry best practices, including IAM roles, VPC configurations, encryption, and access controls.
- Conduct regular code reviews, provide feedback, and ensure adherence to best practices and standards.
- Stay up to date with the latest GCP services and cloud technologies to drive innovation within the team.
Job Qualifications:
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- 2-3 years of experience as a Cloud Data Engineer or similar role, with a strong focus on GCP cloud services.
- Proficiency in designing and building scalable data architectures using GCP data services such as BigQuery, Cloud Storage, Dataflow, Cloud SQL, and Dataproc.
- Strong experience with ETL/ELT frameworks and tools like Cloud Composer (Apache Airflow), Cloud Data Fusion, or Dataflow (Apache Beam).
- Expertise in SQL, Python, or other programming languages used in data engineering.
- Hands-on experience with data lakes, data warehouses, and data pipeline orchestration on GCP.
- Familiarity with CI/CD pipelines and infrastructure-as-code (IaC) tools like Terraform, Deployment Manager, or Cloud Build.
- Understanding data governance, security, and compliance standards, including encryption, IAM policies, and role-based access control (RBAC).
- Experience in data modeling, data normalization, and performance optimization for cloud-based data solutions.
- GCP Certifications such as Google Cloud Professional Data Engineer or Google Cloud Professional Cloud Architect.
- Experience with Apache Spark or Databricks on GCP.
- Knowledge of machine learning workflows and working with data science teams is a plus.
- Familiarity with DevOps practices and tools such as Docker and Kubernetes.
- Excellent communication and collaboration skills, with the ability to work effectively in a team environment.
- Strong problem-solving and troubleshooting skills, with a proactive approach to identifying and resolving issues.
- Ability to adapt to a fast-paced, agile environment and manage multiple priorities effectively.
-
Gcp Data Engineer
3 weeks ago
Guntur, Andhra Pradesh, India Tech T7 Innovations Full timeJob Title: Google Cloud (GCP) Data EngineerLocation: RemoteJob Type: Full-TimeExperience Level: Minimum 5 years +Mandatory Skills: GCS + Google BigQuery + Airflow/Composer + PythonCompany Description:Tech T7 Innovations is a company that provides IT solutions to clients worldwide. The team consists of highly skilled and experienced professionals who are...
-
GCP Data Architect
3 weeks ago
Guntur, Andhra Pradesh, India HCLTech Full timeRole: GCP Data ArchitectTotal years of experience: 12+ YearsLocation: Noida, Bangalore, Chennai, Pune, HyderabadRelevant experience for engagement: 10 YearsRelevant skillset:Maintain architecture principles, guidelines and standardsData WarehousingProgramming Language: Python/JavaBig DataData AnalyticsGCP ServicesRelevant technology domains:Experience in...
-
GCP Platform Engineer
2 weeks ago
Guntur, India PamTen Inc Full timeRemote within India Full Time roleThe IT Cloud Platform Services teams is a dynamic mix of tech enthusiasts, problem solvers, and creative thinkers, united by our passion for leveraging cutting-edge technology to transform healthcare and empower our customers to take real-time control of their health.Where you come in:As a Cloud Platform Engineer 2, you will...
-
AWS Data Engineer
3 weeks ago
Guntur, Andhra Pradesh, India Teamware Solutions Full timeAbout the CompanyGreetings from Teamware Solutions a division of Quantum Leap Consulting Pvt. LtdAbout the RoleWe are hiring an AWS Data EngineerLocation: BangaloreWork Model: HybridExperience: 5-7 yearsNotice Period: Immediate- 15 daysOpen roles: 22Data Engineer: Data Engineer, DevOps, Data Modeller, QA, Business systems analyst, Data Analyst, DBAAs an AWS...
-
Cloud Security Architecture
4 weeks ago
Guntur, Andhra Pradesh, India ITC Infotech Full timeITC Infotech is looking for Cloud Solution Architect- Bangalore.Job DescriptionExperience with Security Architecture, out of which at least 5 Years of experience on cloud platforms - Amazon Web Services (AWS), Microsoft Azure, and GCPDesigning, developing, and building security reference architectures for public, private, and hybrid Cloud based systems...
-
Senior data engineer
2 weeks ago
Guntur, India Evolent Full timeResponsibilities:1. Lead the design, development, and deployment of complex data solutions, ensuring scalability and maintainability.2. Architect and implement high-performance data pipelines and data warehousing solutions that support critical business functions.3. Collaborate closely with cross-functional teams, including product managers, UX designers,...
-
AWS Data Engineer
3 weeks ago
Guntur, Andhra Pradesh, India 9NEXUS Full timeWe are seeking a skilled AWS Data Engineer to design, build, and maintain scalable data pipelines and ETL solutions in a cloud environment. The ideal candidate will have hands-on experience with AWS EMR, Glue, , PySpark, Airflow and Python and will work closely with data analysts, data scientists, and other engineering teams to support data-driven...
-
SAP Analytics Cloud
3 weeks ago
Guntur, Andhra Pradesh, India VSquare Systems Pvt. Ltd. Full timeJD 1: SAP Analytics Cloud (SAC) Consultant (3–5 Years Experience)Experience: 3–5 YearsLocation: PuneNotice Period: Immediate to 2 weeksRole OverviewWe are seeking a motivated SAC Consultant with 3–5 years of experience in developing and supporting analytics solutions using SAP Analytics Cloud. The candidate should be skilled in building interactive...
-
Lead Security Engineer
2 weeks ago
Guntur, India Arcana Full timeAs our Lead Security Engineer, you'll own and elevate Arcana's overall security posture - cloud, on-prem, and everything in between. You'll design and enforce policies, automate controls, and harden infrastructure end-to-end. While your primary focus will be on our GCP resources, you'll also partner with teams across networking, applications, and compliance...
-
Cloud Engineer
3 weeks ago
Guntur, Andhra Pradesh, India Alp Consulting Ltd. Full timeBachelor's degree in Computer Science, Information Systems, or a related field.6+ years of experience with Azure Data Factory and other Azure Data Services.Proficient in SQL, T-SQL, and experience with stored procedures and views.Strong understanding of ETL/ELT principles, data integration, and data transformation.Hands-on experience with Azure Data Lake,...