GCP Data Pipeline Engineer

Company DescriptionZilo AI is a prominent manpower service provider dedicated to connecting businesses with highly skilled and dependable professionals across various industries. With a deep understanding that talent drives success, Zilo AI is committed to supplying the right expertise to support business growth. Our focus on talent optimization ensures that clients receive tailored solutions to meet their business needs. Joining Zilo AI provides an opportunity to work within a forward-thinking organization that values innovation and excellence.We Are Hiring: Data Engineer (Google Cloud Platform)Location: Tier 1 Cities-On-site (Mumbai, Delhi, Bengaluru, Chennai, Hyderabad, Kolkata, Pune and Ahmedabad)Are you a Data Engineer with strong hands-on experience in Google Cloud Platform (GCP) and large-scale distributed data pipelines? We are looking for a highly skilled professional who can design, build, and maintain production-grade data systems using Spark, PySpark, BigQuery, GCS, and modern orchestration tools.If you thrive in complex data environments and enjoy optimizing large-scale pipelines, this is the role for you.Role: Data Engineer (GCP – Spark / PySpark / BigQuery)As a Data Engineer, you will build and scale our cloud-native data ecosystem, develop robust ETL/ELT pipelines, and enable advanced analytics across the organization. You will work closely with data architects, analysts, and product teams to deliver high-impact data solutions.Mandatory Skill Sets (Must-Have Experience)Google Cloud Platform (GCP) Expertise✔ Hands-on experience working with Google Cloud Platform ✔ Experience using GCS (Google Cloud Storage) in data pipelines ✔ Experience with BigQuery for data modeling or analytics ✔ Experience running data processing jobs using DataFlow or DataProc ✔ Familiarity with Stackdriver / Cloud Logging for monitoring and logging ✔ Holding a GCP Certification is a strong plusSpark / PySpark & Distributed Processing✔ Strong experience writing production-grade Spark jobs using PySpark ✔ Ability to optimize Spark workloads (partitioning, caching, join strategies) ✔ Experience processing large datasets (100GB+ or distributed) ✔ Ability to share a PySpark code sample for: Ingesting CSV → cleaning fields → adding full_name column → writing Parquet outputData Pipeline Development✔ Proven experience building end-to-end ETL/ELT pipelines in production ✔ Hands-on experience with Python for data processing ✔ Experience with Airflow or Cloud Composer for orchestration ✔ Ability to troubleshoot pipeline failures or data inconsistencies ✔ Experience implementing security best practices in data pipelinesData Modelling & Storage✔ Experience creating data models (star schema, snowflake, or custom designs) ✔ Experience working with file formats such as JSON, AVRO, PARQUET ✔ Ability to design scalable and efficient data models in analytical systemsData Quality, Analytics & SQL✔ Experience implementing data quality checks within pipelines ✔ Strong SQL capability, including writing complex analytical queries ✔ Experience working with credit or financial datasets is a plusResponsibilities-Design and develop scalable data pipelines using Spark, PySpark, and GCP-Build and optimize ELT workflows using BigQuery and Cloud Storage-Implement orchestration using Airflow/Cloud Composer-Ensure data reliability, quality, and security across all workflows-Troubleshoot performance bottlenecks and optimize distributed jobs-Collaborate with data architects and analysts on data modeling and analytics-Monitor pipeline health using Google Cloud monitoring toolsPreferred Qualifications-Degree in Computer Science, Engineering, or related field-Experience with streaming (Pub/Sub / Kafka)-Experience integrating CI/CD into data workflows-Knowledge of cloud cost optimizationWhat We Offer-Opportunity to work with a modern, cloud-native data stack-Exposure to scalable GCP technologies and distributed computing-Competitive compensation & benefits-Collaborative and fast-growing data teamHow to ApplyIf you're excited to work on cutting-edge cloud data engineering challenges, we’d love to review your application. Please apply with your updated resume by filling up this short FormForm Link: https://forms.Gle/r9idZQLeuy8vwNYX6