Senior GCP Data Engineer
3 weeks ago
Role: Sr GCP Data Engineer (Google Cloud Platform)
Location: Remote - India
Job description:
• Develop, construct, test and maintain data acquisition pipelines for large volumes of structed and unstructured data. This includes batch and real-time processing (in google cloud)
• Build large and complex datasets based on business requirements
• Construct ‘big data’ pipeline architecture
• Identify opportunities for data acquisition via working with stakeholders and business clients
• Translate business needs to technical requirements
• Leverage a variety of tools in the Google Cloud Ecosystem such as Python, Data Flow, DataStream, CDC (Change Data Capture), Cloud Functions, Cloud Run, Pub Sub, BigQuery, Cloud Storage to integrate systems and data pipelines
• Use logs & alerts to effectively monitor pipelines
• Use SAP SLT to replicate SAP tables to Google Cloud using SLT
• Develop JSON messaging structure for integrating with various applications
• Leverage DevOps and CI/CD practices (GitHub, Terraform) to ensure the reliability and scalability of data pipelines
• Partition/Cluster and retrieve content in Big Query and use IAM roles & Policy Tags to secure the data
• Use roles to secure access to datasets, authorized views to share data between projects
• Design and build an ingestion pipeline using Rest API
• Recommends ways to improve data quality, reliability, and efficiency
Experience Requires:
• 8+ years of experience in a GCP data engineering role (offshore)
• Atleast 1 year of hands on experience in GCP tools highlighted(Python, SQL, Data Flow, DataStream, CDC (Change Data Capture), Cloud Functions, Cloud Run, Pub Sub, BigQuery, Cloud Storage)
-
GCP Data Engineer
2 days ago
Thiruvananthapuram, Kerala, India VME Vhire Solutions Full time ₹ 10,00,000 - ₹ 25,00,000 per yearPermanent with Client - MNCContact - About ClientBeing the leading provider of Warehouse logistics solutions, we are gearing up to revolutionize our datalandscape by building a cutting-edge Enterprise Data Lakehouse Platform. We are forming multiple teams thatwill spearhead the creation of the platform's foundational components. These teams go beyond...
-
Senior Data Engineer
1 week ago
Thiruvananthapuram, Kerala, India Quantiphi Full time ₹ 6,00,000 - ₹ 18,00,000 per yearDescription : While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional...
-
Senior Data Engineer
1 week ago
Thiruvananthapuram, India HireAlpha Full timeSenior Data Engineer (Informatica Platform Engineer / Administrator) – Bangalore We’re looking for a Senior Data Engineer with strong expertise in Informatica platform administration to join our Group IT Data Engineering team.This role is ideal for professionals who enjoy building and maintaining enterprise-grade data platforms that power large-scale...
-
Data Engineer
6 days ago
Thiruvananthapuram, India UST Full timeGreetings from USTWho we are:Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences and human-centered solutions for a better world.UST is a mission-driven group of over 38,000+ practical problem...
-
Data Engineer
1 week ago
Thiruvananthapuram, India UST Full timeGreetings from USTWho we are:Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences and human-centered solutions for a better world.UST is a mission-driven group of over 38,000+ practical problem...
-
Data Engineer
2 weeks ago
thiruvananthapuram, India UST Full timeGreetings from UST Who we are: Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences and human-centered solutions for a better world.UST is a mission-driven group of over 38,000+ practical problem...
-
Data Engineer
1 week ago
Thiruvananthapuram, India UST Full timeGreetings from UST Who we are: Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences and human-centered solutions for a better world.UST is a mission-driven group of over 38,000+ practical problem...
-
Data Engineer
7 days ago
Thiruvananthapuram, India UST Full timeGreetings from UST Who we are: Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences and human-centered solutions for a better world. UST is a mission-driven group of over 38,000+ practical problem...
-
Data Engineer
4 days ago
thiruvananthapuram, India UST Full timeGreetings from UST Who we are: Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences and human-centered solutions for a better world. UST is a mission-driven group of over 38,000+ practical problem...
-
Data Engineer
1 week ago
Thiruvananthapuram, India UST Full timeGreetings from UST Who we are: Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences and human-centered solutions for a better world. UST is a mission-driven group of over 38,000+ practical problem...
