GCP Data Engineering

2 months ago

hyderabad, India BILVANTIS TECHNOLOGIES Full time

We are looking for a skilled and experienced GCP Data Lead Engineer to join our Data team in Hyderabad location.


About the job


Position: GCP Data Engineering - Lead Programmer Analyst

Total Experience: Minimum 5 yrs to 7 yrs of experience.

Notice Period: Immediate -15 days

Must have skills: GCP Cloud, GCS, BigQuery, Python, PySpark, DWH Architecture, ETL, SQL


Job Description: GCP Data Engineering - Lead Programmer Analyst


We are seeking a highly skilled and knowledgeable Data Engineer to join our Data Management team on a transformative Move to Cloud (M2C) project. The ideal candidate will have a strong background in creating robust data ingestion pipelines and a deep understanding of ETL processes, particularly within the Google Cloud Platform ecosystem and using tools such as dbt Labs over BigQuery.


Responsibilities:

  • Develop and maintain scalable and reliable data pipelines using PySpark and SQL to support the migration from Oracle's on-premises data warehouse (structured data source) and unstructured data sources to BigQuery.
  • Design and implement bulletproof data ingestion and integration processes that ensure data quality and consistency.
  • Use the dbt tool to create and manage ETL processes that transform and load data efficiently from/into BigQuery.
  • Ensure the resilience and efficiency of data transformation jobs that can handle large volumes of data within a cloud-based architecture.
  • Work closely with our Data Engineers to gather requirements for the currently developed data pipelines.
  • Provide expertise in GCP services like DataProc, DataFlow, Cloud Functions, Workflows, Cloud Composer, and BigQuery, advocating for best practices in cloud-based data management.
  • Collaborate with data architects and other stakeholders to optimize data models and warehouse design for the cloud environment.
  • Develop and implement monitoring, quality, and validation processes to ensure the integrity of data pipelines and data.
  • Document all data engineering processes and create clear specifications for future reference and compliance.


Requirements:

  • Bachelor's or Master’s degree in Computer Science, Engineering, or a related field.
  • Minimum of 5 years of experience as a Data Engineer with a focus on cloud-based solutions.
  • Proficient in GCP services, with a strong emphasis on data-related products such as DataProc, DataFlow, Cloud Functions, Workflows, Cloud Composer, and BigQuery.
  • Extensive experience with ETL tools, particularly dbt Labs, and a clear understanding of ETL best practices.
  • Experience in building and optimizing data pipelines, architectures, and data sets from structured/unstructured data sources.
  • Strong analytical skills with the ability to understand complex requirements and translate them into technical solutions.
  • Excellent problem-solving abilities and a commitment to quality.
  • Strong communication skills, with the ability to work collaboratively in a team environment.
  • Relevant certifications in the Google Cloud Platform or other data engineering credentials are desirable.
  • Proficiency in SQL and Python with knowledge of Spark.
  • Fluent in English, with strong written and verbal communication skills.

  • Gcp data engineer

    7 days ago

    Hyderabad, India Tata Consultancy Services Full time

    Role - GCP Data Engineer Experience - 5 to 12 yrsLocation - Chennai, Hyderabad, Bangalore, Pune, GurgaonMust Have -Big query, Dataflow , Cloud SQL , Cloud composer , Python , Vertex and Data flowMust-Have· Google Data Engineer (Overall 5+ Years with 2 years relevant experience)· Experience working in GCP based Big Data deployments (Batch/Realtime)...

  • GCP Data Engineer

    6 days ago

    Hyderabad, India Tata Consultancy Services Full time

    Role - GCP Data EngineerExperience - 5 to 12 yrsLocation - Chennai,Hyderabad,Bangalore,Pune,GurgaonMust Have -Big query, Dataflow , Cloud SQL , Cloud composer , Python , Vertex and Data flowMust-Have· Google Data Engineer (Overall 5+ Years with 2 years relevant experience)· Experience working in GCP based Big Data deployments (Batch/Realtime) leveraging...

  • Data Engineer- GCP

    1 month ago

    Hyderabad, India NucleusTeq Full time

    Experience- 6 Year + Location- Indore/Bangalore/Hyderabad/Raipur Role Description This is a full-time hybrid role for a Data Engineer GCP at NucleusTeq . The Data Engineer will be responsible for tasks such as GCP cloud Migration ,data engineering, data modeling, ETL processes, data warehousing to...

  • GCP Data Engineer

    1 month ago

    hyderabad, India Tekgence Inc Full time

    1 of our MNC client is looking for GCP Data Engineers on an urgent basis. Kindly find below the details for your reference. Role: GCP Data Engineer Experience: 5 - 11 Years Notice Period: Immediate - 30 days max Budget: 20 - 32 LPA max Location: Pune/Chennai/Hyderabad Mandate Skills: GCP BigQuery, SQL Scripting & Python coding experience. GCP...

  • GCP Data Engineer

    1 month ago

    hyderabad, India Tekgence Inc Full time

    1 of our MNC client is looking for GCP Data Engineers on an urgent basis. Kindly find below the details for your reference.Role: GCP Data EngineerExperience: 5 - 11 YearsNotice Period: Immediate - 30 days maxBudget: 20 - 32 LPA maxLocation: Pune/Chennai/HyderabadMandate Skills:GCP BigQuery, SQL Scripting & Python coding experience.GCP...

  • GCP Data Engineer

    4 weeks ago

    Hyderabad, India HCLTech Full time

    Job description:About the Role:We are looking for a skilled Near Real-Time (NRT) Data Engineer with expertise in Google BigQuery to join our dynamic Data Engineering team.The successful candidate will be responsible for designing, implementing, and maintaining near real-time data pipelines and solutions that empower our organization with data-driven...

  • GCP Data Engineer

    4 weeks ago

    hyderabad, India HCLTech Full time

    Job description: About the Role: We are looking for a skilled Near Real-Time (NRT) Data Engineer with expertise in Google BigQuery to join our dynamic Data Engineering team. The successful candidate will be responsible for designing, implementing, and maintaining near real-time data pipelines and solutions that empower our organization with data-driven...

  • GCP Data Engineer

    4 weeks ago

    hyderabad, India HCLTech Full time

    Job description:About the Role:We are looking for a skilled Near Real-Time (NRT) Data Engineer with expertise in Google BigQuery to join our dynamic Data Engineering team.The successful candidate will be responsible for designing, implementing, and maintaining near real-time data pipelines and solutions that empower our organization with data-driven...

  • Gcp data engineer

    4 weeks ago

    Hyderabad, India HCLTech Full time

    Job description:About the Role:We are looking for a skilled Near Real-Time (NRT) Data Engineer with expertise in Google Big Query to join our dynamic Data Engineering team.The successful candidate will be responsible for designing, implementing, and maintaining near real-time data pipelines and solutions that empower our organization with data-driven...

  • GCP Data Engineer

    4 weeks ago

    Hyderabad, India HCLTech Full time

    Job description:About the Role:We are looking for a skilled Near Real-Time (NRT) Data Engineer with expertise in Google BigQuery to join our dynamic Data Engineering team.The successful candidate will be responsible for designing, implementing, and maintaining near real-time data pipelines and solutions that empower our organization with data-driven...

  • GCP Data Engineer

    4 weeks ago

    Hyderabad, India HCLTech Full time

    Job description:About the Role:We are looking for a skilled Near Real-Time (NRT) Data Engineer with expertise in Google BigQuery to join our dynamic Data Engineering team.The successful candidate will be responsible for designing, implementing, and maintaining near real-time data pipelines and solutions that empower our organization with data-driven...

  • GCP Data engineer

    11 hours ago

    Hyderabad, India People Tech Group Inc Full time

    Hello connections,We are hiring '' GCP Data Engineer '' and below are the requirement. This will be through ''People Tech Group Company''. Role DescriptionPosition: GCP Data engineerCompany : People Tech GroupExperience: 3yrs +Location : Hyderabad (Hitech CITY )Work Location: HyderabadMode of Work: WFO Roles and responsibilities:Develop and maintain data...

  • GCP Data engineer

    5 hours ago

    Hyderabad, India People Tech Group Inc Full time

    Hello Linkedin connections, We are hiring '' GCP Data Engineer '' and below are the requirement. This will be through ''People Tech Group Company''. Role Description Position: GCP Data engineer Company : People Tech Group Experience: 3yrs + Location : Hyderabad (Hitech CITY ) Work Location: Hyderabad Mode of Work: WFO Roles and...

  • GCP Data engineer

    17 hours ago

    Hyderabad, India People Tech Group Inc Full time

    Hello Linkedin connections,We are hiring '' GCP Data Engineer '' and below are the requirement. This will be through ''People Tech Group Company''. Role DescriptionPosition: GCP Data engineerCompany : People Tech GroupExperience: 3yrs +Location : Hyderabad (Hitech CITY )Work Location: HyderabadMode of Work: WFO Roles and responsibilities:Develop and maintain...

  • GCP Data Engineering Lead

    4 weeks ago

    Hyderabad, Telangana, India AruGuru Tech Full time

    We are seeking an experienced GCP Data Engineering Lead to join AruGuruTech, a leading IT services and consultancy company with a global presence. As a senior professional in the field of data engineering, you will be responsible for designing, implementing, and maintaining large-scale data solutions on the Google Cloud Platform (GCP).With 6-12 years of...

  • GCP Data Engineer

    2 months ago

    Hyderabad, India AruGuru Tech Full time

    Role: Lead GCP Data EngineerExperience: 6 to 12 YearsJob Location : Hyderabad/BangalorePosition: PermanentWhat's the opportunity?In this role, you would be primarily responsible for developing, designing, and implementing high-quality dynamic websites and internal and external applications by understanding our customer's business requirements.Mandatory...

  • GCP Data Lead

    2 months ago

    Hyderabad, India AruGuru Tech Full time

    GCP Data LeadAruGuruTech is a leading IT services and consultancy company with a vision for limitless innovation and customer satisfaction. We are committed to delivering exceptional solutions that ignite our customers' imagination. With a global presence, we promote equality, trust, and empowerment.Job OverviewWe are seeking a dedicated and experienced GCP...

  • GDT WPB GCP Data Engineer

    5 months ago

    Hyderabad, India Virtusa Full time

    GDT WPB GCP Data Engineer - CREQ193213 Description GDT WPB GCP Data Engineer JD: Mandatory Skills Big Query or Snowflake AWS or Cloud SQL Knowledge Mandatory to have knowledge on any ETL Tool ONLY amongst (Data Fusion, Data Flow, Data Proc , Talend, Datastage, Informatica ISS Cloud version Mandatory SQL PL/SQL experience Strong Scheduler knowledge anyone...

  • Gcp data engineer

    3 weeks ago

    Hyderabad, India Technogen India Pvt. Ltd. Full time

    Role- Sr Engr Cslt-Data EngineeringLocation- Hyderabad/ Chennai/ BangaloreExp-6.5 to 9 yearsPrimary: SQL, GCP (Big Query, Composer, Data Flow), Spark/ Pyspark/ Scala/ Kafka/ HiveWhat you’ll be doing...We’re seeking a skilled Lead Senior Data Engineering Analyst to join our high-performing team and propel our telecom business forward. You’ll contribute...

  • Gcp data engineer

    2 days ago

    Hyderabad, India Tata Consultancy Services Full time

    Greetings from Tata Consultancy Services!! We are hiring for GCP Data Engineer !! Experience: 6-9.5 years Location: Chennai / Hyderabad Interview Date: 22-Nov-2024 (Friday) Mode of Interview: Virtual (Online) Required Technical Skillset: Strong SQL knowledge – able to translate complex scenarios into queries Strong Programming...