GCP Data Engineer

3 weeks ago

Bengaluru, Karnataka, India Kognivera It Solutions Private Limited Full time
Job Description

About the Role: We are seeking a Lead GCP Data Engineer who can demonstrate a broad range of technical skills across Google Cloud Platform (GCP) and third-party technologies. As a pivotal member of our team, you will be responsible for designing robust data architectures, developing efficient data pipelines, and optimizing data processing workflows on GCP. Your expertise will be instrumental in driving our data engineering initiatives forward, ensuring scalability, reliability, and performance.

Key Responsibilities

- Data Architecture Design: Design scalable and efficient data architectures on GCP that meet the organization's data processing and analysis requirements. Collaborate closely with data scientists, business analysts, and stakeholders to define effective data models and structures.
- Data Pipeline Development: Develop and implement data pipelines using GCP services such as Google Cloud Storage, Big Query, Dataflow, and Pub/Sub. Ensure data quality, reliability, and governance throughout the data lifecycle.
- Data Transformation and Integration: Utilize technologies like Apache Beam, Apache Spark, and Cloud Dataproc to transform and integrate data from diverse sources. Perform data cleansing, aggregation, enrichment, and normalization to support downstream applications and analytics.
- Performance Optimization: Optimize data processing workflows to enhance performance and efficiency. Monitor pipelines, identify bottlenecks, and implement optimizations such as improved data partitioning, sharding, and leveraging GCP's autoscaling capabilities.
- Continuous Improvement: Stay abreast of advancements in data engineering and cloud technologies. Explore and implement new GCP features and services to enhance data processing capabilities and drive innovation within the team.
- Research and Innovation: Conduct research on emerging data engineering technologies, tools, and best practices. Evaluate new methodologies to improve data engineering processes and bring innovative solutions to the organization.
- Task Automation: Automate data engineering tasks using scripting, workflows, or tools like Cloud Composer and Cloud Functions. Streamline data ingestion, transformation, monitoring, and other operational processes to improve efficiency and reduce manual effort.
-

Attributes & Competencies

- Education: BE/BTech, MTech, or MCA.
- Experience: Minimum 5+ years in development/migration projects, with at least 3 years focused on GCP. Experience in GCP-based Big Data deployments (batch/real-time) using Big Query, Big Table, Google Cloud Storage, Pub/Sub, Data Fusion, Dataflow, Dataproc, and Airflow.

Technical Skills

- Google Certified Professional Cloud Architect with experience automating and orchestrating workloads on GCP or other public clouds.
- Proficiency in at least one configuration management system (Chef, Puppet, Ansible, Salt, etc.).
- Strong programming skills in Python, Go, with expertise in Git and Git workflows.
- Demonstrated proficiency in CI/CD tools such as Jenkins, TeamCity, or Spinnaker.

  • GCP Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Crescendo Global Full time

    Job Description :We are looking for an experienced and skilled Data Engineer with a strong background in Google Cloud Platform (GCP) and a deep understanding of data engineering principles to join our dynamic team. The ideal candidate will have mastery in GCP data services and extensive expertise in ETL processes and data integration. You will play a...

  • GCP Data engineer

    10 hours ago

    Bengaluru, Karnataka, India Gravity Infosolutions Full time

    Job Title: GCP Data EngineerExperience: 6+ YearsLocation: RemoteType: ContractSkills:Strong in SQL and Python codingExperience in GCP, bigqueryJob Description: We are looking for a skilled GCP Data Engineer with strong experience in SQL and Python. The ideal candidate should have hands-on expertise in Google Cloud Platform (GCP) services, especially...

  • GCP Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Live Connections Full time

    Exp – 8 to 15 yearsLocation- Noida, Bang, Chennai & PuneGCP Data EngineerExperience in designing & implementing solution in mentioned areas:Strong Google Cloud Platform Data Components – BigQuery, BigTable, CloudSQL, Dataproc, Data Flow, Data Fusion, Etc Maintain architecture principles, guidelines and standardsData WarehousingProgramming Language:...

  • GCP Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Magnify360 Hi-Tech Solutions Pvt. Ltd Full time

    Experience : 4 to 8 years.Immediate joiners Skills :- Scala/Python (PySpark)- Big Data Technologies- Scripting (Shell/Python)- GCP Data Tech Stack- Airflow, CI/CD, Kafka- Distributed Databases- key responsibilities :- Design, develop, and maintain data pipelines using GCP services like Dataflow, Dataproc, and Pub/Sub.- Develop and implement data ingestion...

  • GCP Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Kiash Solutions LLP Full time

    Company OverviewOur organization is dedicated to leveraging the power of Google Cloud Platform (GCP) to innovate and drive business solutions. We are in search of talented GCP Engineers to join our dynamic team and help us build and manage robust cloud infrastructure.Job OverviewWe are hiring for various roles within our GCP engineering team. Each position...

  • GCP Data Infrastructure Engineer

    5 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Key ResponsibilitiesWe are seeking a skilled professional to design, code, and develop new features for large-scale data processing systems using GCP data tools. The ideal candidate will have a strong background in data engineering and experience working with GCP data tools.Main ResponsibilitiesData Engineering Expertise: Design and develop new features for...

  • GCP Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Synechron Full time

    As a Data Engineer, you'll focus on solving problems and creating value for business by building solutions that are reliable and scalable to work with the size and scope of the company. You will be tasked with creating a custom-built pipeline on GCP stack, and you will be part of teams that implement vendor sourced enterprise software, configuring that...

  • GCP Data Engineer

    6 days ago

    Bengaluru, Karnataka, India Synechron Full time

    As a Data Engineer, you'll focus on solving problems and creating value for business by building solutions that are reliable and scalable to work with the size and scope of the company. You will be tasked with creating a custom-built pipeline on GCP stack, and you will be part of teams that implement vendor sourced enterprise software, configuring that...

  • GCP Data Engineer

    4 days ago

    Bengaluru, Karnataka, India RapidBrains Full time

    Job Title: GCP Data EngineerExperience Required: 5 - 11 YearsLocation: Bangalore, Pune, Gurgaon, Chennai, Hyderabad, Bhopal, JaipurContract Duration: 6+ MonthsWork Time: 12:00 PM to 9:00 PMJob SummaryWe are looking for an experienced GCP Data Engineer with a strong background in Google Cloud Platform and data streaming technologies. The ideal candidate...

  • GCP Data Engineering Architect

    1 week ago

    Bengaluru, Karnataka, India Sutherland Full time

    Job DescriptionRole & responsibilitiesLead the GCP pillar within the Data Engineering CoE, establishing technical standards, best practices, and reusable accelerators for Google Cloud Platform data implementations. This role is critical for supporting high-value client engagements other GCP-focused opportunities. Key Responsibilities Develop architecture...