GCP Data Engineer with Big Query

3 days ago

Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 12,000 - ₹ 25,000 per year

Skill - GCP , BigQuery, Python

Location - Coimbatore

Experience - 5.5 to 12 Years

Job Summary

Data Engineer with a focus on Google Cloud Platform (GCP).

Hands-on experience with GCP services: Dataflow Pub/Sub Cloud Storage Cloud Composer.

Expertise in workflow orchestration using Apache Airflow.

Experience with cloud SQL or other relational databases (PostgreSQL preferred).

Solid understanding of data modeling ETL processes and streaming/batch data pipelines.

Familiarity with DevOps practices CI/CD and containerization (Docker Kubernetes).

Responsibilities

Key Responsibilities and Skills

  • Build Data Pipelines on GCP: They need hands-on experience with core GCP services used for data processing.
  • Dataflow: For building both batch and streaming data pipelines.
  • Pub/Sub: For real-time messaging and event ingestion.
  • Cloud Storage: For storing raw and processed data.
  • Cloud Composer: A managed service for Apache Airflow used for orchestrating complex data workflows.
  • Orchestrate Workflows: A key requirement is expertise in Apache Airflow. This tool is essential for scheduling monitoring and managing data workflows often represented as Directed Acyclic Graphs (DAGs).
  • Manage Databases: The role requires experience with Cloud SQL or other relational databases with a preference for PostgreSQL. This suggests they will be working with structured data storage.
  • Understand Data Concepts: A strong foundation in the theoretical side of data engineering is crucial.
  • Data Modeling: Designing the structure of databases to meet business needs.
  • ETL Processes: The classic Extract Transform Load pipeline for moving and processing data.
  • Streaming/Batch Data Pipelines: Understanding the differences and how to build both types of pipelines.
  • Apply DevOps Practices: Familiarity with modern software development practices is a plus.
  • CI/CD: Continuous Integration and Continuous Delivery for automating code deployment.
  • Containerization: Using technologies like Docker and Kubernetes to package and deploy applications consistently across different environments.

  • GCP+ Big query Developer_Anjali

    2 days ago

    Chennai, Tamil Nadu, India Coders Brain Technology Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role: GCP/BigQuery Developer Necessary to have: Relevant Industry Work Experience (5+ years for Dev, & Lead) Experience extracting data from a variety of sources, and a desire to expand those skills (Excellent knowledge in SQL and Spark is mandatory) Strong knowledge of Google BigQuery and architecting data pipelines from on-prem to GCP. Experience building...

  • Data Engineer

    4 days ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data EngineerLocation - Chennai, Gurugram (5days Office)Full timeShift timing 2pm to 11pmKey Responsibilities:· Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks.· Collaborate with cross-functional teams to design and implement data models and data integration solutions.· Create and maintain robust SQL...

  • GCP ETL Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Ford Motor Company Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Looking for a ETL Engineer with experience in Python, Airflow/Composer, Big Query and general GCP Cloud KnowledgeBachelor of Engineering or equivalentGCP certification (preferred)Looking for a ETL Engineer with experience in Python, Airflow/Composer, Big Query and general GCP Cloud Knowledge

  • GCP ETL Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Ford Motor Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    DescriptionLooking for a ETL Engineer with experience in Python, Airflow/Composer, Big Query and general GCP Cloud KnowledgeResponsibilitiesLooking for a ETL Engineer with experience in Python, Airflow/Composer, Big Query and general GCP Cloud KnowledgeQualificationsBachelor of Engineering or equivalentGCP certification (preferred)

  • GCP Data Engineer +Spanner

    3 days ago

    Chennai, Tamil Nadu, India Changeleaders Consulting Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Experience in GCP Services Data flow, big query and Pub/SubExperience in Google Cloud Storage, Cloud Run or Cloud Composer.

  • Gcp Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India Diensten Tech Full time ₹ 1,20,000 - ₹ 2,40,000 per year

    HiI hope this message finds you wellIm reaching out from Diensten Tech Ltd (DTL) with an exciting permanent job opportunity that perfectly matches your experience and skill set.Role Details:Experience: 5–10 YearsSkills: Big Query,, GCPWork Location: ChennaiBig Query,, Artificial Intelligence & Expert Systems, DataForm, Google Cloud Platform - Biq Query,...

  • GCP ETL Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Ford Global Career Site Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Looking for a ETL Engineer with experience in Python, Airflow/Composer, Big Query and general GCP Cloud Knowledge

  • Big Data Lead Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Synechron Full time ₹ 1,20,000 - ₹ 3,00,000 per year

    Big Data Lead Engineer - Hadoop - Spark & GCP (BigQuery/DataProc) with Java & Python ExpertiseJob SummarySynechron is seeking an experienced Big Data Lead Engineer to drive the design, development, and management of large-scale data solutionsThis full-time leadership role involves guiding a team of data engineers and architects to deliver scalable,...

  • Data Engineer

    5 days ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    HiRole - Azure Data engineerLocation - Chennai, GurugramWork from officeFull timeImmediate joiner Only (15days Notice period only)Candidates with Certification in Databricks is good to haveTechnical Skills:o Proficiency in SQL for complex queries, database design, and optimization.o Strong experience with PySpark for data transformation and processing.o...

  • Gcp Data Engineer

    6 days ago

    Chennai, Tamil Nadu, India Altimetrik Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Job Description:We are seeking a skilled and proactive GCP Data Engineer with strong experience in Python and SQL to build and manage scalable data pipelines on Google Cloud Platform (GCP). The ideal candidate will work closely with data analysts, architects, and business teams to enable data-driven decision-making.Key Responsibilities:Design and develop...