GCP Data Pipeline Architect

1 week ago

Industrial Area, India Compunnel Inc. Full time

GCP Data EngineerWork Location: Noida/Chandigarh/RemoteWe are seeking an experienced GCP Data Engineer to join our team. The ideal candidate will work closely with business, IT, analysts, and data science groups to collect requirements and build high-performance data pipelines. This role involves designing scalable data solutions, ensuring data quality, and supporting real-time data ingestion using Google Cloud Platform services. Key Responsibilities: - Work closely with various business, IT, analyst, and data science groups to collect business requirements. - Design, develop, deploy, and support high-performance data pipelines (inbound and outbound). - Model data platform by applying business logic and building objects in the semantic layer. - Optimize data pipelines for performance, scalability, and reliability. - Implement CI/CD pipelines to ensure continuous deployment and delivery of data products. - Ensure quality of critical data elements and collaborate with business/system owners on remediation plans. - Document the design and support strategy of data pipelines. - Capture, store, and socialize data lineage and operational metadata. - Troubleshoot and resolve data engineering issues. - Develop REST APIs to expose data to other teams. - Mentor and guide junior data engineers.Required Qualifications: - 2+ years of experience in Google Cloud with services like BigQuery, Composer, GCS, DataStream, and Dataflows. - 6+ years of experience in data engineering solutions such as data platforms, ingestion, data management, or publication/analytics.Knowledge, Skills & Abilities: - Expert knowledge of SQL and Python programming. - Experience working with Airflow as a workflow management tool and building operators to connect, extract, and ingest data. - Experience in tuning queries for performance and scalability. - Experience in real-time data ingestion using GCP PubSub, Kafka, Spark, or similar. - Excellent organizational, prioritization, and analytical abilities. - Proven experience with incremental execution through successful launches. - Excellent problem-solving and critical-thinking skills; ability to comprehend complex data issues across multiple business environments. - Experience working in an Agile environment.Please share the following details along with the most updated resume to geeta.negi@compunnel.com if you are interested in the opportunity: - Total Experience - Relevant experience - Current CTC - Expected CTC - Notice Period (Last working day if you are serving the notice period) - Current Location - SKILL 1 RATING OUT OF 5 - SKILL 2 RATING OUT OF 5 - SKILL 3 RATING OUT OF 5(Mention the skill)

  • Associate Director

    5 days ago

    Industrial Area, India XCUTIVES INC. Full time

    EXPERIENCE - 10-15 years Job SummaryWe are seeking a highly skilled and experienced professional to lead and support our data warehousing and data center architecture initiatives. The ideal candidate will have deep expertise in Data Warehousing, Data Lakes, Data Integration, and Data Governance, with hands-on experience in ETL tools and cloud platforms such...

  • Data Pipeline Engineer

    3 days ago

    Industrial Area, India Tata Consultancy Services Full time

    Greetings From TCS!!!"Opportunities don't happen, you create them"Job Title: Azure DatabricksExp Range: 4-10 YearsJob Location: Noida or Bangalore or Chennai or Hyderabad or KolkataJob Description:Required skill: Azure Databricks, PySpark and Python.Must have: 1.Azure Data Bricks (Python)2.Pyspark3. ADLS4. complex problem solving skillGood to Have: 1.Azure...

  • Lead Data Architect

    2 weeks ago

    Industrial Area, India Coforge Full time

    Role: Senior Technical Lead – Associate Architect (Microsoft Fabric)Experience : 8-12 yearsLocation: Greater Noida, Work from office 5 daysKey Responsibilities:Architect & Design Solutions: Lead the end-to-end architecture of data platforms using Microsoft Fabric, ensuring alignment with business goals and technical standards.Data Engineering Leadership:...

  • GCP Data Engineer

    2 days ago

    Greater Hyderabad Area, India Tata Consultancy Services Full time

    Required Technical Skill Set - GCP, Python, SQL, Big Query, ETL Data Pipeline Location of Requirement - Chennai, Kochi, Bangalore,HYD,Pune Must-Have Competencies , Python, Pandas, NumPy SQL, Data Warehouse concept GCP services (Big Query, Cloud Run, Pub/Sub, Cloud Storage, Spanner, Cloud Composer, Dataflow, Cloud Functions) Docker, Kubernetes, GitHub Good...

  • GCP Data Engineer

    4 days ago

    Greater Hyderabad Area, India Tata Consultancy Services Full time

    Required Technical Skill Set - GCP, Python, SQL, Big Query, ETL Data Pipeline Location of Requirement - Chennai, Kochi, Bangalore,HYD,Pune Must-Have Competencies , Python, Pandas, NumPy SQL, Data Warehouse concept GCP services (Big Query, Cloud Run, Pub/Sub, Cloud Storage, Spanner, Cloud Composer, Dataflow, Cloud Functions) Docker, Kubernetes, GitHub Good...

  • GCP Data Engineer

    3 days ago

    Greater Hyderabad Area, India Tata Consultancy Services Full time

    Required Technical Skill Set - GCP, Python, SQL, Big Query, ETL Data PipelineLocation of Requirement - Chennai, Kochi, Bangalore,HYD,Pune Must-Have Competencies ,Python, Pandas, NumPySQL, Data Warehouse conceptGCP services (Big Query, Cloud Run, Pub/Sub, Cloud Storage, Spanner, Cloud Composer, Dataflow, Cloud Functions)Docker, Kubernetes, GitHubGood...

  • Data Engineering Lead

    3 days ago

    Industrial Area, India Varahe Analytics Private Limited Full time

    Role : Technical Lead - Database Platform Location: Noida & BengaluruAbout the Company & Role : We are one of India’s premier integrated political consulting firms specializing in building data-driven 360-degree election campaigns. We help our clients with strategic advice and implementation which brings together data-backed insights and in-depth ground...

  • GCP Data Engineer

    2 days ago

    Greater Hyderabad Area, IN Tata Consultancy Services Full time

    Required Technical Skill Set - GCP, Python, SQL, Big Query, ETL Data PipelineLocation of Requirement - Chennai, Kochi, Bangalore,HYD,Pune Must-Have Competencies ,Python, Pandas, NumPySQL, Data Warehouse conceptGCP services (Big Query, Cloud Run, Pub/Sub, Cloud Storage, Spanner, Cloud Composer, Dataflow, Cloud Functions)Docker, Kubernetes, GitHubGood...

  • Associate Director, Data Architecture

    5 days ago

    Industrial Area, India XCUTIVES INC. Full time

    Role - Associate Director – D&A Presales Architect Location - Noida, IndiaExperience - 10-15 years Key Responsibilities - Architect and design scalable Data Warehousing and Data Lake solutions - Lead presales engagements, including RFP/RFI/RFQ lifecycle management - Create and present compelling proposals and solution designs to clients - Collaborate with...

  • Data Engineer

    3 days ago

    Industrial Area, India Sonata Software Full time

    Job Description SummaryWe are seeking a highly skilled Data Engineer to join our growing team. The ideal candidate has strong experience building and maintaining robust, scalable, cloud-native data pipelines and Datawarehouse using tools such as Snowflake, Five Tran, Airflow, and DBT. You will work closely with data analysts, scientists, and engineering...