Data Engineer-GCP

1 month ago

Mumbai Metropolitan Region, India Pixeldust Technologies Full time

Job Description:

We are seeking a skilled and experienced GCP (Google Cloud Platform) Data Engineering Specialist to join our team. The ideal candidate should have 3+ years of relevant experience with expertise in BigQuery, Dataflow, Spark, and Pub/Sub. As a Data Engineering Specialist, you will be responsible for designing, developing, and maintaining data pipelines, data integration solutions, and ETL processes on the GCP platform to support our data-driven applications and analytics initiatives.


Responsibilities:

  • Design, develop, and maintain data pipelines and ETL processes using BigQuery, Dataflow, Spark, and Pub/Sub on the GCP platform.
  • Collaborate with data scientists, data analysts, and other stakeholders to gather requirements and define data engineering solutions that meet business needs.
  • Optimize and troubleshoot data pipelines for performance, reliability, and scalability.
  • Ensure data quality and integrity by implementing data validation, cleansing, and transformation processes.
  • Monitor and manage data processing workflows, troubleshoot, and resolve data processing issues.
  • Develop and maintain documentation for data engineering processes, workflows, and best practices.
  • Stay updated with the latest advancements in GCP data engineering technologies and recommend and implement improvements to existing data engineering processes.
  • Strong GCP, Hadoop, Hive, Spark, Unix Shell scripting, and Python experience.
  • Exceptional troubleshooting skills in the following: GCP, Hadoop, Hive, Spark, Unix Shell scripting.
  • Construct and maintain ELT/ETL job processes sourcing from disparate systems throughout the enterprise and load into an enterprise data lake.
  • Effectively acquire and translate user requirements into technical specifications to develop automated data pipelines to satisfy business demand


Skills and Qualifications

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
  • 3 years of relevant experience in data engineering with a strong focus on GCP technologies, including BigQuery, Dataflow, Spark, and Pub/Sub.
  • Hands-on experience in designing, developing, and optimizing data pipelines, ETL processes, and data integration solutions on GCP.
  • Proficiency in programming languages such as Python, or Scala, with experience in writing efficient and scalable code for data processing.
  • Strong understanding of data modeling, data warehousing, and data integration concepts.
  • Knowledge of data lake and data warehouse architectures, data governance, and data security best practices on GCP.
  • Excellent problem-solving skills with the ability to troubleshoot and resolve complex data engineering issues.
  • Strong communication and collaboration skills to work effectively with cross-functional teams and stakeholders.
  • GCP certifications such as Google Cloud Certified - Data Engineer, Google Cloud Certified - Professional Data Engineer, or related certifications are a plus.

  • GCP Data Engineer

    2 weeks ago

    mumbai, India Skyleaf Consultants LLP Full time

    Position: GCP Data EngineerAbout Company: a well-known conglomerateLocation: Mumbai or Pune, IndiaExperience: 3-5 yearsQualifications:Bachelor’s degree in information technology, data engineering, or related field.3-4 years experience with Google BigQuery; SAP BI to GCP BigQuery migration preferred.Primary Skills:Strong understanding of BigQuery...

  • GCP Data Engineer

    4 weeks ago

    Mumbai, India Quantiphi Full time

    Designation: Data EngineerExperience Level: 4-15 YearsLocation: Mumbai/Bangalore/TrivandrumRole & Responsibilities:In-depth knowledge and experience of GCP Data services : Bigquery / Dataproc / Composer / Pubsub / Dataflow / GCS/ BigTableMust have proficient experience in GCP Databases : Bigtable / Spanner / CloudSQL / AlloyDBSolid understanding of...

  • GCP Data Engineer

    3 weeks ago

    mumbai, India Quantiphi Full time

    Designation: Data Engineer Experience Level: 4-15 Years Location: Mumbai/Bangalore/Trivandrum Role & Responsibilities: In-depth knowledge and experience of GCP Data services : Bigquery / Dataproc / Composer / Pubsub / Dataflow / GCS/ BigTable Must have proficient experience in GCP Databases : Bigtable / Spanner / CloudSQL / AlloyDB Solid understanding of...

  • GCP Data Engineer

    4 weeks ago

    Mumbai, India Quantiphi Full time

    Designation: Data EngineerExperience Level: 4-15 YearsLocation: Mumbai/Bangalore/TrivandrumRole & Responsibilities:In-depth knowledge and experience of GCP Data services : Bigquery / Dataproc / Composer / Pubsub / Dataflow / GCS/ BigTableMust have proficient experience in GCP Databases : Bigtable / Spanner / CloudSQL / AlloyDBSolid understanding of...

  • GCP Data Engineer

    3 weeks ago

    Mumbai, India Quantiphi Full time

    Designation:Data EngineerExperience Level:4-15 YearsLocation:Mumbai/Bangalore/TrivandrumRole & Responsibilities:In-depth knowledge and experience of GCP Data services : Bigquery / Dataproc / Composer / Pubsub / Dataflow / GCS/ BigTableMust have proficient experience in GCP Databases : Bigtable / Spanner / CloudSQL / AlloyDBSolid understanding of relational...

  • GCP Data Engineer

    2 weeks ago

    Mumbai, India Skyleaf Consultants LLP Full time

    Position: GCP Data EngineerAbout Company: a well-known conglomerateLocation: Mumbai or Pune, IndiaExperience: 3-5 yearsQualifications:Bachelor’s degree in information technology, data engineering, or related field.3-4 years experience with Google BigQuery; SAP BI to GCP BigQuery migration preferred.Primary Skills:Strong understanding of BigQuery...

  • GCP Data Engineer

    2 weeks ago

    Mumbai, India Skyleaf Consultants LLP Full time

    Position: GCP Data EngineerAbout Company: a well-known conglomerateLocation: Mumbai or Pune, IndiaExperience: 3-5 yearsQualifications:Bachelor’s degree in information technology, data engineering, or related field.3-4 years experience with Google BigQuery; SAP BI to GCP BigQuery migration preferred.Primary Skills:Strong understanding of BigQuery...

  • GCP Data Engineer

    2 weeks ago

    mumbai, India Skyleaf Consultants LLP Full time

    Position: GCP Data Engineer About Company: a well-known conglomerate Location: Mumbai or Pune, India Experience: 3-5 years Qualifications: Bachelor’s degree in information technology, data engineering, or related field. 3-4 years experience with Google BigQuery; SAP BI to GCP BigQuery migration preferred. Primary Skills: Strong understanding of...

  • GCP Data Engineer

    4 weeks ago

    mumbai, India CIEL HR Services Full time

    Hi, Total years of experience: 5+ Description: Job Description: GCP Data Engineer Engineers : Prepare DAG Prepare Airflow based ETL Jobs Document architecture & technical design Analyze and implement Securities Services use case Implement required Securities for product data acquisition

  • GCP Data Engineer

    4 weeks ago

    Bangalore Metropolitan Area, India Tredence Inc. Full time

    Have Implemented and Architected solutions on Google Cloud Platform using the components of GCPExperience with Apache Beam/Google Dataflow/Apache Spark in creating end to end data pipelines.Experience in some of the following: Python, Hadoop, Spark, SQL, Big Query, Big Table Cloud Storage, Datastore, Spanner, Cloud SQL, Machine Learning.Experience...

  • GCP Data Engineer

    2 weeks ago

    Mumbai, India Persistent Systems Full time

    About Position:As a GCP Data Engineer with strong experience in Python, Google Cloud and BigqueryRole: GCP Data EngineerLocation: MumbaiExperience: 4-8 yearsJob Type: Full Time EmploymentWhat You'll Do:Establish data model design based on business requirements. Should be hands on.Develop end to end streaming and batch data analytics pipelines. From data...

  • GCP Data Engineer

    2 weeks ago

    Mumbai, India Persistent Systems Full time

    About Position:As a GCP Data Engineer with strong experience in Python, Google Cloud and BigqueryRole: GCP Data EngineerLocation: MumbaiExperience: 4-8 yearsJob Type: Full Time EmploymentWhat You'll Do:Establish data model design based on business requirements. Should be hands on.Develop end to end streaming and batch data analytics pipelines. From data...

  • GCP Data Engineer

    5 days ago

    Mumbai, India Persistent Systems Full time

    About Position:As a GCP Data Engineer with strong experience in Python, Google Cloud and Bigquery- Role: GCP Data Engineer- Location: Mumbai- Experience: 4-8 years- Job Type: Full Time EmploymentWhat You'll Do:- Establish data model design based on business requirements. Should be hands on.- Develop end to end streaming and batch data analytics pipelines....

  • DevOps Engineer

    2 weeks ago

    Mumbai Metropolitan Region, India Sia Partners Full time

    Sia Partners is a next-generation management consulting firm. We offer a unique blend of AI and design capabilities, augmenting traditional consulting to deliver superior value to our clients. With expertise in more than 30 sectors and services, we optimize client projects worldwide. Through our Consulting for Good approach, we strive for next-level impact...

  • Senior Data Engineer

    2 weeks ago

    Mumbai Metropolitan Region, India Ventura Full time

    Ventura is an omnichannel trading and investment platform with a network of branches, sub-brokers and Digital Channels. Founded in 1994, the company is entering the next phase of growth by pivoting to a digital-first approach and strengthening its direct-to-consumer franchise. The company has carved out a separate fintech vertical tasked with digital...

  • DevOps Engineer

    2 days ago

    Mumbai Metropolitan Region, India EZINFORMATICS SOLUTIONS PVT LTD Full time

    Company DescriptionEZINFORMATICS SOLUTIONS PVT LTD is a team of designers, developers, authors, thinkers, and visionaries with vast industrial experience in various IT services. They specialize in three different spheres: Cyber Security, Information Technology, and Consulting Services. They prioritize security, customer data unification, and industry...

  • Senior Data Engineer

    5 days ago

    Mumbai, Maharashtra, India Cornerstone OnDemand Full time

    We are seeking a Senior DataEngineer with expertise in Looker and Google Cloud Platform (GCP) to join ourinnovative team. With at least 10 years of overall experience in dataengineering, including a minimum of 5 years working with Looker and GCP, theideal candidate will lead the design, development, and maintenance of LookMLmodels and dashboards. This role...

  • Senior Data Engineer

    5 days ago

    Mumbai, India Cornerstone OnDemand Full time

    We are seeking a Senior DataEngineer with expertise in Looker and Google Cloud Platform (GCP) to join ourinnovative team. With at least 10 years of overall experience in dataengineering, including a minimum of 5 years working with Looker and GCP, theideal candidate will lead the design, development, and maintenance of LookMLmodels and dashboards. This role...

  • Technical Architect

    3 weeks ago

    Mumbai, India NEC Full time

    Job DescriptionRole: Technical ArchitectExperience: 10+ yearsLocation: UK - Work From OfficeSkills:7 years of hands-on experience in architecting and building IT solutions in GCP (AWS is a plus), 10 years hand-on experience in either infrastructure engineering, software engineering, or data engineering7 years of hands-on experience in architecting and...

  • Data Engineer

    3 weeks ago

    Mumbai, India Antal International Full time

    Job DescriptionData Engineer Job Responsibilities:· Develops and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity.· Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering...