Kloud9 - Lead Data Engineer - Google Cloud Platform

5 days ago

Any LocationRemote, India Kloud9 Full time

Job Summary :

We are seeking a highly skilled Data Engineering Technical Lead with deep expertise in Google Cloud Platform (GCP), Apache Spark, and SAP S/4HANA (Data ingestion) to architect, design and implement scalable, high-performance data lake solutions. The ideal candidate will have extensive experience in building data ingestion pipelines, managing big data processing using Apache Spark, and integrating SAP S/4HANA with GCP cloud infrastructure.

Key Requirements :

- Over 7 years of professional experience in data engineering, specializing in implementing large-scale enterprise Data Engineering projects with the latest technologies.

- Over 5 years of hands-on experience in GCP technologies and over 3 years of lead experience.

- Design and implement end-to-end data architectures leveraging GCP services (e.g., Big Query, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer) for large-scale data ingestion and processing.

- Build and optimize large-scale data pipelines using Apache Spark on GCP (via Dataproc or other Spark services). Ensure high performance and scalability in Spark-based data processing workloads.

- Lead the integration of SAP S/4HANA data with GCP for real-time and batch data processing. Manage data extraction, transformation, and loading (ETL) processes from SAP S/4HANA into cloud storage and data lakes.

- Develop and manage scalable data ingestion pipelines for structured and unstructured data using tools like Cloud Dataflow, Cloud Pub/Sub, and Apache Spark.

- Provide architectural guidance for designing secure, scalable, and efficient data solutions on the Google Cloud Platform, integrating with on-premise/cloud systems like SAP S/4HANA.

- Implement both real-time streaming and batch processing pipelines using Apache Spark, Dataflow, and other GCP services to meet business requirements.

- Implement data governance, access controls, and security best practices to ensure the integrity, confidentiality, and compliance of data across systems.

- Collaborate with business stakeholders, data scientists, and engineering teams to define data requirements, ensuring the architecture aligns with business goals.

- Optimize Apache Spark jobs for performance, scalability, and cost-efficiency, ensuring that the architecture can handle growing data volumes.

- Provide technical leadership to the data engineering team, mentoring junior engineers in data architecture, Apache Spark development, and GCP best practices.

Technical Expertise :

- Expert-level programming proficiency in Python, Java, and Scala.

- Extensive hands-on experience with big data technologies, including Spark, Hadoop, Hive, Yarn, MapReduce, Pig, Kafka, and PySpark.

- Proficient in Google Cloud Platform services, such as BigQuery, Dataflow, Cloud Storage, Dataproc, and Cloud Composer Google Pub/Sub, and Google Cloud Functions.

- Expertise in Apache Spark for both batch and real-time processing, as well as proficiency in Apache Beam, Hadoop, or other big data frameworks.

- Experienced in using Cloud SQL, BigQuery, and Looker Studio (Google Data Studio) for cloud-based data solutions.

- Skilled in orchestration and deployment tools like Cloud Composer, Airflow, and Jenkins for continuous integration and deployment (CI/CD).

- Expertise in designing and developing integration solutions involving Hadoop/HDFS, real-time systems, data warehouses, and analytics solutions.

- Experience with DevOps practices, including version control (Git), CI/CD pipelines, and infrastructure-as-code (e.g., Terraform, Cloud Deployment Manager).

- Strong background in working with relational databases, NoSQL databases, and in-memory databases.

- Experience managing large datasets within Data Lake and Data Fabric architectures.

- Strong knowledge of security best practices, IAM, encryption mechanisms, and compliance frameworks (GDPR, HIPAA) within GCP environments.

- Experience in implementing data governance, data lineage, and data quality frameworks.

- In-depth knowledge of web technologies, application programming languages, OLTP/OLAP technologies, data strategy disciplines, relational databases, data warehouse development, and big data solutions.

- Led end-to-end processes for the design, development, deployment, and maintenance of data engineering projects.

- Excellent debugging and problem-solving skills.

- Retail and e-commerce domain knowledge is a plus.

- Positive attitude with strong analytical skills and the ability to guide teams effectively.

Preferred Qualifications :

- GCP Certifications : Such as Professional Data Engineer or Professional Cloud Architect.

- Apache Spark and Python certifications.

- Experience with Data visualization tools like Tableau, Power BI etc.

(ref:hirist.tech)

  • Kloud9 - Technical Architect - Data Engineering

    5 days ago

    Any Location/Remote, India Kloud9 Full time

    Job Summary :We are seeking a highly skilled Data Engineering Technical Architect with deep expertise in Google Cloud Platform (GCP), Apache Spark, and SAP S/4HANA (Data ingestion) to architect, design and implement scalable, high-performance data lake solutions. The ideal candidate will have extensive experience in building data ingestion pipelines,...

  • Senior Cloud Architect

    4 days ago

    Any Location/Remote, India Kloud9 Full time

    About the RoleWe are seeking a highly skilled Senior Cloud Architect to lead our cloud data engineering efforts at Kloud9. This is an exciting opportunity to join a dynamic team and work on cutting-edge projects.Job DescriptionIn this role, you will be responsible for designing, developing, and implementing scalable, high-performance data lake solutions on...

  • High-Performance Data Architect

    4 days ago

    Any Location/Remote, India Kloud9 Full time

    Job Summary:We are seeking a highly skilled Data Engineering expert to architect, design and implement scalable data lake solutions on Google Cloud Platform (GCP). The ideal candidate will have extensive experience in building data ingestion pipelines, managing big data processing using Apache Spark, and integrating SAP S/4HANA with GCP cloud...

  • Techolution - Associate Google Cloud Platform Data Engineer

    1 month ago

    Any Location, India Techolution Full time

    Job Description : We are looking for a Google Cloud Data Engineer who will help us build a highly scalable and reliable platform to match our exponential growth. As a Google Cloud Data Engineer, you will be responsible for building a solid back end infrastructure which will enable data delivery in near real-time using next-gen technologies.Title : Google...

  • Senior Google Cloud Platform Engineer

    5 days ago

    Remote, India Infometry Full time

    Experience : 8 - 10 yrsWe are seeking a highly skilled Senior Data Engineer with extensive experience in data engineering, particularly in building and maintaining data pipelines. This role requires strong expertise in GCP, Dataflow, Apache Beam SDK, and PySpark, along with a deep understanding of Google Cloud Platform (GCP) services. The ideal candidate...

  • Senior Big Data Engineer

    4 weeks ago

    Remote, India AEXONIC TECHNOLOGIES PRIVATE LIMITED Full time

    Position : Senior Big Data Engineer (GCP)Experience : 5+ YrsShift Time : EST Shift 5 : 30 PM to 2 : 30 AMRemote / Hybrid - Full TimeWe are seeking a highly skilled Senior Big Data Engineer with extensive experience in Big Data technologies, including Hadoop, Hive, PySpark, and expertise in Google Cloud Platform (GCP) and BigQuery. The successful candidate...

  • MarsDevs - DevOps Engineer - AWS/Google Cloud Platform

    1 month ago

    Any Location, India Marsdevs Full time

    Company Description : MarsDevs is a remote-first, offshore digital product development company. We specialize in building MVPs, Mobile and Web Applications, and SaaS Products for Early to Growth Startups, as well as vertical solutions for SMBs. Our commitment lies in providing high-quality code, no-vendor-locking guarantee, speed, and building technical...

  • Google Cloud Platform

    3 months ago

    Remote, India Nets-international Communication Full time

    **Requirements**: - Strong background in cloud computing. - Proficiency in Google Cloud Platform (GCP) services. - Experience with GCP IAM, Compute Engine, Kubernetes Engine, etc. - Knowledge of automation tools (Terraform, Deployment Manager). - Expertise in scripting languages (Python, Bash). - Ability to optimize costs and ensure high availability. -...

  • Cloud Data Engineer

    1 month ago

    Any Location, India Techolution Full time

    Job OverviewWe are seeking a seasoned Cloud Data Engineer to join our team at Techolution, where we harness expertise in novel technologies to deliver innovative solutions for our enterprise clients. This role will involve designing and developing highly scalable data pipelines on Google Cloud Platform (GCP).About the RoleThis is an excellent opportunity for...

  • Principal/Senior Data Engineer

    1 month ago

    Remote, India BlueByte Technologies Full time

    Location : Remote PositionDuration : 12 Months with possible extensionEmployment Type : C2CClient : Zendesk (They will be working in USA timings EST zone till 2:00 PM, as per India timings working hours will be 2:30 PM to 11:30 PM IST.)Location preference : PAN India Skills Needed :- Data transformation (DBT)- Data curation Job Description : A data curation...

  • Zimetrics Technologies

    1 month ago

    Pune/Remote, India Zimetrics Technologies Private Limited Full time

    We are looking for an experienced Data Engineer with expertise in Google Cloud Platform (GCP), particularly BigQuery, and strong skills in SQL, Data Warehousing (DWH), and ETL processes. The ideal candidate will also have experience in Dataflow and Python for building scalable data pipelines and ensuring efficient data transformation and integration.Key...

  • Data Engineering Lead

    3 weeks ago

    Remote, India AEXONIC TECHNOLOGIES PRIVATE LIMITED Full time

    Job OverviewAEXONIC TECHNOLOGIES PRIVATE LIMITED is seeking a seasoned Senior Big Data Engineer to spearhead the development of cloud-based data pipelines and solutions.Salary & BenefitsThe estimated annual salary for this position is approximately $250,000. We offer a comprehensive benefits package, including medical insurance, retirement plan, and generous...

  • Cloud Engineering Expert for Data Pipelines

    4 days ago

    Remote, India Infometry Full time

    Job Description: We are seeking a skilled Cloud Engineering Expert to design, develop, and maintain robust data pipelines using Java on Google Cloud Platform. The ideal candidate will have extensive experience in GCP services, including Dataflow, Apache Beam SDK, and PySpark.Key Responsibilities:Architect and implement data pipelines using Java and leverage...

  • Data Engineer

    1 month ago

    Any Location, India Blueridge Global Full time

    Are you passionate about building state-of-the-art data platforms and powering the next generation of compute and AI applications, we'd love to hear from you. This is an exciting opportunity to leverage your expertise in distributed computing frameworks to make a significant impact as we push the boundaries of supply chain planning and eventually...

  • Cloud Data Engineering Specialist

    3 weeks ago

    Remote, India Cloud Destinations Full time

    Job Title: Cloud Data Engineering SpecialistWe are seeking an experienced Cloud Data Engineering Specialist to design, develop, and maintain scalable data pipelines and data warehouses at Cloud Destinations.About the Role:This is a senior-level position that requires strong proficiency in SQL and Python, as well as experience with ETL processes, data quality...

  • Senior Cloud Infrastructure Engineer

    5 days ago

    Any Location, India Arting Digital Full time

    Job Title : Senior Cloud Infrastructure EngineerExperience : 5+ years Location : Remote Primary Skills : AWS, Ansible, Terraform, Jenkins, Relational and Non-Relational Databases. Roles and Responsibilities : We are seeking a highly skilled and motivated Cloud Infrastructure Engineer to join our growing team. In this role, you will be responsible for...

  • Cloud Data Engineering and Administration Lead

    4 days ago

    Remote, India Georgia IT, Inc. Full time

    About the JobWe are seeking an experienced Cloud Data Engineering and Administration Lead to oversee the administration of Informatica AXON, EDC, and IDQ implementations on the cloud. The ideal candidate will have expertise in Azure cloud platform administration and data engineering, as well as experience with business intelligence tools like PowerBI and...

  • Data Engineer Leader for Cloud Data Pipelines

    3 weeks ago

    Remote, India Cloud Destinations Full time

    Company Overview: At Cloud Destinations, we are a dynamic team seeking a highly skilled Senior Data Engineer to play a critical role in designing, developing, and maintaining robust data pipelines and data warehouses. Our company is built on innovative cloud solutions, driving business decisions through data-driven insights.

  • Looking for Google Cloud Trainer

    7 months ago

    Remote, India Intellecto Global Services Full time

    We are seeking a part-time Google Cloud Trainer to join our team. **Responsibilities**: - Develop Training Curriculum: Create and update training materials, presentations, and hands-on exercises to cover a wide range of topics related to Google Cloud Platform services, including compute, storage, networking, databases, machine learning, and more. - Deliver...

  • Senior Developer- Google Cloud Platform

    1 month ago

    Remote, India SS Consultancy Services Full time

    Experience in developing, deploying and maintaining cloud-based application in GCP. The role involves leveraging various GCP services and tools to build scalable, reliable applications. The candidate should have strong background in Cloud computing and deep understanding of GCP services as Cloud Storage, Big Query, Cloud Run, Cloud Scheduler, Artifact...