Cloud Architect

2 weeks ago

bangalore, India Cygnus Professionals Inc. Full time

Position: GCP Architect (Apache Spark)

Location: Bangalore

Duration: Full Time (4 Days WFO)

Job Description for Data Engineering Architect (GCP & Apache Spark)

Job Summary:

We are seeking a highly skilled Data Engineering Architect with deep expertise in Google Cloud Platform (GCP), Apache Spark, to architect, design and implement scalable, high-performance data lake solutions. The ideal candidate will have extensive experience in building data ingestion pipelines, managing big data processing using Apache Spark,

Key Requirements:

- Over 12 years of professional experience in data engineering, specializing in implementing large-scale enterprise Data Engineering projects with the latest technologies.
- Over 5 years of hands-on experience in GCP technologies and over 3 years of architect experience.
- Design and implement end-to-end data architectures leveraging GCP services (e.g., Big Query, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer) for large-scale data ingestion and processing.
- Build and optimize large-scale data pipelines using Apache Spark on GCP (via Dataproc or other Spark services). Ensure high performance and scalability in Spark-based data processing workloads.
- Lead the integration of SAP S/4HANA data with GCP for real-time and batch data processing. Manage data extraction, transformation, and loading (ETL) processes from SAP S/4HANA into cloud storage and data lakes.
- Develop and manage scalable data ingestion pipelines for structured and unstructured data using tools like Cloud Dataflow, Cloud Pub/Sub, and Apache Spark.
- Provide architectural guidance for designing secure, scalable, and efficient data solutions on the Google Cloud Platform, integrating with on-premise/cloud systems like SAP S/4HANA.
- Implement both real-time streaming and batch processing pipelines using Apache Spark, Dataflow, and other GCP services to meet business requirements.
- Implement data governance, access controls, and security best practices to ensure the integrity, confidentiality, and compliance of data across systems.
- Collaborate with business stakeholders, data scientists, and engineering teams to define data requirements, ensuring the architecture aligns with business goals.
- Optimize Apache Spark jobs for performance, scalability, and cost-efficiency, ensuring that the architecture can handle growing data volumes.
- Provide technical leadership to the data engineering team, mentoring junior engineers in data architecture, Apache Spark development, and GCP best practices.

Technical Expertise:

- Expert-level programming proficiency in Python, Java, and Scala.
- Extensive hands-on experience with big data technologies, including Spark, Hadoop, Hive, Yarn, MapReduce, Pig, Kafka, and PySpark.
- Proficient in Google Cloud Platform services, such as BigQuery, Dataflow, Cloud Storage, Dataproc, and Cloud Composer Google Pub/Sub, and Google Cloud Functions.
- Expertise in Apache Spark for both batch and real-time processing, as well as proficiency in Apache Beam, Hadoop, or other big data frameworks.
- Experienced in using Cloud SQL, BigQuery, and Looker Studio (Google Data Studio) for cloud-based data solutions.
- Skilled in orchestration and deployment tools like Cloud Composer, Airflow, and Jenkins for continuous integration and deployment (CI/CD).
- Expertise in designing and developing integration solutions involving Hadoop/HDFS, real-time systems, data warehouses, and analytics solutions.
- Experience with DevOps practices, including version control (Git), CI/CD pipelines, and infrastructure-as-code (e.g., Terraform, Cloud Deployment Manager).
- Strong background in working with relational databases, NoSQL databases, and in-memory databases.
- Experience managing large datasets within Data Lake and Data Fabric architectures.
- Strong knowledge of security best practices, IAM, encryption mechanisms, and compliance frameworks (GDPR, HIPAA) within GCP environments.
- Experience in implementing data governance, data lineage, and data quality frameworks.
- In-depth knowledge of web technologies, application programming languages, OLTP/OLAP technologies, data strategy disciplines, relational databases, data warehouse development, and big data solutions.
- Led end-to-end processes for the design, development, deployment, and maintenance of data engineering projects.
- Excellent debugging and problem-solving skills.
- Retail and e-commerce domain knowledge is a plus.
- Positive attitude with strong analytical skills and the ability to guide teams effectively.

Preferred Qualifications:

- GCP Certifications: Such as Professional Data Engineer or Professional Cloud Architect.
- Apache Spark and Python certifications.

Experience with Data visualization tools like Tableau, Power BI etc.

  • Cloud architect

    23 hours ago

    Bangalore, India Tata Consultancy Services Full time

    Job Purpose and primary objectives: Cloud Architect Key responsibilities: Assist in implementing and supporting cloud implementations on Amazon AWS. Supervisory / Managerial responsibilities (please specify if the position will have persons reporting to it): Other duties include architecting and implementing cloud solutions for different...

  • Cloud Architect

    4 weeks ago

    Bangalore, India CAST Software India Pvt Ltd Full time

    **Cloud Architect Role**We are seeking an experienced Cloud Architect to lead our digital transformation. The ideal candidate will be responsible for designing, implementing, and managing cloud solutions that meet our organization's needs. You will play a crucial role in our cloud strategy, working closely with various teams to ensure optimal cloud...

  • Cloud Architect

    1 week ago

    bangalore, India Tata Consultancy Services Full time

    Job Purpose and primary objectives:Cloud ArchitectKey responsibilities:Assist in implementing and supporting cloud implementations on Amazon AWS.Supervisory / Managerial responsibilities (please specify if the position will have persons reporting to it):Other duties include architecting and implementing cloud solutions for different applications,...

  • Cloud Architect

    1 week ago

    bangalore, India Tata Consultancy Services Full time

    Job Purpose and primary objectives:Cloud ArchitectKey responsibilities:Assist in implementing and supporting cloud implementations on Amazon AWS.Supervisory / Managerial responsibilities (please specify if the position will have persons reporting to it):Other duties include architecting and implementing cloud solutions for different applications,...

  • Cloud Architect

    3 weeks ago

    bangalore, India HCLTech Full time

    • Expertise in Cloud Migration and demonstrable experience of having successfully migrated a customer to a major public cloud platform (AWS/Azure) • Expertise in using public cloud platforms (AWS/Azure) to build solutions solving business problems and making innovative use of SaaS and PaaS offerings. • Understands whole solution architecture concepts...

  • Cloud Architect

    3 weeks ago

    bangalore, India HCLTech Full time

    • Expertise in Cloud Migration and demonstrable experience of having successfully migrated a customer to a major public cloud platform (AWS/Azure) • Expertise in using public cloud platforms (AWS/Azure) to build solutions solving business problems and making innovative use of SaaS and PaaS offerings. • Understands whole solution architecture...

  • Cloud Architect

    4 weeks ago

    Bangalore, India The Metromax Group Full time

    Key Responsibilities:Cloud Architecture:- Design and implement scalable, secure, and high-performing cloud architectures on AWS.- Collaborate with development and data teams to architect cloud-native applications and microservices.Cloud Migration:- Lead cloud migration strategies for moving on-premises applications and data to AWS.- Develop and enforce best...

  • Cloud Architect

    1 month ago

    Bangalore, India Akal Information Systems Ltd Full time

    Job Title: Open Source PaaSWe are seeking a highly skilled Cloud Architect to join our team at Akal Information Systems Ltd.Key Responsibilities:Design and implement open source private cloud architecture for CIDRImplement, design, and automate PaaS rollouts in the UIDAI's private cloud architectureCoordinate with UIDAI appointed vendors in supporting...

  • Cloud Architect

    4 weeks ago

    Bangalore, India PeopleXM Full time

    Job Summary:We are seeking a highly skilled Cloud Architect to join our team at PeopleXM. In this role, you will be responsible for designing and implementing scalable cloud-based systems that meet the needs of our customers.Key Responsibilities:Design and implement cloud-based infrastructure using AWS, Azure, and GCPDevelop and maintain CI/CD pipelines...

  • Cloud Infrastructure Architect

    4 weeks ago

    Bangalore, India Selsoftinc Full time

    Job Title: Cloud Infrastructure ArchitectAbout the Role:We are seeking a skilled Cloud Infrastructure Architect to lead the design, implementation, and management of Google Cloud Platform (GCP) infrastructure for our company. As a Cloud Infrastructure Architect, you will collaborate with cross-functional teams to ensure that the cloud environment is...

  • Cloud Solutions Architect

    1 month ago

    Bangalore, India Softpath Technologies LLC Full time

    Job Title: Cloud Solutions ArchitectJob Summary:We are seeking a highly skilled Cloud Solutions Architect to join our team at SoftPath Technologies LLC. As a Cloud Solutions Architect, you will be responsible for designing and implementing complex cloud solutions for our clients.Key Responsibilities:Design and implement cloud architecture solutions that are...

  • Cloud Architect

    1 month ago

    Bangalore, India CAST Software India Pvt Ltd Full time

    Cloud Architect Job DescriptionWe are seeking an experienced Cloud Architect to lead our cloud strategy and design scalable, secure, and resilient cloud architectures across various platforms. The ideal candidate will be responsible for developing and implementing a comprehensive cloud strategy aligned with business objectives, assessing current cloud...

  • Cloud Architect Professional

    4 weeks ago

    Bangalore, India Growel Softech Pvt. Ltd. Full time

    Job Title: Cloud Architect ProfessionalJob Summary: We are seeking a highly skilled Cloud Architect Professional to join our team at Growel Softech Pvt. Ltd. As a Cloud Architect Professional, you will be responsible for designing, deploying, and managing scalable solutions across AWS, Azure, and GCP platforms.About the Role:Design and Architect Cloud...

  • Cloud Solutions Architect

    4 weeks ago

    Bangalore, India Growel Softech Pvt. Ltd. Full time

    Job Title: Cloud Solutions ArchitectAbout the Role:We are seeking an experienced Cloud Solutions Architect to join our team at Growel Softech Pvt. Ltd. The successful candidate will be responsible for designing, deploying, and managing scalable solutions across AWS, Azure, and GCP platforms. The ideal candidate will have a deep understanding of cloud...

  • Cloud Solutions Architect

    1 month ago

    Bangalore, India Hrizen techno services Full time

    Job Title: Cloud Solutions ArchitectAt Hrizen techno services, we are seeking a highly skilled Cloud Solutions Architect to join our team. As a Cloud Solutions Architect, you will be responsible for designing and implementing cloud-based solutions on Microsoft Azure, prioritizing security, scalability, and performance.Key Responsibilities:Design and...

  • Cloud Solutions Architect

    1 month ago

    Bangalore, India Growel Softech Pvt. Ltd. Full time

    Job Title: Cloud Solutions ArchitectJob Summary:We are seeking a highly skilled Cloud Solutions Architect to join our team at Growel Softech Pvt. Ltd. The ideal candidate will have extensive experience in designing, deploying, and managing scalable solutions across AWS, Azure, and GCP platforms.Key Responsibilities:Design and architect cloud solutions across...

  • AWS Cloud Architect

    1 week ago

    bangalore, India Accolite Full time

    AWS Cloud Architect Responsibilities Support a scalable, secure, highly-available cloud in a complex environment experience in migrating Java microservices to Kubernetes and experience with Kafka Participate in design and automation of cloud solutions Devise enterprise strategies around containerization, serverless and API management Define and document...

  • Cloud and Infrastructure Architect

    4 weeks ago

    Bangalore, India Akal Information Systems Ltd Full time

    Job OverviewA skilled Cloud & EMS Architect is needed to implement and manage our open-source private cloud architecture. The ideal candidate will have extensive experience in IT architecture design and implementation, with a focus on private cloud solutions.Key ResponsibilitiesImplement private cloud architecture for CIDR, migrating existing architecture to...

  • Cloud Solutions Architect

    1 month ago

    Bangalore, India Squareroot Consulting Pvt Ltd. Full time

    Job Title: Cloud Solutions ArchitectAbout the Role:At Squareroot Consulting Pvt Ltd., we are seeking a highly skilled Cloud Solutions Architect to design and deliver end-to-end cloud platforms and solutions using Docker/Containers/Kubernetes for highly scalable deployment.Key Responsibilities:- Design and develop cloud-based products/solutions using cloud...

  • Cloud Architect for Private Cloud Implementations

    4 weeks ago

    Bangalore, India Akal Information Systems Ltd Full time

    CIDR Private Cloud ArchitectureWe are seeking a skilled Cloud Architect to design and implement a secure and efficient private cloud architecture for CIDR, utilizing open-source tools and technologies. The ideal candidate will have extensive experience in IT architecture designing and implementation, with a focus on private cloud solutions.Key...