Cloud Architect

4 weeks ago

Bengaluru, India Cygnus Professionals Inc. Full time
Position: GCP Architect (Apache Spark)Location: BangaloreDuration: Full Time (4 Days WFO)

Job Description for Data Engineering Architect (GCP & Apache Spark)

Job Summary:

We are seeking a highly skilled

Data Engineering Architect

with deep expertise in

Google Cloud Platform (GCP) ,

Apache Spark , to architect, design and implement scalable, high-performance data lake solutions. The ideal candidate will have extensive experience in building data ingestion pipelines, managing big data processing using

Apache Spark ,

Key Requirements:

Over 12 years of professional experience in data engineering, specializing in implementing large-scale enterprise Data Engineering projects with the latest technologies.Over 5 years of hands-on experience in

GCP

technologies and over 3 years of architect experience.Design and implement end-to-end data architectures leveraging

GCP services

(e.g.,

Big Query, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer ) for large-scale data ingestion and processing.Build and optimize large-scale data pipelines using

Apache Spark

on GCP (via

Dataproc

or other Spark services). Ensure high performance and scalability in Spark-based data processing workloads.Lead the integration of

SAP S/4HANA

data with GCP for real-time and batch data processing. Manage data extraction, transformation, and loading (ETL) processes from SAP S/4HANA into cloud storage and data lakes.Develop and manage

scalable data ingestion pipelines

for structured and unstructured data using tools like

Cloud Dataflow ,

Cloud Pub/Sub , and

Apache Spark .Provide architectural guidance for designing secure, scalable, and efficient data solutions on the

Google Cloud Platform , integrating with on-premise/cloud systems like

SAP S/4HANA .Implement both real-time streaming and batch processing pipelines using

Apache Spark ,

Dataflow , and other GCP services to meet business requirements.Implement data governance, access controls, and security best practices to ensure the integrity, confidentiality, and compliance of data across systems.Collaborate with business stakeholders, data scientists, and engineering teams to define data requirements, ensuring the architecture aligns with business goals.Optimize Apache Spark jobs for performance, scalability, and cost-efficiency, ensuring that the architecture can handle growing data volumes.Provide technical leadership to the data engineering team, mentoring junior engineers in data architecture, Apache Spark development, and GCP best practices.

Technical Expertise:

Expert-level programming proficiency in Python, Java, and Scala.Extensive hands-on experience with big data technologies, including Spark, Hadoop, Hive, Yarn, MapReduce, Pig, Kafka, and PySpark.Proficient in Google Cloud Platform services, such as

BigQuery ,

Dataflow ,

Cloud Storage ,

Dataproc , and

Cloud Composer

Google Pub/Sub, and Google Cloud Functions.Expertise in

Apache Spark

for both batch and real-time processing, as well as proficiency in

Apache Beam ,

Hadoop , or other big data frameworks.Experienced in using Cloud SQL, BigQuery, and Looker Studio (Google Data Studio) for cloud-based data solutions.Skilled in orchestration and deployment tools like Cloud Composer, Airflow, and Jenkins for continuous integration and deployment (CI/CD).Expertise in designing and developing integration solutions involving Hadoop/HDFS, real-time systems, data warehouses, and analytics solutions.Experience with DevOps practices, including version control ( Git ), CI/CD pipelines, and

infrastructure-as-code

(e.g.,

Terraform ,

Cloud Deployment Manager ).Strong background in working with relational databases, NoSQL databases, and in-memory databases.Experience managing large datasets within Data Lake and Data Fabric architectures.Strong knowledge of security best practices, IAM, encryption mechanisms, and compliance frameworks (GDPR, HIPAA) within GCP environments.Experience in implementing data governance, data lineage, and data quality frameworks.In-depth knowledge of web technologies, application programming languages, OLTP/OLAP technologies, data strategy disciplines, relational databases, data warehouse development, and big data solutions.Led end-to-end processes for the design, development, deployment, and maintenance of data engineering projects.Excellent debugging and problem-solving skills.Retail and e-commerce domain knowledge is a plus.Positive attitude with strong analytical skills and the ability to guide teams effectively.

Preferred Qualifications:

GCP Certifications : Such as

Professional Data Engineer

or

Professional Cloud Architect .Apache Spark and Python certifications.Experience with Data visualization tools like Tableau, Power BI etc.

  • Cloud Architect Lead

    3 days ago

    Bengaluru, Karnataka, India Rapyder Cloud Solutions Full time

    Rapyder Cloud Solutions seeks an experienced Cloud Architect Lead to join our team in developing advanced cloud architectures using automated techniques to deliver highly secure environments.We require a clear understanding of Kubernetes Architectural components, communications, and flows.Prior involvement in Kubernetes implementation projects is...

  • Cloud Solutions Architect

    1 month ago

    Bengaluru, Karnataka, India Cloud Software Group Full time

    Key Responsibilities: As a Cloud Solutions Architect at Cloud Software Group, you will be responsible for implementing, configuring, and customizing Oracle Fusion Financials applications to meet the organization's requirements. Your role will also involve providing ongoing support and troubleshooting for Oracle Fusion Financials modules, collaborating with...

  • Cloud Architect

    6 months ago

    Bengaluru, Karnataka, India iXceed Solutions Full time

    **Job Title**: Cloud Architect **Job Location**: Banglure **Job Type**: Hybrid Are you ready to shape the future of cloud computing? We are seeking a talented Cloud Architect to lead the design and implementation of cutting-edge cloud solutions for our organization. As a Cloud Architect, you will play a pivotal role in driving our cloud strategy,...

  • Cloud Architect

    6 months ago

    Bengaluru, Karnataka, India iXceed Solutions Full time

    **Job Title**: Cloud Architect **Job Location**: Banglure **Job Type**: Hybrid Are you ready to shape the future of cloud computing? We are seeking a talented Cloud Architect to lead the design and implementation of cutting-edge cloud solutions for our organization. As a Cloud Architect, you will play a pivotal role in driving our cloud strategy,...

  • Cloud Architect

    6 months ago

    Bengaluru, Karnataka, India iXceed Solutions Full time

    **Job Title**: Cloud Architect **Job Location**: Banglure **Job Type**: Hybrid Are you ready to shape the future of cloud computing? We are seeking a talented Cloud Architect to lead the design and implementation of cutting-edge cloud solutions for our organization. As a Cloud Architect, you will play a pivotal role in driving our cloud strategy,...

  • Cloud Architect

    6 months ago

    Bengaluru, Karnataka, India iXceed Solutions Full time

    **Job Title**: Cloud Architect **Job Location**: Banglure **Job Type**: Hybrid Are you ready to shape the future of cloud computing? We are seeking a talented Cloud Architect to lead the design and implementation of cutting-edge cloud solutions for our organization. As a Cloud Architect, you will play a pivotal role in driving our cloud strategy,...

  • Cloud Architect

    6 months ago

    Bengaluru, Karnataka, India iXceed Solutions Full time

    **Job Title**: Cloud Architect **Job Location**: Banglure **Job Type**: Hybrid Are you ready to shape the future of cloud computing? We are seeking a talented Cloud Architect to lead the design and implementation of cutting-edge cloud solutions for our organization. As a Cloud Architect, you will play a pivotal role in driving our cloud strategy,...

  • Cloud Architect

    6 months ago

    Bengaluru, Karnataka, India iXceed Solutions Full time

    **Job Title**: Cloud Architect **Job Location**: Banglure **Job Type**: Hybrid Are you ready to shape the future of cloud computing? We are seeking a talented Cloud Architect to lead the design and implementation of cutting-edge cloud solutions for our organization. As a Cloud Architect, you will play a pivotal role in driving our cloud strategy,...

  • Cloud Architect

    3 weeks ago

    Bengaluru, Karnataka, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Job DescriptionWe are seeking a highly skilled Cloud Architect with expertise in Integration Services to join our team at Cloud Kinetics Technology Solutions Private Ltd.OverviewThe successful candidate will design, develop, and implement integration solutions using Microsoft Azure services, focusing on building scalable cloud-based architectures. The role...

  • Cloud Architect

    6 months ago

    Bengaluru, Karnataka, India iXceed Solutions Full time

    **Job Title**: Cloud Architect **Job Location**: Banglure **Job Type**: Hybrid Are you ready to shape the future of cloud computing? We are seeking a talented Cloud Architect to lead the design and implementation of cutting-edge cloud solutions for our organization. As a Cloud Architect, you will play a pivotal role in driving our cloud strategy,...

  • Cloud Architect

    3 days ago

    Bengaluru, Karnataka, India MathCo Full time

    About the RoleWe are seeking an experienced Cloud Architect to join our Data Engineering team. As a Cloud Architect, you will be responsible for designing, storing, processing, and maintaining large-scale data infrastructure.Key ResponsibilitiesConsult with clients and stakeholders to understand business problemsDesign and implement data pipelines, ensuring...

  • Cloud Architect

    3 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Job Purpose and primary objectives:Cloud ArchitectKey responsibilities:Assist in implementing and supporting cloud implementations on Amazon AWS.Supervisory / Managerial responsibilities(please specify if the position will have persons reporting to it):Other duties include architecting and implementing cloud solutions for different applications, automating...

  • Cloud Architect

    6 months ago

    Bengaluru, India Verint Systems Inc. Full time

    Description: As a Cloud Architect, you will assist in implementing and supporting large-scale cloud implementations on AWS. Other Duties include architecting and implementing cloud solutions for different Verint Applications, automating cloud operations delievrables, Cloud Deployments and Programming. Key Accountabilities - Enhance our cloud capability by...

  • Cloud architect

    3 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Job Purpose and primary objectives:Cloud ArchitectKey responsibilities:Assist in implementing and supporting cloud implementations on Amazon AWS.Supervisory / Managerial responsibilities (please specify if the position will have persons reporting to it):Other duties include architecting and implementing cloud solutions for different applications,...

  • Cloud architect

    1 month ago

    Bengaluru, India HCLTech Full time

    • Expertise in Cloud Migration and demonstrable experience of having successfully migrated a customer to a major public cloud platform (AWS/Azure)• Expertise in using public cloud platforms (AWS/Azure) to build solutions solving business problems and making innovative use of Saa S and Paa S offerings.• Understands whole solution architecture concepts...

  • Cloud Architect

    1 month ago

    Bengaluru, India HCLTech Full time

    • Expertise in Cloud Migration and demonstrable experience of having successfully migrated a customer to a major public cloud platform (AWS/Azure) • Expertise in using public cloud platforms (AWS/Azure) to build solutions solving business problems and making innovative use of SaaS and PaaS offerings. • Understands whole solution architecture concepts...

  • Cloud Architect

    1 month ago

    Bengaluru, India HCLTech Full time

    • Expertise in Cloud Migration and demonstrable experience of having successfully migrated a customer to a major public cloud platform (AWS/Azure) • Expertise in using public cloud platforms (AWS/Azure) to build solutions solving business problems and making innovative use of SaaS and PaaS offerings. • Understands whole solution architecture concepts...

  • Cloud Architect

    1 month ago

    Bengaluru, India HCLTech Full time

    • Expertise in Cloud Migration and demonstrable experience of having successfully migrated a customer to a major public cloud platform (AWS/Azure) • Expertise in using public cloud platforms (AWS/Azure) to build solutions solving business problems and making innovative use of SaaS and PaaS offerings. • Understands whole solution architecture...

  • Cloud Architect

    6 months ago

    Bengaluru, Karnataka, India Atlas Systems Full time

    **AWS Architect** **Work Location: Bangalore** About Atlas: Atlas Systems Inc. is a Software Solutions company headquartered in East Brunswick, NJ. Incorporated in 2003, Atlas provides comprehensive range of solutions in the area of GRC, Technology, Procurement, Healthcare Provider and Oracle to customers across the globe. Combining our unparalleled...

  • Cloud Architect

    3 months ago

    Bengaluru, Karnataka, India Philips Full time

    **Job Title**: Cloud Architect **Cloud Architect** We are seeking a skilled Cloud Architect to join our team. The Cloud Architect will play a critical role in designing, implementing, and maintaining the cloud infrastructure and edge services required to support field-deployed console systems. Leveraging expertise in C#, **AWS**, telemetry, and edge...