Cygnus Professionals Inc. | Cloud Architect

2 weeks ago

bangalore, India Cygnus Professionals Inc. Full time

Position: GCP Architect (Apache Spark)

Location: Bangalore

Duration: Full Time (4 Days WFO)

Job Description for Data Engineering Architect (GCP & Apache Spark)

Job Summary:

We are seeking a highly skilled Data Engineering Architect with deep expertise in Google Cloud Platform (GCP) , Apache Spark , to architect, design and implement scalable, high-performance data lake solutions. The ideal candidate will have extensive experience in building data ingestion pipelines, managing big data processing using Apache Spark ,

Key Requirements:


  • Over 12 years of professional experience in data engineering, specializing in implementing large-scale enterprise Data Engineering projects with the latest technologies.
  • Over 5 years of hands-on experience in GCP technologies and over 3 years of architect experience.
  • Design and implement end-to-end data architectures leveraging GCP services (e.g., Big Query, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer ) for large-scale data ingestion and processing.
  • Build and optimize large-scale data pipelines using Apache Spark on GCP (via Dataproc or other Spark services). Ensure high performance and scalability in Spark-based data processing workloads.
  • Lead the integration of SAP S/4HANA data with GCP for real-time and batch data processing. Manage data extraction, transformation, and loading (ETL) processes from SAP S/4HANA into cloud storage and data lakes.
  • Develop and manage scalable data ingestion pipelines for structured and unstructured data using tools like Cloud Dataflow , Cloud Pub/Sub , and Apache Spark .
  • Provide architectural guidance for designing secure, scalable, and efficient data solutions on the Google Cloud Platform , integrating with on-premise/cloud systems like SAP S/4HANA .
  • Implement both real-time streaming and batch processing pipelines using Apache Spark , Dataflow , and other GCP services to meet business requirements.
  • Implement data governance, access controls, and security best practices to ensure the integrity, confidentiality, and compliance of data across systems.
  • Collaborate with business stakeholders, data scientists, and engineering teams to define data requirements, ensuring the architecture aligns with business goals.
  • Optimize Apache Spark jobs for performance, scalability, and cost-efficiency, ensuring that the architecture can handle growing data volumes.
  • Provide technical leadership to the data engineering team, mentoring junior engineers in data architecture, Apache Spark development, and GCP best practices.


Technical Expertise:


  • Expert-level programming proficiency in Python, Java, and Scala.
  • Extensive hands-on experience with big data technologies, including Spark, Hadoop, Hive, Yarn, MapReduce, Pig, Kafka, and PySpark.
  • Proficient in Google Cloud Platform services, such as BigQuery , Dataflow , Cloud Storage , Dataproc , and Cloud Composer Google Pub/Sub, and Google Cloud Functions.
  • Expertise in Apache Spark for both batch and real-time processing, as well as proficiency in Apache Beam , Hadoop , or other big data frameworks.
  • Experienced in using Cloud SQL, BigQuery, and Looker Studio (Google Data Studio) for cloud-based data solutions.
  • Skilled in orchestration and deployment tools like Cloud Composer, Airflow, and Jenkins for continuous integration and deployment (CI/CD).
  • Expertise in designing and developing integration solutions involving Hadoop/HDFS, real-time systems, data warehouses, and analytics solutions.
  • Experience with DevOps practices, including version control (Git ), CI/CD pipelines, and infrastructure-as-code (e.g., Terraform , Cloud Deployment Manager ).
  • Strong background in working with relational databases, NoSQL databases, and in-memory databases.
  • Experience managing large datasets within Data Lake and Data Fabric architectures.
  • Strong knowledge of security best practices, IAM, encryption mechanisms, and compliance frameworks (GDPR, HIPAA) within GCP environments.
  • Experience in implementing data governance, data lineage, and data quality frameworks.
  • In-depth knowledge of web technologies, application programming languages, OLTP/OLAP technologies, data strategy disciplines, relational databases, data warehouse development, and big data solutions.
  • Led end-to-end processes for the design, development, deployment, and maintenance of data engineering projects.
  • Excellent debugging and problem-solving skills.
  • Retail and e-commerce domain knowledge is a plus.
  • Positive attitude with strong analytical skills and the ability to guide teams effectively.

Preferred Qualifications:

  • GCP Certifications : Such as Professional Data Engineer or Professional Cloud Architect .
  • Apache Spark and Python certifications.

Experience with Data visualization tools like Tableau, Power BI etc.

  • Cygnus Professionals Inc. | Cloud Architect

    2 weeks ago

    bangalore, India Cygnus Professionals Inc. Full time

    Position: GCP Architect (Apache Spark)Location: BangaloreDuration: Full Time (4 Days WFO) Job Description for Data Engineering Architect (GCP & Apache Spark) Job Summary: We are seeking a highly skilled Data Engineering Architect with deep expertise in Google Cloud Platform (GCP), Apache Spark, to architect, design and implement scalable, high-performance...

  • Cygnus Professionals Inc. | Oracle CPQ | bangalore

    4 days ago

    bangalore, India Cygnus Professionals Inc. Full time

    Position: Oracle CPQ Location: PAN India Duration: Long Term Contract The Oracle CPQ (Configure, Price, Quote) developer will perform hands-on technical implementation, with a focus on delivering functional solutions on the Oracle CPQ platform as well as Provide guidance and best practices around CPQ solutions including product configuration, administration...

  • Cygnus Professionals Inc. | GCP Data Lead | bangalore

    2 weeks ago

    bangalore, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • Cygnus Professionals Inc. | GCP Data Lead | bangalore

    1 week ago

    bangalore, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data Lead Location: Bangalore, India (Day 1 Onsite) Job Type: Contract Job Description: Strong GCP experience Prior experience in building pipelines using dataflow, data fusion, data proc, etc Very strong python coding skill Experience in optimising GCP resources for cost and performance Experience in containerisation of...

  • Cygnus Professionals Inc. | GCP Data Lead | bangalore

    2 weeks ago

    bangalore, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data Lead Location: Bangalore, India (Day 1 Onsite) Job Type: Contract Job Description: Strong GCP experience Prior experience in building pipelines using dataflow, data fusion, data proc, etc Very strong python coding skill Experience in optimising GCP resources for cost and performance Experience in containerisation of...

  • Cloud Architect

    2 weeks ago

    bangalore, India Cygnus Professionals Inc. Full time

    Position: GCP Architect (Apache Spark) Location: Bangalore Duration: Full Time (4 Days WFO) Job Description for Data Engineering Architect (GCP & Apache Spark) Job Summary: We are seeking a highly skilled Data Engineering Architect with deep expertise in Google Cloud Platform (GCP), Apache Spark, to architect, design and implement scalable,...

  • Cloud Architect

    2 weeks ago

    bangalore, India Cygnus Professionals Inc. Full time

    Position: GCP Architect (Apache Spark) Location: Bangalore Duration: Full Time (4 Days WFO) Job Description for Data Engineering Architect (GCP & Apache Spark) Job Summary: We are seeking a highly skilled Data Engineering Architect with deep expertise in Google Cloud Platform (GCP) , Apache Spark , to architect, design and implement scalable,...

  • Cloud Architect

    2 weeks ago

    bangalore, India Cygnus Professionals Inc. Full time

    Position: GCP Architect (Apache Spark)Location: BangaloreDuration: Full Time (4 Days WFO)Job Description for Data Engineering Architect (GCP & Apache Spark)Job Summary:We are seeking a highly skilled Data Engineering Architect with deep expertise in Google Cloud Platform (GCP) , Apache Spark , to architect, design and implement scalable, high-performance...

  • Cloud Infrastructure Architect

    4 weeks ago

    Hyderabad/Noida/Chennai/Pune/Bangalore, India TETRAHED INC Full time

    Job Title: Senior Infrastructure ArchitectJob Summary:We are seeking a highly skilled Senior Infrastructure Architect to join TETRAHED INC in designing and implementing scalable, secure, and efficient cloud infrastructure solutions. As a key member of our team, you will be responsible for architecting and developing cloud solutions that meet the complex...

  • GCP Data Lead

    2 weeks ago

    bangalore, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data Lead Location: Bangalore, India (Day 1 Onsite) Job Type: Contract Job Description: Strong GCP experience Prior experience in building pipelines using dataflow, data fusion, data proc, etc Very strong python coding skill Experience in optimising GCP resources for cost and performance Experience in containerisation of...

  • Gcp data lead

    1 week ago

    Bangalore, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data Lead Location: Bangalore, India (Day 1 Onsite) Job Type: Contract Job Description: Strong GCP experience Prior experience in building pipelines using dataflow, data fusion, data proc, etc Very strong python coding skill Experience in optimising GCP resources for cost and performance Experience in...

  • Cloud Data Solutions Architect

    4 weeks ago

    Noida/Bangalore/Hyderabad/Pune/Chennai, India TETRAHED INC Full time

    About the Role:TETRAHED INC. is seeking an experienced Data Architect to lead the design and implementation of our data platform on Google Cloud Platform (GCP).This role involves architecting, building, and optimizing data solutions to drive business insights and decisions. As a key member of the data engineering team, you will be responsible for defining...

  • GCP Data Lead

    1 week ago

    bangalore, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • Cloud Architect Professional

    4 weeks ago

    Bangalore, India Growel Softech Pvt. Ltd. Full time

    Job Title: Cloud Architect ProfessionalJob Summary: We are seeking a highly skilled Cloud Architect Professional to join our team at Growel Softech Pvt. Ltd. As a Cloud Architect Professional, you will be responsible for designing, deploying, and managing scalable solutions across AWS, Azure, and GCP platforms.About the Role:Design and Architect Cloud...

  • ServiceNow Platform Architect

    4 weeks ago

    Remote/Bangalore/Hyderabad/Chennai, India LanceSoft, Inc Full time

    Job Title: Cloud ServiceNow ArchitectJob Description:We are seeking an experienced Cloud ServiceNow Architect to join our team at LanceSoft, Inc. As a Cloud ServiceNow Architect, you will be responsible for designing and implementing cloud-based solutions using ServiceNow.Responsibilities:Design and implement cloud-based ServiceNow solutionsCollaborate with...

  • AWS Cloud Solutions Architect

    4 weeks ago

    Hyderabad/Bangalore/Pune, India Crox Consulting Inc Full time

    Job Title: AWS Cloud Solutions ArchitectJob Summary: We are seeking a skilled AWS Cloud Solutions Architect to design, develop, and manage cloud-based applications and infrastructure for Crox Consulting Inc.Key Responsibilities:Design, develop, and deploy cloud-based solutions using AWS services.Implement and manage AWS infrastructure, including compute,...

  • Senior IoT Architect

    4 weeks ago

    Bangalore/Pune, India Wenger & Watson Inc. Full time

    Job Title: Senior IoT ArchitectJob Summary:We are seeking a Senior IoT Architect to join our team at Wenger & Watson Inc. The ideal candidate will have experience in designing and implementing end-to-end Industrial IoT solutions that leverage device, edge, and cloud technologies.Key Responsibilities:Develop and implement end-to-end IoT solutions that are...

  • Cloud Architect

    4 weeks ago

    Bangalore, India The Metromax Group Full time

    Key Responsibilities:Cloud Architecture:- Design and implement scalable, secure, and high-performing cloud architectures on AWS.- Collaborate with development and data teams to architect cloud-native applications and microservices.Cloud Migration:- Lead cloud migration strategies for moving on-premises applications and data to AWS.- Develop and enforce best...

  • Automation Test Architect

    1 month ago

    Bangalore, India TETRAHED INC Full time

    Automation ArchitectTETRAHED INC is seeking a highly skilled Automation Architect to join our innovative team in Bangalore. This is an exciting opportunity for a seasoned professional who is passionate about automation and eager to drive quality through cutting-edge technology.Key Responsibilities:Design and implement advanced automation frameworks tailored...

  • Cloud Strategy and Architecture Lead

    4 weeks ago

    Noida/Pune/Hyderabad/Bangalore/Chennai, India TETRAHED INC Full time

    **Job Title:** Cloud Strategy and Architecture LeadAt TETRAHED INC, we are seeking an experienced Cloud Strategy and Architecture Lead to develop and execute our cloud strategy aligned with business objectives.**Key Responsibilities:**Lead the design, architecture, and implementation of cloud-based solutions on platforms like AWS, Azure, and Google...