Lead Cloud Architect

4 days ago

Aurangabad, Maharashtra, India beBeePython Full time US$ 1,80,000 - US$ 2,00,000

Job Description

">

We are seeking a Lead Python Engineer to lead our team in designing and building scalable data pipelines using Apache Beam and Databricks.

">Key Responsibilities:">
  • ">
  • ">Architecture and Reuse:">
    • ">
    • Design a shared component library/SDK for pipelines: ingestion, parsing/OCR, extraction (RegEx now, LLM/SLM later), validation, enrichment, publishing.">
    • Define patterns/templates for Apache Beam pipelines and Databricks jobs; standardize configuration, packaging, versioning, CI/CD, and documentation.">
    • Create pluggable interfaces so multiple teams can swap extractors (Regex/LLM), OCR providers, and EMR publishers without code rewrites.">
    • Define repository strategy - shared/child repos for each use case">
    ">
  • ">Performance and Reliability:">
    • ">
    • Own end-to-end profiling and tuning: cProfile/py-spy/line_profiler, memory (tracemalloc), CPU vs I/O analysis.">
    • Instrument services with Elastic APM and correlate traces/metrics with Splunk logs; build dashboards and runbooks.">
    • Implement concurrency best practices: asyncio for I/O-bound, ThreadPool/ProcessPool for CPU-bound, batching, rate limiting, retries, etc.">
    • Implement robust LLM API rate limiting/governance: enforce provider TPM and concurrency caps, request queueing/token budgeting, and emit APM/Splunk metrics (throttle rate, queue depth, cost per job) with alerts.">
    • Establish service level objectives/alerts for throughput, latency, error rates; set up dead letter queues and recovery patterns.">
    ">
  • ">Team Enablement:">
    • ">
    • Mentor developers, lead design reviews, codify best practices, write clear documentation and examples.">
    • Partner with machine learning engineers on the future LLM/SLM path (evaluation harness, safety/PII, cost/performance).">
    ">Requirements">
    • ">
    • 7+ years of Python experience with strong expertise in performance and concurrency (asyncio, concurrent.futures, multiprocessing), profiling and memory tuning.">
    • Observability expertise: Elastic APM instrumentation and dashboarding; Splunk for logs and correlation; OpenTelemetry familiarity.">
    • Must have implemented large language model-based solutions and supported them in production.">
    • API engineering for high-throughput integrations (REST, OAuth2), resilience patterns, and secure handling of sensitive data.">
    • Strong architecture/design skills: clean interfaces, packaging shared libraries, versioning, CI/CD (GitHub Actions/Azure DevOps), testing.">
    • 3+ years of building large-scale data pipelines with Apache Beam and/or Spark, including hands-on Databricks experience (Jobs, Delta Lake, cluster tuning).">
    • Document processing: OCR (Tesseract, AWS Textract, Azure Form Recognizer), PDF parsing, text normalization.">
    • LLM/SLM integration experience (e.g., OpenAI/Azure AI, local SLMs), prompt/eval frameworks, PII redaction/guardrails.">
    • Cloud and tooling: AWS/Azure/GCP, Dataflow/Flink, Terraform, Docker; cost/performance tuning on Databricks.">
    • Security/compliance mindset (HIPAA), secrets management, least-privilege access.">
    ",

  • Cloud Architect

    3 days ago

    Aurangabad, Maharashtra, India beBeeCloudExpert Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Job Title: Cloud ArchitectWe are seeking a seasoned professional to lead the design and implementation of our cloud infrastructure on AWS.Key Responsibilities:Develop scalable, secure, and highly available AWS architectures.Collaborate with stakeholders to understand business requirements and translate them into robust AWS solutions.Implement AWS monitoring...

  • Cloud Security Solutions Architect

    4 days ago

    Aurangabad, Maharashtra, India beBeeCloudSecurity Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Cloud security architects are responsible for designing and implementing secure cloud-based systems, including public, private, and hybrid environments.Key ResponsibilitiesDeveloping security reference architectures to ensure the confidentiality, integrity, and availability of cloud data.Leading the development of a comprehensive security vision and strategy...

  • Cloud Data Architect

    6 days ago

    Aurangabad, Maharashtra, India beBeeEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Our organization is seeking an experienced Engineering Manager to lead our Big Data and Azure teams.About the Role:As a seasoned professional, you will be responsible for designing and delivering scalable data pipelines and analytics platforms on Microsoft Azure and Databricks.The ideal candidate will have expertise in cloud-native architecture, data...

  • Chief Cloud Strategist

    4 days ago

    Aurangabad, Maharashtra, India beBeeCloud Full time ₹ 15,00,000 - ₹ 25,00,000

    Cloud Architect RoleAs a Cloud Architect, you will play a pivotal role in designing and implementing secure, scalable, and cost-optimized solutions across multiple cloud platforms. Your expertise will be essential in driving the success of our cloud infrastructure.Lead multi-cloud strategies and design hybrid cloud platforms integrating on-prem and public...

  • Leading Identity Architect

    2 days ago

    Aurangabad, Maharashtra, India beBeeIdentity Full time ₹ 1,00,00,000 - ₹ 1,20,00,000

    Job Title: Identity Architect LeadA trusted solution advisor to both internal and external stakeholders, providing strategic input, solution design, and business alignment for Identity & Access Management initiatives.About the Role:We are seeking an experienced Identity Architect Lead who will serve as a trusted advisor to our clients, delivery teams, and...

  • Cloud Expert

    6 days ago

    Aurangabad, Maharashtra, India beBeeArchitecture Full time US$ 1,50,000 - US$ 2,50,000

    Job Title :: Salesforce ArchitectLocation :: Remote (India)Domain experience :: Artificial Intelligence, AutomationExperience :: Minimum 10 years relevant experience in Sales Cloud, Service Cloud, Marketing CloudJob Description:We are seeking a talented Salesforce Solution Architect to lead the design and implementation of scalable high-performance...

  • Cloud Visionary

    4 days ago

    Aurangabad, Maharashtra, India beBeeCloud Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job DescriptionA career as a Cloud Architect presents an opportunity to be at the forefront of shaping cloud-based solutions for optimal scalability and growth.This role involves working closely with clients to design cloud applications that leverage the full potential of various cloud platforms, primarily Google, AWS, and Azure, along with marketplace...

  • Cloud Infrastructure Professional

    4 days ago

    Aurangabad, Maharashtra, India beBeeInfrastructure Full time ₹ 35,00,000 - ₹ 40,00,000

    Cloud Infrastructure ExpertWe are seeking an experienced Cloud Architect to lead the design, security, and deployment strategies of our cloud infrastructure. The ideal candidate will architect scalable, secure, and resilient cloud solutions and enforce best practices such as CIS Benchmarks and STIG compliance.Key ResponsibilitiesDesign and implement secure,...

  • Cloud Infrastructure Strategist

    6 days ago

    Aurangabad, Maharashtra, India beBeeCloud Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Cloud Architect LeadA seasoned Cloud Architect is needed to spearhead our cloud infrastructure strategy. The ideal candidate will have extensive experience in designing, developing, and deploying cloud-based applications using modern programming languages and frameworks.Key Responsibilities:Design and implement scalable and secure cloud architecture for...

  • Cloud Solution Specialist

    3 days ago

    Aurangabad, Maharashtra, India beBeeCloudSolution Full time ₹ 12,00,000 - ₹ 25,00,000

    Job OverviewWe are seeking an experienced Cloud Solutions Specialist to design and implement scalable, secure, and cost-effective cloud solutions tailored to business needs.Collaborate with stakeholders to translate requirements into cloud architecture and ensure best practices.ResponsibilitiesCreate cloud-based architectures and solutions using AWS, Azure,...