Cloud and AI Architecture: Case Studies · Part 4 of 4

Virelane AI

A late-stage enterprise AI platform and its architecture on AWS, GCP, and Azure.

Virelane AI has thousands of paying seats inside institutions that measure downtime in regulatory filings rather than support tickets. Whether the product works and whether anyone will buy it are settled questions. What's open is whether the company can run at this size indefinitely: hold its latency targets as volume grows, keep several hundred engineers building one coherent platform, absorb a bad day without it becoming a regulatory event, and do all of it at a margin that survives an IPO prospectus.

Virelane AI is fictional: a composite typical of a late-stage enterprise software company, modeled on no specific business.

The company

Virelane AI sells software that reads, reasons over, and answers questions against an enterprise's own internal documents: loan files and underwriting memos at banks, claims and policy files at insurers, clinical documentation at hospital systems, case files and procurement records inside government agencies. A customer connects its document corpus, Virelane indexes it, and the customer's own employees ask it questions or trigger structured workflows, extracting every covenant in a loan agreement, flagging a claim for manual review, summarizing a patient's chart before a specialist appointment, instead of reading the underlying documents by hand.

Virelane's customers are the kind of institution whose compliance department can veto a vendor regardless of how good the product is, whose data can't leave a specific country, and whose auditors expect a documented answer for every action a piece of software took on their behalf.

Customers and business model

Virelane sells multi-year enterprise contracts. A single account, a top-20 bank, a national insurer, a hospital network, a federal agency, can be worth anywhere from a few hundred thousand to tens of millions of dollars a year, priced on some mix of seats, document volume, and inference usage. Sales cycles run six to eighteen months and pass through security review, procurement, and often a pilot run inside the customer's own environment before a contract is signed. Once signed, contracts tend to renew for years, so expansion within existing accounts drives as much growth as new logos do.

Onboarding a large account is its own project: connecting to the customer's identity provider, agreeing on which region their data will live in, running the customer's security assessment against Virelane's controls, and for the largest or most regulated accounts, standing up dedicated infrastructure rather than a shared multi-tenant deployment.

Funding stage and what's next

Virelane is late-stage, Series D or beyond, or already profitable on its core business, with revenue in the hundreds of millions or a credible board-level path there. Headcount runs from several hundred to a few thousand, with an engineering organization large enough to split into platform teams (core infrastructure, AI and model serving, security and compliance, data platform) and product teams organized by industry vertical. The next milestone is an IPO or another round of growth financing, either of which puts the company's financial controls and operational maturity in front of scrutiny well beyond any single customer's procurement questionnaire.

What changes between Series A and late stage

Compliance is the most visible difference, and it's the one that gets overstated. A Series A company selling to enterprises already fills out security questionnaires and already commits to a data region. Late stage doesn't introduce that work; it changes four other things, and they matter more to how the system gets built.

Performance becomes a contract term rather than a goal. At Series A, slow is a complaint. At Virelane's stage, a latency target appears in a signed agreement with a service credit attached, which means the architecture has to hold a number under a load nobody controls, in every region, including during a deployment. That turns capacity planning, GPU reservation, and request prioritization into standing engineering work rather than something done when a dashboard turns red.

Risk management moves from instinct to a standing function. A Series A company manages risk by having a few people who know where the sharp edges are. At several hundred engineers, that knowledge doesn't fit in anyone's head, so it has to live in systems: a threat model that's reviewed, a vendor risk process, a business continuity plan that's tested on a schedule, a standing security operations team with its own on-call.

Process maturity stops being overhead and starts being the thing that lets the company grow. Change approval, mandatory review, automated scanning, progressive rollout, and a documented on-call rotation all slow an individual team down. They're worth it because the alternative at this size is every team discovering the same failure independently. The design question is where to put the gates so they catch real problems without turning every release into a committee.

Efficiency becomes a headline metric. A Series A company is measured on growth. A late-stage company approaching an IPO is measured on growth and gross margin, and infrastructure is now a large enough share of cost of revenue to move that number. GPU utilization, per-customer cost to serve, and the price of running multi-region duplication all become things a finance organization asks about by name.

Audit readiness sits on top of all four. It's demanding, and it's largely a consequence of doing the other four well enough to have evidence of it.

Usage pattern

Three workloads share the platform, and they compete for the same GPU capacity.

Interactive queries are what a customer's employees do all day: asking a question against an indexed corpus and waiting for an answer. Volume follows business hours in each region, so load moves from Europe to the Americas over the day and the European and US peaks overlap little. This is the workload with an SLA attached, and it gets first claim on inference capacity.

Bulk ingestion and re-indexing arrives in large, scheduled blocks. Onboarding a new account can mean embedding an enterprise's entire historical archive, millions of documents in one push. Changing the embedding model means that same work again across every existing customer, because vectors produced by two different models occupy unrelated coordinate spaces: a query embedded by the new model and a document embedded by the old one produce a similarity score that means nothing. The dimensions often differ too, so the index can't hold both. Changing the generation model, the LLM that writes the answer, costs none of this; the index is untouched and the swap is a configuration change. Both kinds of bulk work are interruptible and neither has a human waiting, so they run on whatever capacity the interactive tier isn't using, on spot and preemptible instances.

Scheduled extraction workflows sit in between. A customer's overnight batch run pulling every covenant out of a day's loan files has a deadline, so it can't be preempted indefinitely, and no person is watching it, so it doesn't need a sub-second response. It runs in the trough between regional peaks.

Separating these three is what makes the cost model work. Sizing GPU capacity for interactive peak and leaving it idle between peaks wastes the most expensive resource in the platform.

Requirements

Functional

• Ingestion and indexing pipelines across structured and unstructured document formats, at the scale of an entire enterprise's historical archive rather than one team's working set
• LLM-based question answering and decision support over that indexed content, including retrieval-augmented generation and, for some workflows, extraction into structured fields consumed by the customer's own downstream systems
• Multi-tenant workspace isolation, now expressed per named enterprise customer and often per business unit within one customer
• Enterprise SSO and SCIM-based user provisioning against each customer's own identity provider
• Fine-grained access control down to the document or field level, mirroring each customer's internal permission model
• Administrative and audit tooling covering who accessed what, what the model was asked, what it answered, and what a human did with that answer
• Regional deployment so a customer's data and inference stay inside a required jurisdiction
• A self-service admin console customer security teams use to configure their own retention, redaction, and access policies

Non-functional

Every requirement below is a contract term, an audit control, or both. At earlier stages most of them were informal commitments or aspirations.

The three risks that dominate at this stage

Earlier stages each had one risk that dominated: first whether anyone wanted the product, then whether the sales motion repeated. Virelane has cleared both, and what's left is harder to fix with a good quarter.

Organizational risk. A few hundred engineers spread across platform and vertical product teams can drift apart faster than any one architect can track. Left alone, one team standardizes on one message queue, another on a different one, and a third builds its own logging pipeline because the shared one didn't fit its needs. No single decision is the cost. The cost is the accumulating tax of nothing being reusable across teams nominally building the same platform.

Regulatory and security risk. A single serious breach, or a failed audit at a top account, can cost Virelane its largest customers and trigger scrutiny from more than one regulator simultaneously. This is qualitatively different from an early-stage breach: the blast radius reaches other regulated institutions watching how Virelane responds, well beyond the one affected customer.

Financial risk, reframed as efficiency. Virelane's financial risk has shifted from running out of money to growing revenue without growing infrastructure cost at the same rate. GPU inference, multi-region duplication, and the tooling all this governance requires are now large enough line items that an inefficient architecture shows up directly in gross margin, and by extension in the company's valuation.

Architecture: AWS

Every service named below is a mainstream AWS offering, chosen because Virelane's requirements leave little room for a simpler answer. What's different at this stage is that nearly every requirement has to be satisfied at the same time, across multiple regions, for customers who will ask to see the evidence.

Compute and containers: why Kubernetes now

Earlier in this curriculum, compute options asks whether a given startup needs Kubernetes at all. For most of them the answer is no: a managed container service or serverless platform is less to operate, and operational simplicity is worth more than the flexibility Kubernetes offers when one team runs a handful of services. Virelane is past that trade-off. It doesn't have one team; it has platform, AI-serving, security, data, and several vertical product teams, each shipping independently, each running a different mix of long-lived services, spiky request-driven APIs, scheduled batch jobs, and GPU inference workloads. What justifies Kubernetes here is needing one substrate all of those teams share, so scheduling, autoscaling, service discovery, and rollout tooling are solved once instead of five times. No single workload would justify it alone.

Virelane runs Amazon EKS in each active region. Node groups are split by workload shape: general-purpose Graviton instances for stateless services, a GPU node group (P5 or Inferentia2 instances, depending on the model) tainted so only inference workloads schedule onto it, and a small Fargate profile for short-lived, low-privilege batch jobs that don't need a persistent node. Karpenter handles node provisioning, scaling GPU capacity up for a batch job and back down afterward rather than holding idle GPU nodes on the theory they might be needed. Each product vertical gets its own namespace with resource quotas and network policies, so a runaway job in one vertical can't starve another's inference budget.

Global entry and multi-region topology

Virelane runs the customer-facing tier active-active across three regions, two in the US and one in the EU, because the availability terms in its largest contracts and its EU residency commitments require it, with more regions added as further residency contracts demand them. Route 53 does latency-based routing with health checks, backed by AWS Global Accelerator for a stable set of anycast entry points so a regional failover doesn't require a DNS propagation wait. Each region's EKS cluster is otherwise self-sufficient: it can serve its own customers' traffic end to end without a cross-region call on the request path, which keeps a regional incident from cascading into a global one.

flowchart TB
  U[Customer traffic] --> R53[Route 53 + Global Accelerator]
  R53 --> US1[us-east-1: EKS + Aurora writer]
  R53 --> US2[us-west-2: EKS + Aurora reader]
  R53 --> EU1[eu-west-1: EKS + Aurora reader]
  US1 -.->|global replication| US2
  US1 -.->|global replication| EU1
  US1 --> BR[Bedrock + SageMaker: regional model serving]
  US2 --> BR
  EU1 --> BR2[Bedrock + SageMaker: EU region]
  

Each region runs the same stack, covered piece by piece in the sections below:

flowchart TB
  IN[Customer traffic via
Global Accelerator
or Direct Connect] --> EKS
  IDC[Cognito user pools
customer IdP via
SAML/OIDC] --> EKS
  CICD[CodePipeline or
GitHub Actions, scans,
Argo Rollouts canary] -->|deploy| EKS
  EKS[EKS in a private VPC
service mesh mTLS, IRSA
namespace per vertical
Karpenter GPU nodes]
  EKS --> ROUTE[Model routing layer]
  ROUTE --> BR[Bedrock:
foundation models]
  ROUTE --> SMK[SageMaker:
fine-tuned or
dedicated models]
  EKS --> AUR[(Aurora Global DB
accounts, permissions)]
  EKS --> DDB[(DynamoDB
Global Tables
sessions, counters)]
  EKS --> S3[(S3 + Glue + Athena
analytics, eval logs)]
  EKS -->|audit events,
traces| OPS[Shared services
via Transit Gateway
KMS / CloudHSM keys
CloudTrail + Object Lock
GuardDuty, Security Hub,
Macie, Config
OpenTelemetry, SIEM]
  S3 ~~~ OPS
  AUR ~~~ ROUTE
  

Regional model serving and AI inference

Virelane routes inference through two paths. Amazon Bedrock serves general-purpose foundation-model traffic through regional endpoints, called directly rather than through cross-Region inference profiles (geographic profiles stay within a geography but can process a request in another region of it, and global profiles can route anywhere), so a prompt from an EU customer is processed in the EU region it landed in. For customers who won't allow their documents to reach a shared third-party model, or workloads using a model Virelane has fine-tuned on domain-specific data, SageMaker hosts dedicated or multi-model endpoints inside Virelane's own account, still region-pinned to match the customer's data. A thin routing layer in front of both picks the path per request based on the customer's contract terms, so the choice of model provider is a configuration decision, not a code branch scattered through the product.

GPU capacity is the scarcest resource in this picture, and it's managed accordingly: a baseline of Reserved Instances or Savings Plans covers steady-state SageMaker inference, Karpenter bursts EKS GPU node groups up for batch re-indexing jobs, and Spot Instances take on the fully interruptible batch work, overnight reprocessing and the bulk re-embedding an embedding-model change forces, that can tolerate a restart. See GPU and TPU infrastructure for the underlying utilization math.

Multi-region data strategy

Not every dataset needs the same consistency guarantee, and pretending otherwise is a common way this kind of architecture becomes needlessly expensive. Virelane splits its data tier by how it's used:

• Customer-facing relational data (accounts, permissions, workflow state) runs on Aurora Global Database: one primary region handles writes, secondary regions serve local reads with typical replication lag under a second, and a secondary can be promoted to primary in a regional failover. This is active-active for reads and active-passive for writes. Multi-writer replication was considered and rejected because the conflict-resolution complexity isn't worth it when a single write region is an acceptable constraint. This tier holds accounts, permissions, and workflow state, not document content, and replicating it to the EU has to be permitted by each EU customer's residency terms.
• High-throughput key-value data (session state, feature flags, per-request rate-limit counters) runs on DynamoDB Global Tables, which supports multi-region active-active writes natively and fits data that's naturally partitioned and doesn't need cross-row transactions.
• Analytical and batch data (usage analytics, model evaluation logs, historical audit exports) lives in S3 plus Glue and Athena, replicated to a secondary region on a schedule rather than continuously. This tier is active-passive: if the primary region is unavailable, analytics are delayed, and a regional loss costs only the data since the last replication run. That trade-off is fine because nothing customer-facing depends on it in real time.

See relational and NoSQL databases for the general decision criteria behind this split.

Private networking and zero trust

Each region gets its own VPC, connected through a Transit Gateway hub to a shared-services VPC holding logging, CI/CD, and security tooling. VPC endpoints (gateway endpoints for S3 and DynamoDB, AWS PrivateLink interface endpoints for Bedrock and Secrets Manager) give every service private access to them without a packet routing through a NAT gateway to the public internet. Endpoint policies and organization-level policies restrict those endpoints to Virelane's own resources, which limits where data can be sent from inside the network. The largest customers, the ones who won't accept any path over the public internet at all, connect over AWS Direct Connect, terminated into a dedicated VPC reserved for that customer's traffic.

Inside the cluster, zero trust means no service trusts a request just because it arrived on the right port. Service-to-service calls use mutual TLS via a service mesh, and each pod authenticates to AWS APIs through IAM roles for service accounts (IRSA) rather than a shared credential baked into an image. A compromised pod in one namespace can't call another namespace's database directly; it has to go through the same authenticated, authorized path anything else would.

Encryption and key management

Everything is encrypted at rest and in transit by default, using AWS KMS with envelope encryption. What changes per customer is who holds the key. Most accounts use Virelane-managed keys, which are simpler to operate and sufficient for the compliance bar most customers require. The largest regulated accounts, the ones whose contracts specify it or whose own auditors require it, get a per-tenant key the customer controls (a KMS key in the customer's own AWS account shared with Virelane through a grant, so the customer can revoke access by disabling it), with a key policy restricting decrypt access to the specific IAM roles that service handles their workload. A small number of the most stringent government and financial accounts use keys in CloudHSM or in an external key store (AWS KMS XKS) they operate, so the key material stays outside AWS-managed KMS. KMS rotates standard keys automatically; keys held in an HSM or external store rotate on a schedule the key owner runs.

Security operations and audit logging

Amazon GuardDuty watches for anomalous behavior across accounts, catching the kind of credential misuse or unusual API activity a top-20 bank's own security team will specifically ask whether Virelane monitors for. Security Hub (with Security Hub CSPM running the CIS benchmark checks) aggregates findings into one compliance posture view, the artifact a SOC 2 auditor wants to see rather than a claim that controls exist. Macie scans S3 for sensitive data that's ended up somewhere it shouldn't, which carries more weight here because a misplaced document could be a client's protected health information. AWS Config continuously checks that live infrastructure still matches the rules it's supposed to follow, catching, for instance, a security group someone opened too widely during an incident and never closed back down. AWS Organizations Service Control Policies act as guardrails at the account level: no team can provision a public-facing database or disable CloudTrail, regardless of what IAM permissions an individual role happens to have.

CloudTrail runs organization-wide, logging management API calls across every account, plus data events for the buckets holding customer documents (data events are off by default), into a dedicated logging account, with S3 Object Lock in compliance mode making the log files themselves immutable, so nobody, including an AWS account administrator, can quietly edit the audit trail after the fact. Those logs, along with application-level audit events (who asked the model what, and what it answered), feed a central SIEM Virelane's security team watches, rather than sitting in a bucket nobody queries until an incident forces it.

Identity and access

Customer sign-in goes through Amazon Cognito user pools, which federate into each customer's own identity provider, Okta, Entra ID, Ping, over SAML or OIDC. Cognito has no SCIM endpoint, so Virelane's application (or an identity vendor in front of it) exposes SCIM for provisioning and deprovisioning, so an employee removed from the customer's directory loses Virelane access on the same day, not whenever someone remembers to do it manually. AWS IAM Identity Center covers Virelane's own engineers' access to AWS accounts. The account structure itself is a control: a multi-account landing zone (built on AWS Control Tower) gives security, logging, shared services, and each region's workloads their own account, so a permission granted in one account never implicitly reaches another. See identity and access management for the underlying model.

Observability

Distributed tracing (via OpenTelemetry, exported to a managed backend) follows a request from the edge through the service mesh to the model call and back. That matters more here than in earlier case studies because a slow response might mean an overloaded GPU node, a cross-region database call that shouldn't be happening, or a downstream customer API, and the on-call engineer needs to know which within minutes, not after reconstructing it from logs. SLOs are defined per tier (customer-facing API latency, model inference latency, batch job completion time) and alerts route to the team that owns the failing tier rather than a single undifferentiated pager. See observability.

Cost governance and FinOps

Every resource is tagged by customer, team, and environment, feeding Cost and Usage Reports that get rolled up into per-customer cost-to-serve numbers, the input FinOps needs to know whether a given account is profitable at its current usage pattern rather than at its contract value. Compute Savings Plans cover the predictable baseline; Spot Instances absorb interruptible batch and re-indexing work; S3 Intelligent-Tiering moves aged documents and logs to cheaper storage classes automatically rather than by manual lifecycle policy alone.

CI/CD and platform engineering

Deploys go through CodePipeline (or GitHub Actions federated via OIDC, avoiding long-lived AWS keys in CI), with mandatory code review, automated dependency and container scanning, and progressive rollout via Argo Rollouts, a canary that shifts a small percentage of traffic first and rolls back automatically on an SLO regression. Regulated customer segments add a change-approval step before production deploys, satisfying the change-management control their auditors expect without slowing down every other team's release cadence. See IaC and delivery.

Architecture: GCP

Most of this architecture mirrors the AWS design service for service: a managed Kubernetes control plane, a managed key-management service, a cloud-native SIEM. Two places diverge rather than relabel: the multi-region database, and the mechanism GCP offers for enforcing a security perimeter.

Virelane runs GKE Standard, not Autopilot, in each active region. Autopilot is a reasonable default for a team that wants Google to manage node provisioning entirely, but Virelane's platform team needs control over node pool composition (a tainted GPU pool, a Graviton-equivalent Tau T2A pool for stateless services) and custom network policies that Standard mode exposes more directly. GKE's node auto-provisioning handles the same elastic GPU scaling Karpenter handles on AWS: spin up an inference node pool for a batch re-indexing job, scale it back to zero once it's done. Namespaces, resource quotas, and network policies split traffic by product vertical the same way they do on EKS. That part of the design doesn't change, because the underlying problem (many independent teams, one shared substrate) isn't cloud-specific.

flowchart TB
  U[Customer traffic] --> GLB[Cloud Load Balancing: global anycast]
  GLB --> R1[us-central1: GKE + Spanner]
  GLB --> R2[europe-west1: GKE + Spanner]
  R1 <-.->|Spanner multi-region
quorum replication| R2
  R1 --> V1["Gemini Enterprise Agent Platform
(formerly Vertex AI):
regional endpoint"]
  R2 --> V2["Gemini Enterprise Agent Platform
(formerly Vertex AI):
EU regional endpoint"]
  

Cloud Load Balancing's global anycast frontend is a structural advantage here: a single global IP address routes to whichever region's GKE cluster is healthiest and closest, with no separate accelerator product needed on top of the load balancer the way AWS layers Global Accelerator in front of Route 53. Cloud DNS handles domain resolution behind it. As on AWS, each regional cluster serves its own customers end to end without depending on a live cross-region call.

Inside each region:

flowchart TB
  IN[Customer traffic via
Cloud Load Balancing
or Cloud Interconnect] --> GKE
  IDP[Identity Platform
federation with
customer IdP] --> GKE
  CICD[Cloud Build or
GitHub Actions, scans,
canary rollout] -->|deploy| GKE
  GKE[GKE Standard inside a
VPC Service Controls perimeter
service mesh mTLS
namespace per vertical
auto-provisioned GPU pools]
  GKE -->|Private Service
Connect| VTX["Gemini Enterprise
Agent Platform
(formerly Vertex AI)
Model Garden +
dedicated endpoints"]
  GKE --> SP[(Spanner
accounts, permissions)]
  GKE --> FS[(Firestore
sessions, flags)]
  GKE --> BQ[(BigQuery
analytics, eval logs,
billing export)]
  GKE -->|audit events,
traces| OPS[Cloud KMS CMEK /
Cloud EKM keys
Cloud Audit Logs, SIEM
Security Command Center
Cloud Trace + Monitoring]
  BQ ~~~ OPS
  SP ~~~ VTX
  

The multi-region database: where GCP earns its keep

On AWS, Virelane accepted a single write region for its relational tier because multi-region writes with strong consistency aren't what Aurora Global Database is built for. (Aurora DSQL, AWS's newer distributed SQL service, does offer active-active multi-region writes with strong consistency, though with a narrower PostgreSQL feature set.) On GCP, that trade-off doesn't have to be made. Cloud Spanner is a globally distributed relational database with synchronous replication and external consistency across regions. Clients in any region can submit writes to the same row; Spanner routes them to the leader region and commits them with a quorum of replicas, so the database resolves ordering and the application doesn't have to work around eventual consistency. Writes from far from the leader pay the cross-region round trip. Spanner has no two-region US and EU configuration: spanning the Atlantic means a three-continent multi-region configuration such as nam-eur-asia1 (leader in Iowa, read-only replicas in Belgium and Taiwan), which places replicas in Asia and is another residency question for EU customers.

That capability isn't free. Spanner costs more per node than a comparable managed Postgres instance, and its query model, while SQL-like, differs from vanilla Postgres enough to require some application changes. For Virelane it's worth it because the customer-facing tier (accounts, permissions, workflow state) is the workload that benefits from true multi-region consistency: a permission change made by an admin in one region should be immediately visible to a request served from another, and reasoning about eventual consistency for a security-relevant field like a permission grant is a risk rather than an inconvenience. Firestore handles the naturally partitioned session and feature-flag data that doesn't need cross-row transactions, the role DynamoDB Global Tables plays on AWS, except that a Firestore multi-region location stays within the US or within Europe, so cross-continent replication needs a per-continent database. Hot counters such as rate limits suit a different store, because Firestore limits sustained writes to a single document. Analytical and batch data lands in BigQuery, which doubles as both the data warehouse and, via BigQuery ML and its native integration with the Gemini Enterprise Agent Platform (formerly Vertex AI), a natural home for model evaluation and usage analytics without a separate ETL hop.

Regional model serving and AI inference

The Gemini Enterprise Agent Platform serves both hosted foundation models (through Model Garden) and Virelane's own fine-tuned models through regional endpoints, mirroring the Bedrock-plus-SageMaker split above: general traffic goes to a managed foundation model in the customer's region, sensitive or fine-tuned workloads go to a dedicated endpoint that never leaves Virelane's own project boundary. GPU and TPU capacity for training and batch inference draws on committed-use discounts for the steady-state load and Spot VMs (the successor to preemptible VMs) for interruptible batch work, the same reserved-plus-burstable pattern as AWS's Reserved Instances and Spot. Provisioned Throughput on the Agent Platform reserves hosted-model capacity for the accounts with latency SLAs, as Bedrock Provisioned Throughput does on AWS.

Private networking and zero trust: VPC Service Controls

Private Service Connect gives services private access to Google-managed APIs (the Gemini Enterprise Agent Platform, Cloud KMS, BigQuery) without traversing the public internet, and Cloud Interconnect provides the dedicated private connectivity the largest customers require, the same role PrivateLink and Direct Connect play on AWS.

What GCP adds on top is VPC Service Controls: a managed perimeter around a group of projects that blocks Google API calls from moving data across it, even when the request carries valid credentials. Identity-based access control asks whether a principal has permission; the perimeter adds an independent check on where the request comes from and where the data is going. A correctly authenticated request from Virelane's own compliance team cannot copy customer data out of the perimeter around that customer's project unless the perimeter explicitly allows it. For a platform whose worst case is a legitimate-looking credential used to walk data out the front door, that is a stronger guarantee than identity alone. AWS and Azure can build the same control from parts: on AWS a data perimeter of VPC endpoint policies plus organization-wide SCPs and resource control policies, on Azure a Network Security Perimeter around PaaS resources. VPC Service Controls packages it as one product, while the AWS version is assembled from several policy types and takes more design work.

Encryption and key management

Cloud KMS with customer-managed encryption keys (CMEK) covers the same tiered model as AWS: default Google-managed encryption for most accounts, per-tenant keys the customer controls for the largest regulated ones. For the few customers who require it, Cloud External Key Manager (Cloud EKM) keeps the key material in a third-party key manager outside Google's infrastructure, and Google calls out to it per operation. That is the same guarantee AWS KMS gives through an external key store, and a stronger one than a key hosted in the provider's own HSM service. A few of Virelane's most regulated government and financial accounts require it.

Security operations and audit logging

Security Command Center Premium plays much of the role GuardDuty and Security Hub play together on AWS: threat detection and posture management against CIS benchmarks in one console, with sensitive-data discovery coming from Sensitive Data Protection. Cloud Audit Logs record administrative actions across every project (Admin Activity logs are always on; Data Access logs, off by default for most services, are enabled for the services holding customer data), routed to a centralized, access-controlled logging project whose log buckets carry a locked retention policy that prevents early deletion, the same immutability guarantee Object Lock provides on AWS. Those logs, plus application-level audit events, feed the same central SIEM referenced above; Virelane runs one security operations function across clouds rather than duplicating it per provider.

Identity and access

Customer sign-in goes through Identity Platform, which supports per-tenant SAML and OIDC federation with each customer's identity provider (Okta, Entra ID, Ping); as on AWS, SCIM provisioning is something Virelane's application or an identity vendor supplies. Workforce Identity Federation covers Virelane's own engineers, letting its corporate identity provider authenticate against Google Cloud without a separate set of Google credentials. A resource hierarchy of folders and projects (security, logging, shared services, one per region per environment) plays the same isolating role AWS Organizations' multi-account structure does, with Organization Policies acting as the guardrail layer equivalent to Service Control Policies.

Observability, cost governance, and delivery

Cloud Trace and Cloud Monitoring, both OpenTelemetry-compatible, provide the same distributed tracing and SLO-based alerting as the AWS design, routed the same way: by tier, to the team that owns it. Billing export to BigQuery, visualized in Looker Studio, gives the same per-customer, per-team cost allocation the AWS Cost and Usage Reports provide, and it lands in the warehouse already used for product analytics. Committed use discounts cover the predictable baseline, and Active Assist's rightsizing recommendations catch overprovisioned nodes and idle resources that tagging alone wouldn't surface. Cloud Build or GitHub Actions federated via Workload Identity Federation drive the same review-gate-scan-canary pipeline as the AWS design, deploying to GKE via the same progressive-rollout pattern, with one pipeline template reused by every product vertical instead of five.

Architecture: Azure

The structural shape of this design matches the AWS and GCP versions above: managed Kubernetes, a tiered key-management strategy, a security operations console tied to a SIEM. What differs here is identity. A large share of Virelane's regulated customers, banks, insurers, hospital systems, government agencies, already run their own workforce on Microsoft 365 and Entra ID. On Azure, federating with a customer's identity provider often means connecting to the same identity system the customer's own IT department already administers.

Virelane runs AKS in each active region, with node pools split by workload shape the same way as the EKS and GKE designs: a general-purpose pool for stateless services, a tainted GPU pool for inference, and cluster autoscaler policies that bring GPU nodes up for a batch job and back down once it finishes. As on the other two clouds, Kubernetes is here as the shared substrate multiple independent platform and product teams standardize on, rather than each solving scheduling and rollout separately.

flowchart TB
  U[Customer traffic] --> AFD[Azure Front Door + WAF]
  AFD --> R1[East US: AKS + Cosmos DB]
  AFD --> R2[West Europe: AKS + Cosmos DB]
  R1 <-.->|multi-region writes, tunable consistency| R2
  R1 --> AI1["Microsoft Foundry
(formerly Azure AI Foundry):
regional PTU deployment"]
  R2 --> AI2["Microsoft Foundry
(formerly Azure AI Foundry):
EU regional PTU deployment"]
  

Azure Front Door provides global anycast entry with an integrated WAF, routing to the nearest healthy region's AKS cluster the same way Cloud Load Balancing does on GCP. Each region again serves its own customers end to end, so a regional incident stays regional.

Inside each region:

flowchart TB
  IN[Customer traffic via
Front Door + WAF
or ExpressRoute] --> AKS
  IDP[Entra ID multi-tenant app
and External ID] --> AKS
  CICD[Azure DevOps or
GitHub Actions, scans,
canary rollout] -->|deploy| AKS
  AKS[AKS
service mesh mTLS
Entra workload identity
namespace per vertical
autoscaled GPU pool]
  AKS -->|Private Link| AI["Microsoft Foundry PTUs
(formerly Azure AI Foundry)
+ Azure ML endpoints"]
  AKS --> COS[(Cosmos DB
accounts, permissions)]
  AKS --> SQL[(Azure SQL
Hyperscale)]
  AKS --> SYN[(Synapse
analytics, eval logs)]
  AKS -->|audit events,
traces| OPS[Key Vault /
Managed HSM keys
Defender for Cloud
Sentinel SIEM
Monitor + App Insights]
  SYN ~~~ OPS
  COS ~~~ AI
  

Multi-region data strategy: Cosmos DB's tunable consistency

Azure Cosmos DB natively supports multi-region writes, every region can accept writes to the same logical database, with a choice of five consistency levels between strong (linearizable) and eventual. Strong consistency isn't available with multiple write regions, and Microsoft calls bounded staleness an anti-pattern there, so Virelane runs the customer-facing tier (accounts, permissions, workflow state) at session consistency: a client reads its own writes, and other clients and regions converge asynchronously with no staleness bound. That is a deliberately different point on the consistency spectrum than either the AWS design (a single write region on Aurora, avoiding the question) or the GCP design (Spanner's externally consistent writes at a higher cost per node). It suits a user's own actions, which are immediately visible to that user, and it costs less to run at scale than Spanner. It is weaker for the security-relevant fields discussed in the GCP section: a permission revoked in one region can be served as still granted in another until replication catches up, so the application has to check revocations at a source it can read consistently, or accept the lag. Where immediate cross-region visibility of a permission change is a hard requirement, that favors Spanner.

Azure SQL Hyperscale handles workloads that need a relational, single-writer model closer to Aurora's, and Azure Synapse Analytics serves as the analytical warehouse for usage analytics and model evaluation logs (Microsoft now directs new analytics projects to Microsoft Fabric, which would replace Synapse in a new build), replicated to a secondary region on a schedule rather than continuously, matching the active-passive treatment this tier gets in the AWS and GCP designs.

Regional model serving and AI inference

Microsoft Foundry (formerly Azure AI Foundry), with the Azure OpenAI Service as one of its model sources, serves general foundation-model traffic through regional deployments. For guaranteed throughput at predictable cost, important for Virelane's largest accounts, whose contracts specify a response-time SLA Virelane can't miss because a shared capacity pool got busy, Virelane provisions dedicated Provisioned Throughput Units (PTUs) rather than relying on shared pay-per-token capacity. Bedrock Provisioned Throughput and the Agent Platform's Provisioned Throughput fill the same role on the other two clouds. Fine-tuned or fully self-hosted models run on dedicated Azure Machine Learning endpoints, the same role SageMaker and the Gemini Enterprise Agent Platform's dedicated endpoints play above. GPU capacity for training and batch work follows the same reserved-plus-spot pattern: Azure Reservations for the steady-state baseline, Spot VMs for interruptible re-indexing jobs.

Private networking and zero trust

Azure Private Link gives services a private IP path to PaaS resources (Key Vault, Cosmos DB, Microsoft Foundry); public network access is then disabled on each resource, since a private endpoint alone doesn't close it. ExpressRoute provides the dedicated private connectivity Virelane's largest, most security-conscious accounts require, playing the same role PrivateLink and Direct Connect play on AWS and Private Service Connect and Cloud Interconnect play on GCP. Inside the cluster, mutual TLS between services and workload identities tied to Entra ID (rather than a static credential baked into an image) enforce the same zero-trust posture as the other two designs: a request is authenticated and authorized every time, not because of where it came from on the network.

Encryption and key management

Platform-managed encryption covers most accounts. Azure Key Vault holds the per-tenant keys of the larger regulated accounts, and Key Vault Managed HSM covers those requiring a dedicated, single-tenant HSM behind their key, Azure's equivalent of AWS CloudHSM. Unlike Cloud EKM or AWS KMS's external key store, the key material still lives inside Microsoft's infrastructure rather than an external key manager. Key Vault and Managed HSM both support rotation policies.

Security operations and audit logging

Microsoft Defender for Cloud provides posture management and threat detection across the workload, and Microsoft Sentinel serves as the SIEM. Azure's design has an adoption advantage here: many of Virelane's regulated customers already run Sentinel and Defender internally for their own security operations, so a shared vocabulary can shorten the security review that gates every large deal. Azure Monitor Activity Logs record control-plane actions, are kept only 90 days by default, and so are exported to storage with an immutability policy, feeding the same central audit trail referenced above.

Identity and access

Identity is where Azure's position differs. On AWS and GCP, signing in a customer's employees means configuring a SAML or OIDC trust between Virelane's user pool and each customer's identity provider. When the customer already runs Entra ID as its workforce directory, common among enterprises standardized on Microsoft 365, Virelane can register the application as a multi-tenant Entra app so those users sign in with their own tenant's credentials and conditional access policies, and Entra External ID or B2B collaboration covers customers that don't. Okta and Ping customers still need a federation per customer. The capability isn't unique to Azure; the vendor and a large share of the customer base share an identity platform, which removes a configuration step. Subscriptions (Azure's account-boundary equivalent) split by environment and region, with Azure Policy acting as the same guardrail layer AWS's Service Control Policies and GCP's Organization Policies provide.

Observability, cost governance, and delivery

Azure Monitor and Application Insights, both OpenTelemetry-compatible, provide the same distributed tracing and SLO-based alerting as the other two designs, routed by tier to the owning team. Azure Cost Management provides per-subscription and per-tag cost breakdowns with budget alerts and anomaly detection, as AWS Cost Explorer and Budgets and Google's Cloud Billing reports do for their clouds. Reserved Instances and Savings Plans cover the predictable baseline, and Azure Advisor's rightsizing recommendations catch overprovisioned resources. Azure DevOps or GitHub Actions federated via workload identity drive the same review-gate-scan-canary pipeline as the other two clouds, deploying to AKS through the same progressive-rollout pattern.

Product names, instance families, and GPU generations in the table below change often. Check each provider's current lineup before planning infrastructure from it.

Capability comparison

Kubernetes: EKS vs. GKE vs. AKS

All three run upstream Kubernetes, so the differences are operational. GKE offers a hands-off Autopilot mode. EKS leaves more node, networking, and add-on configuration to the team (EKS Auto Mode now manages much of it) and pairs with an AWS-heavy stack through IAM roles for service accounts. AKS integrates with Entra ID workload identity, which matters for a customer base that already administers its own Azure tenant. Virelane's choice follows the cloud that hosts the deployment, since each of the three connects most simply to that cloud's own data and AI services.

Architecture decision records

Decision

Adopt managed Kubernetes (EKS, GKE, or AKS, one per cloud Virelane deploys on) as the shared compute substrate, now that multiple independent platform and product teams need to run on it.

Why

One or two teams can run everything on managed serverless and container platforms without paying Kubernetes' operational tax. Virelane has platform, AI-serving, security, and several vertical product teams shipping independently, each with a different mix of long-running services, spiky APIs, and GPU batch work. A shared substrate turns scheduling, autoscaling, and rollout tooling into something solved once, instead of five times with five different failure modes.

Alternatives considered

• Stay on managed serverless/container platforms (Fargate, Cloud Run, Container Apps) per team: workable with a handful of teams, and it means each team reinvents deployment, secrets handling, and networking independently
• A single shared cluster with no per-team isolation: rejected because one team's resource spike would starve another's, with no clean boundary to contain it

Trade-off

Kubernetes is an ongoing operational cost: someone has to own upgrades, node pool design, and cluster security, and that's a standing team, not a one-time setup. Virelane accepts that cost because it's now smaller than the alternative cost of five teams each solving the same problem badly.

Revisit when

If the org ever consolidates back down to one or two teams running a narrow set of workloads, the case for a shared cluster substrate weakens and a simpler managed platform may again be cheaper to run.

Decision

Active-active multi-region for the customer-facing tier; active-passive for the analytical and batch tier.

Why

Not every dataset needs the same availability guarantee, and treating them identically means paying active-active costs for data that could tolerate a delay. Customer-facing requests (auth, permissions, live queries) fail visibly and immediately if a region goes down, and the largest contracts carry availability terms a single region can't meet, so they run active in more than one region. Analytics and batch exports are read later, by people, not by a waiting customer, so a delay during a regional incident is an acceptable degradation rather than an outage.

Alternatives considered

• Active-active everywhere: rejected because of the cost and complexity of continuous multi-region replication for data nobody needs synchronously
• Active-passive everywhere: rejected because it would mean a customer-facing outage every time the primary region has a problem, which the SLA commitments in Virelane's largest contracts don't allow

Trade-off

Running two consistency models means two operational playbooks and two sets of failover tests to maintain, instead of one.

Revisit when

If a specific analytics workload becomes something customers query live and expect current, it graduates to the active-active tier's guarantees, and its cost model along with it.

Decision

Customer-managed encryption keys for the largest regulated accounts; provider-managed keys for the rest.

Why

Keys the customer controls (held in the customer's own cloud account or an external key manager) let a customer revoke Virelane's access to their data unilaterally, by disabling the key, which is what the largest regulated accounts' own security policies require before they'll sign. Provider-managed keys are simpler to operate and already meet the compliance bar every other customer needs.

Alternatives considered

• Customer-managed keys for every account: rejected as unnecessary operational overhead (key rotation, access policy maintenance) for accounts whose contracts don't require it
• Provider-managed keys for every account: rejected because it would disqualify Virelane from the specific top-tier deals that require customer control over key material

Trade-off

Supporting both tiers means the platform has to handle a customer disabling their own key gracefully, degrading that tenant's access cleanly instead of the whole service failing unpredictably.

Revisit when

If customer-managed keys become a baseline expectation across the whole customer base rather than a top-tier differentiator, maintaining two tiers stops paying for itself.

Decision

Zero-trust service-to-service authentication (mutual TLS plus workload identity) instead of relying on network perimeter alone.

Why

A perimeter-only model assumes anything inside the VPC is trustworthy, which fails the moment any single service is compromised: the attacker then has the run of everything else inside the same network boundary. Authenticating every service-to-service call independently means a compromised pod can only do what its own identity is explicitly allowed to do, not everything reachable on the network.

Alternatives considered

• Perimeter security alone (VPC boundaries, security groups): sufficient at earlier stages, rejected here because the blast radius of one compromised service is now unacceptable

Trade-off

A service mesh and per-service identity add latency and operational surface area: certificate rotation, mesh upgrades, another thing that can misconfigure and break traffic.

Revisit when

Virelane doesn't revisit this downward; at this scale authentication only gets more granular.

Decision

Hybrid model serving: managed foundation models for general-purpose traffic, dedicated or self-hosted endpoints for the most sensitive customer workloads.

Why

Some customers' contracts, or their own regulators, won't allow their documents to reach a shared managed-model service at all, even one run by the same cloud provider. A dedicated endpoint running a fine-tuned open model keeps that data entirely inside Virelane's own account boundary. For every other customer, a managed foundation model is cheaper to run and gets model improvements from the provider without Virelane retraining anything.

Alternatives considered

• Managed foundation models only: rejected because it would disqualify Virelane from the segment of customers with the strictest data-handling requirements
• Self-hosted models only: rejected as an unnecessary infrastructure and fine-tuning burden for customers who don't require it

Trade-off

Running two serving paths means two sets of evaluation, monitoring, and upgrade processes, and a routing layer that has to get the decision right per request.

Revisit when

If self-hosted model quality and cost converge close enough to the managed option, collapsing back to a single serving path becomes worth the simplification.

Decision

Multi-account (AWS) / multi-project (GCP) / multi-subscription (Azure) landing zone segregation instead of one shared environment per cloud.

Why

A single shared account means a permission granted for one purpose is implicitly available everywhere else in that account. Separate accounts per function (security, logging, shared services) and per region/environment mean a mistake or compromise in one boundary doesn't automatically reach another.

Alternatives considered

• One account with fine-grained IAM policies only: rejected because IAM misconfigurations are common enough that a second, structural boundary is worth having

Trade-off

More accounts means more surface area to keep configured consistently, which is what a landing-zone tool (Control Tower, an organization policy hierarchy) exists to automate rather than leaving to manual setup.

Revisit when

This structure gets more valuable as the org grows, so Virelane has no planned scale at which it collapses back to fewer accounts.

Decision

Dedicated private connectivity (Direct Connect / Cloud Interconnect / ExpressRoute) provisioned only for the customers whose contracts require it, not universally.

Why

Dedicated circuits are expensive to provision and maintain per customer. Most customers are satisfied with TLS-encrypted traffic over the public internet to a WAF-protected edge; only the most security-conscious accounts specifically require a connection that never touches the public internet at all.

Alternatives considered

• Provision dedicated connectivity for every enterprise account by default: rejected as cost that the majority of customers neither need nor are willing to pay for through their contract

Trade-off

Maintaining two connectivity tiers means sales and solutions engineering have to know, per deal, which tier a prospective customer needs before contract terms are set.

Revisit when

If dedicated connectivity requests become common enough across the customer base that the per-customer provisioning overhead exceeds the cost of just offering it by default.

Decision

Cost allocation and chargeback built in as a first-class requirement from the start of this stage, not retrofitted after costs become a problem.

Why

Infrastructure spend is now large enough to move gross margin, and by the time cost inefficiency becomes visible in a finance review, the fix usually requires re-architecting something that would have been cheap to design correctly the first time. Tagging every resource by customer and team from day one makes per-account profitability a query, not a project.

Alternatives considered

• Add tagging and cost allocation later, once a specific cost problem forces the issue: rejected because retrofitting tags across an already-large resource footprint is itself a significant project, and the absence of the data delays noticing the problem in the first place

Trade-off

Enforcing tagging discipline across every team requires tooling (policies that reject untagged resources) and ongoing enforcement, which is itself a small platform investment.

Revisit when

Not a decision Virelane revisits; the tagging and allocation model only needs refinement (new dimensions, finer granularity) as the business grows, never removal.

Decision

Data residency enforced architecturally, through routing and storage region selection, rather than relying on contractual commitment alone.

Why

A contract that says data stays in the EU is only as good as the infrastructure that actually prevents it from leaving. Routing a customer's traffic to a region-pinned deployment, and configuring storage and model-serving endpoints so there is no code path that could write that customer's data outside the committed region, makes the commitment true by construction instead of true by policy.

Alternatives considered

• Contractual commitment plus manual review: rejected because it depends on every engineer, on every team, remembering a rule with no system enforcing it

Trade-off

Architectural enforcement means every new feature has to be built with residency-aware routing in mind from the start, which is more upfront design work than bolting a check on afterward.

Revisit when

If a customer's residency requirement changes (a jurisdiction adds a new region Virelane doesn't yet operate in), the response is adding that region to the enforced set, not loosening the enforcement itself.

FinOps and cost optimization

At earlier stages, cost optimization mostly meant avoiding overprovisioning. Here it's a discipline with its own tooling and its own owner, because inefficiency is measured in gross margin points that show up in a board deck.

• Committed-use discounts and reserved capacity cover the predictable baseline load, Compute Savings Plans, committed use discounts, or Reservations depending on the cloud, purchased against a forecast that's re-evaluated quarterly rather than set once and forgotten.
• Spot and preemptible capacity absorbs interruptible batch work: overnight re-indexing, the bulk re-embedding an embedding-model change forces, anything that can tolerate a restart mid-job.
• Autoscaling keeps GPU and general-compute node pools sized to actual load rather than provisioned for peak, with node provisioners (Karpenter and its equivalents) tearing down capacity as soon as a batch job finishes instead of leaving it idle.
• Storage tiering moves aged documents, logs, and audit exports to cheaper storage classes automatically once they cross an access-frequency threshold, rather than paying hot-tier prices for data nobody's queried in months.
• GPU utilization is tracked per model and per customer, because idle or underutilized GPU capacity is among the most expensive line items this architecture carries: a model serving requests at 30% utilization is paying for capacity that batching, request coalescing, or right-sized instance selection could recover.

Due diligence questions

Why does Kubernetes make sense here when the "does this startup need Kubernetes?" framework from earlier in this curriculum says to adopt it only when its conditions are met?

Because the condition that framework checks for, multiple independent teams needing a shared substrate, is now true. The framework says to adopt Kubernetes once the coordination problem it solves exists.

Why active-active for the customer-facing tier but not for analytics?

Because they fail differently. A customer-facing outage is visible immediately and breaches an SLA. A delayed analytics refresh is visible only to an internal analyst and is recoverable without customer impact.

Why not customer-managed keys for every account, if they're more secure?

They add operational overhead (rotation, access policy maintenance, customer-side key management) that most accounts' compliance requirements don't call for, in exchange for control those accounts never exercise. Applying it everywhere would be over-engineering for the majority of the customer base.

What does VPC Service Controls on GCP add over IAM-based access control?

IAM asks "does this identity have permission?" and grants access if the answer is yes. VPC Service Controls adds an independent question, "is this data allowed to cross this perimeter at all?", that a valid identity's permissions can't override. It covers a correctly permissioned request that is still the wrong thing to allow. AWS data perimeters (endpoint policies plus SCPs and resource control policies) and Azure Network Security Perimeters address the same case with different mechanisms.

Why run both a managed foundation model path and a self-hosted model path instead of picking one?

Because the two paths serve different requirements. Some customers' data can never reach shared third-party infrastructure; others are well served by a cheaper, provider-maintained model. Collapsing to one path would mean failing one of those two groups.

How would you explain RPO and RTO to someone outside engineering?

RPO is how much data you could afford to lose: the gap between the last successful backup or replication point and the moment of failure. RTO is how long you can afford to be down before recovering. A five-minute RPO means replication has to run continuously or every few minutes; a five-minute RTO usually means failover is automated, because a human following a runbook rarely finishes that fast.

Why does GPU utilization get tracked per customer instead of just per model?

Because per-model utilization can look healthy in aggregate while individual accounts are dramatically over- or under-served relative to what they're paying. Per-customer tracking is what feeds pricing and contract renewal decisions.

What would make you choose Cosmos DB's tunable consistency over Spanner's stronger guarantee, given the two solve a similar problem?

The workload's access pattern. Spanner is worth its cost when cross-region writes need strict ordering and every reader must see a change, such as a permission revocation, immediately. If a session-level guarantee (a user sees their own writes, everyone else converges later with no stated bound) covers the access pattern, paying for stronger guarantees than the workload needs is waste.

What's the single biggest architectural risk at this stage that didn't exist at Series A?

Coordination failure across independent teams, not any single technical component. A team standardizing on its own message queue, or building its own logging pipeline because the shared one didn't fit, doesn't show up as an incident. It shows up eighteen months later as a platform nobody can reason about end to end.

If you had unlimited budget, what would you build differently?

Very little. Unlimited budget doesn't remove the coordination problem, the regulatory obligations, or the consistency trade-offs above; it only makes the wasteful version of each decision affordable.

What becomes more important than developer velocity at this stage?

Earlier stages optimize for shipping fast: pre-seed to find out whether anyone wants the product, Series A to prove the sales motion repeats. Virelane has already won both arguments, and winning them changes what "good" means.

Resilience matters more than a fast feature launch, because a launch that takes down a regional deployment costs more in trust than the feature was worth. Governance, who can change what, and who reviewed it, matters more than a team's ability to ship without asking, because an ungoverned change is now a finding waiting for an auditor to notice it. Predictable operations matter more than a clever one-off optimization, because predictability is what lets hundreds of engineers reason about a system none of them holds in their head alone. Compliance stops being a checklist run before a deal closes and becomes a standing constraint every design has to satisfy continuously. And cost efficiency and organizational scalability, can the platform grow without linearly growing headcount and spend, become the metric investors and acquirers look at, replacing growth rate as the number that matters most.

Velocity still matters. It has simply stopped being the scarcest resource, and an architecture that still optimizes as though it were is optimizing for the wrong constraint.