Skip to content

Step 7 of 7 in the DevOps journey · see the full journey

2025

Cloud Cartographer

A migration study costing the same application on AWS and Azure.

Overview

I measured the sample application under three local load scenarios, then used the observations to build AWS and Azure migration estimates. A later review checked the source, calculator exports and arithmetic before treating any projected savings as evidence.

context
Coursework
status
Archived learning project
role
Solo
started
Nov 2025
setting
Local container load tests and an AWS/Azure migration study; calculator estimates without a live cloud deployment or spend.
built with
  • Python
  • Locust 2.42
  • Prometheus 3.0
  • Loki 3.2
  • Grafana
  • docker-stats-exporter
  • +2

Key decisions

Tie measurements to each test window
A Python harness drives Locust and queries Prometheus over the run's start and end timestamps, keeping the recorded observations connected to the workload.
Keep calculator exports as reproducible inputs
Saved AWS and Azure estimates provide the costing inputs. They are model outputs, not bills or evidence of a completed migration.
Check explanations against the application source
The later review checked the email index and persistent database pool rather than treating missing indexes or pooling as established causes.

Demonstrated results

Comparable load scenarios and resource observations
Runs at 10, 50 and 100 users produced recorded resource series. The database CPU values did not show proportional growth with the user count.
The cost audit found concrete discrepancies
The report and its cited calculator totals differ by $180.07 for AWS and $82.33 for Azure. The review also corrected egress units and identified capped log queries.

Known limitations

Database CPU remains unexplained
The recorded 3.40 vCPU P95 at 100 users was not traced to a confirmed cause. Linear extrapolation from it remains a sizing assumption.
Savings are conditional targets
The lower post-tuning cost depends on an unverified database-sizing hypothesis. There was no live cloud deployment or measured budget reduction.
Log-volume and workload coverage are limited
Two runs reached Loki's 5,000-entry query cap, and the traffic was a narrow registration/login/session workflow rather than a representative application workload.

The finding

At 100 concurrent users, the recorded PostgreSQL CPU P95 exceeded the backend's, while the monitoring stack also accounted for substantial CPU:

Component P95 vCPU P95 RAM
Go backend 0.73 31 MB
React frontend ~0.00 10 MB
PostgreSQL 3.40 272 MB
Monitoring stack 3.16 821 MB

The load profile is very light. Each simulated user registers once, logs in once, then issues a single GET /api/session on a one-to-five second wait. The whole 100-user test generated about 6.8 MB of backend egress.

Measurement workflow

The harness runs headless Locust scenarios at 10, 50 and 100 users for 10, 10 and 15 minutes. It queries Prometheus over each run's exact time window with a 15-second step and computes percentiles in Python. Loki supplies log-query results.

When cAdvisor did not report the needed metrics in this setup, I tried a separate Ubuntu VM to investigate the environment, then used docker-stats-exporter as the replacement. The change provided a measurement path; it does not by itself validate every exported counter or explain the database anomaly.

What the numbers do that the report missed

The P95 across all three scenarios is 3.92, 3.94, 3.40 vCPU for 10, 50 and 100 users.

These observations do not show proportional CPU growth with the user count. That is a reason to investigate background work and the measurement path before using a linear extrapolation, not proof of a specific root cause.

The remaining hypotheses include database configuration, exporter artefacts and background maintenance such as autovacuum or checkpointing.

It also raises a question the report does not answer: if database CPU does not scale with load, what exactly was being sized by the linear ×10 extrapolation below?

Ruling two hypotheses out by reading the source

The original analysis listed four. Two were dismissed by inspecting the application rather than by inference:

  • Missing index on users.email: the column is declared TEXT UNIQUE NOT NULL, which creates a unique index. The login and session lookup path is indexed.
  • No connection pooling: sql.Open("pgx", ...) returns a persistent *sql.DB pool, and no SetMaxOpenConns override is set. One or two active connections does not, by itself, establish that a persistent pool is missing.

That leaves configuration, measurement, and background maintenance. The write-up states the conclusion plainly:

The root cause therefore remains unidentified.

The cost consequence

AWS Azure
Calculator export total $9,240.09 $10,936.86
Document's stated total $9,420.16 $11,019.19
Post-optimisation target ~$2,104.27 ~$2,034.46

One database line is almost the entire bill. The concentration depends entirely on which total you divide by, so here are all four:

Database Share of export Share of document
AWS $8,530.68 92.32% 90.56%
Azure $9,792.50 89.54% 88.87%

The database line is identical in both sources, so the only thing that changes is the denominator. It changes the answer by nearly two points on AWS. The project page previously claimed "over 92%" AWS and "over 90%" Azure. The first is true only against the export; the second is true of neither.

The totals do not reconcile

3-Migration-Cost-Analysis.md cites the calculator CSVs as its source, and its own columns add up correctly: $889.48 plus $8,530.68 is $9,420.16 exactly. But summing the CSV line items gives $9,240.09, so the document's platform subtotal is $180.07 higher than anything in the file it points at. Azure does the same at $82.33.

The gap is entirely in the platform figure. The database line matches to the cent in both clouds, so this is not rounding, and it is not a stale export. It is a subtotal that was never reconciled against its stated source. The document does acknowledge the discrepancy once, in a parenthesis, when computing the 92.3% figure against the calculator total. Everywhere else it uses $9,420.16.

An instance at $8,530.68/month, sized from a number nobody can yet explain.

The recorded decision was to size conservatively from that value and flag the uncertainty as the leading migration risk. This remains a planning assumption; diagnosing the anomaly is necessary before treating the sizing as validated.

The audit

The later review corrected several figures and separated projections from measured observations:

Figure Before After Why
Monitoring stack total 2.89 vCPU / 735 MB 3.16 vCPU / 821 MB The earlier total omitted docker-stats-exporter, the very tool installed to replace cAdvisor
Conditional target difference ~$3,000–4,000/mo $7,113.68/mo on the database, ~$7,315.89/mo overall Arithmetic between estimates, conditional on the unverified tuning target; not realised savings
Cost concentration "over 75%" AWS over 92% Arithmetic on the real line items
Egress ~587 GB/month ~196 GB/month at 1,000 users 0.0068 × 86400 = 587.52 is the per-day figure, mislabelled as monthly, and it omitted the ×10 user scaling the text claimed
Log volume ~1.5 GB/month at 100 users unverified The 50- and 100-user runs both returned exactly 5,000 entries, hitting Loki's query limit. Only the 10-user run is unsaturated
Comparison baseline "0.3–0.5 vCPU typical" unsourced rule of thumb No benchmark or citation exists in the project. The direction of the finding holds; the specific magnitude does not

The egress unit error is the instructive one. A total of 0.0068 GB over a 15-minute test was treated as a per-second rate, multiplied by the seconds in a day, and labelled a month. The ×10 user scaling claimed in the sentence above was never applied. The corrected figure, ~196 GB/month at 1,000 users, remains a model assumption rather than measured monthly transfer.

The log-volume correction matters more, because it was silently wrong in the optimistic direction. Both the 50- and 100-user runs saturated at the query cap, so logs_per_minute was an artefact of each test's duration: 498/min over 10 minutes against 333/min over 15. The apparent drop under higher load was purely the longer window. The true 100-user volume is higher than reported.

What this stage does not claim

  • That the anomaly was solved. It was not, and the documentation says so.
  • That ~$2,104/month is a budget. It is a hypothesis contingent on tuning delivering an 8 vCPU database, which is contingent on a cause nobody has found.
  • That 587 GB/month of egress. That figure is wrong and is corrected above.
  • That the load was representative. It is a registration-and-login workflow, and the report says so rather than implying otherwise.

Interactive models

Explore the model below to inspect the implementation. This is a reconstruction from the project, not a live service.

Cloud estimate comparison and conditional tuning target

CostModelChart

Platform against database, from the calculator exports.

  • aws · on-demand$9,240.09

    platform $709.41 · database $8530.68 · 92.32%

    bar width relative to $10936.86

  • aws · one-year-reserved$9,061.24

    platform $530.56 · database $8530.68 · 94.14%

    bar width relative to $10936.86

  • azure · on-demand$10,936.86

    platform $1144.36 · database $9792.50 · 89.54%

    bar width relative to $10936.86

  • azure · one-year-reserved$10,721.56

    platform $929.06 · database $9792.50 · 91.33%

    bar width relative to $10936.86

  • Cyan is the database. Every bar is the same shape, because the instance was sized from one unexplained measurement in all four cases.
src/content/data/costs.ts · COST_EXPORTS

client:visible

Reconstruction of the calculator exports. The post-optimisation bar is the hypothesis, not a measurement.
Stack and source figures

how it fits together

load harness
  • Locust 2.42 Headless registration-and-login workflow driving real traffic at three user levels.
measurement
  • Prometheus 3.0 + Loki 3.2 + Grafana P95 extracted from the Prometheus API over the exact test window. Log volume is marked unverified because the 50- and 100-user runs both saturated at Loki's 5,000-entry query cap.
  • docker-stats-exporter Replaces cAdvisor, which cannot read kernel counters through Docker on WSL2.
costing
  • AWS Pricing Calculator + Azure calculator exports Saved calculator estimates make the cost inputs inspectable. Comparing them with the report exposed a platform subtotal discrepancy; they are not records of actual cloud spend.
analysis
  • Python The analysis scripts behind the P95 series and the egress unit correction.
DB CPU P95
3.40 vCPU At 100 users. Average 2.78, max 4.07. The max was deliberately not extrapolated.
AWS total
$9,240.09 /mo The calculator export's own total. The cost-analysis document states $9,420.16, which is $180.07 higher and reconciles to nothing in the file it cites.
database share
92.32% $8,530.68 of $9,240.09. Against the document's $9,420.16 the same line is 90.56%, so the headline share moves by two points depending on the denominator.
post-tuning
~$2,104 /mo A hypothesis, not a committed budget reduction. The root cause is unidentified.
load scenarios
3 10, 50 and 100 users over 10, 10 and 15 minutes.
commits
49
Full issue record · 4 entries

what broke, and what happened

  • 2 fixed / 4
  • 2 unresolved / 4

Every entry below says what broke and what became of it. An unresolved one is listed as unresolved rather than dropped.

  1. fixed

    cAdvisor would not report container metrics at all in this environment

    Docker on WSL2 isolates containers in a lightweight VM, which prevents cAdvisor from reading kernel-level statistics. A pristine Ubuntu VM was provisioned with Vagrant purely to isolate the cause, and cAdvisor still failed.

    then: Replaced rather than debugged: `docker-stats-exporter` on port 9487 is now the scrape target in `monitoring/prometheus.yml`, which collects from the Docker socket instead of kernel counters.

  2. unresolved

    PostgreSQL consumed far more CPU than the workload could explain

    P95 3.40 vCPU at 100 users while the entire test generated only about 6.8 MB of backend egress.

    then: Never explained. The measurement stands as recorded, and sizing conservatively on it is the decision: a 48-vCPU instance priced from a number nobody can account for.

  3. fixed

    Two of the four hypotheses were ruled out by reading the application’s own source

    `email` is declared `TEXT UNIQUE NOT NULL`, and `sql.Open("pgx", ...)` returns a persistent pool. These contradict the hypotheses of an absent email index or absent pool; they do not establish the CPU anomaly's cause.

    then: Both were eliminated on inspection rather than by experiment, so two of the four hypotheses were dropped without a reproduction.

  4. unresolved

    The root cause remains unidentified

    What survives: default container database configuration, a measurement artefact in the exporter, and background maintenance overhead such as autovacuum and checkpointing.

    then: Still unidentified, and carried forward as the number one post-migration risk rather than closed with a guess.