Step 7 of 7 in the DevOps journey · see the full journey
Cloud Cartographer
A migration study costing the same application on AWS and Azure.
Overview
I measured the sample application under three local load scenarios, then used the observations to build AWS and Azure migration estimates. A later review checked the source, calculator exports and arithmetic before treating any projected savings as evidence.
- context
- Coursework
- status
- Archived learning project
- role
- Solo
- started
- Nov 2025
- setting
- Local container load tests and an AWS/Azure migration study; calculator estimates without a live cloud deployment or spend.
- Python
- Locust 2.42
- Prometheus 3.0
- Loki 3.2
- Grafana
- docker-stats-exporter
- +2
Key decisions
- Tie measurements to each test window
- A Python harness drives Locust and queries Prometheus over the run's start and end timestamps, keeping the recorded observations connected to the workload.
- Keep calculator exports as reproducible inputs
- Saved AWS and Azure estimates provide the costing inputs. They are model outputs, not bills or evidence of a completed migration.
- Check explanations against the application source
- The later review checked the email index and persistent database pool rather than treating missing indexes or pooling as established causes.
Demonstrated results
- Comparable load scenarios and resource observations
- Runs at 10, 50 and 100 users produced recorded resource series. The database CPU values did not show proportional growth with the user count.
- The cost audit found concrete discrepancies
- The report and its cited calculator totals differ by $180.07 for AWS and $82.33 for Azure. The review also corrected egress units and identified capped log queries.
Known limitations
- Database CPU remains unexplained
- The recorded 3.40 vCPU P95 at 100 users was not traced to a confirmed cause. Linear extrapolation from it remains a sizing assumption.
- Savings are conditional targets
- The lower post-tuning cost depends on an unverified database-sizing hypothesis. There was no live cloud deployment or measured budget reduction.
- Log-volume and workload coverage are limited
- Two runs reached Loki's 5,000-entry query cap, and the traffic was a narrow registration/login/session workflow rather than a representative application workload.
The finding
At 100 concurrent users, the recorded PostgreSQL CPU P95 exceeded the backend's, while the monitoring stack also accounted for substantial CPU:
| Component | P95 vCPU | P95 RAM |
|---|---|---|
| Go backend | 0.73 | 31 MB |
| React frontend | ~0.00 | 10 MB |
| PostgreSQL | 3.40 | 272 MB |
| Monitoring stack | 3.16 | 821 MB |
The load profile is very light. Each simulated user registers once, logs in once,
then issues a single GET /api/session on a one-to-five second wait. The whole
100-user test generated about 6.8 MB of backend egress.
Measurement workflow
The harness runs headless Locust scenarios at 10, 50 and 100 users for 10, 10 and 15 minutes. It queries Prometheus over each run's exact time window with a 15-second step and computes percentiles in Python. Loki supplies log-query results.
When cAdvisor did not report the needed metrics in this setup, I tried a separate
Ubuntu VM to investigate the environment, then used docker-stats-exporter as the
replacement. The change provided a measurement path; it does not by itself validate
every exported counter or explain the database anomaly.
What the numbers do that the report missed
The P95 across all three scenarios is 3.92, 3.94, 3.40 vCPU for 10, 50 and 100 users.
These observations do not show proportional CPU growth with the user count. That is a reason to investigate background work and the measurement path before using a linear extrapolation, not proof of a specific root cause.
The remaining hypotheses include database configuration, exporter artefacts and background maintenance such as autovacuum or checkpointing.
It also raises a question the report does not answer: if database CPU does not scale with load, what exactly was being sized by the linear ×10 extrapolation below?
Ruling two hypotheses out by reading the source
The original analysis listed four. Two were dismissed by inspecting the application rather than by inference:
- Missing index on
users.email: the column is declaredTEXT UNIQUE NOT NULL, which creates a unique index. The login and session lookup path is indexed. - No connection pooling:
sql.Open("pgx", ...)returns a persistent*sql.DBpool, and noSetMaxOpenConnsoverride is set. One or two active connections does not, by itself, establish that a persistent pool is missing.
That leaves configuration, measurement, and background maintenance. The write-up states the conclusion plainly:
The root cause therefore remains unidentified.
The cost consequence
| AWS | Azure | |
|---|---|---|
| Calculator export total | $9,240.09 | $10,936.86 |
| Document's stated total | $9,420.16 | $11,019.19 |
| Post-optimisation target | ~$2,104.27 | ~$2,034.46 |
One database line is almost the entire bill. The concentration depends entirely on which total you divide by, so here are all four:
| Database | Share of export | Share of document | |
|---|---|---|---|
| AWS | $8,530.68 | 92.32% | 90.56% |
| Azure | $9,792.50 | 89.54% | 88.87% |
The database line is identical in both sources, so the only thing that changes is the denominator. It changes the answer by nearly two points on AWS. The project page previously claimed "over 92%" AWS and "over 90%" Azure. The first is true only against the export; the second is true of neither.
The totals do not reconcile
3-Migration-Cost-Analysis.md cites the calculator CSVs as its source, and its
own columns add up correctly: $889.48 plus $8,530.68 is $9,420.16 exactly. But
summing the CSV line items gives $9,240.09, so the document's platform subtotal is
$180.07 higher than anything in the file it points at. Azure does the same at
$82.33.
The gap is entirely in the platform figure. The database line matches to the cent in both clouds, so this is not rounding, and it is not a stale export. It is a subtotal that was never reconciled against its stated source. The document does acknowledge the discrepancy once, in a parenthesis, when computing the 92.3% figure against the calculator total. Everywhere else it uses $9,420.16.
An instance at $8,530.68/month, sized from a number nobody can yet explain.
The recorded decision was to size conservatively from that value and flag the uncertainty as the leading migration risk. This remains a planning assumption; diagnosing the anomaly is necessary before treating the sizing as validated.
The audit
The later review corrected several figures and separated projections from measured observations:
| Figure | Before | After | Why |
|---|---|---|---|
| Monitoring stack total | 2.89 vCPU / 735 MB | 3.16 vCPU / 821 MB | The earlier total omitted docker-stats-exporter, the very tool installed to replace cAdvisor |
| Conditional target difference | ~$3,000–4,000/mo | $7,113.68/mo on the database, ~$7,315.89/mo overall | Arithmetic between estimates, conditional on the unverified tuning target; not realised savings |
| Cost concentration | "over 75%" AWS | over 92% | Arithmetic on the real line items |
| Egress | ~587 GB/month | ~196 GB/month at 1,000 users | 0.0068 × 86400 = 587.52 is the per-day figure, mislabelled as monthly, and it omitted the ×10 user scaling the text claimed |
| Log volume | ~1.5 GB/month at 100 users | unverified | The 50- and 100-user runs both returned exactly 5,000 entries, hitting Loki's query limit. Only the 10-user run is unsaturated |
| Comparison baseline | "0.3–0.5 vCPU typical" | unsourced rule of thumb | No benchmark or citation exists in the project. The direction of the finding holds; the specific magnitude does not |
The egress unit error is the instructive one. A total of 0.0068 GB over a 15-minute test was treated as a per-second rate, multiplied by the seconds in a day, and labelled a month. The ×10 user scaling claimed in the sentence above was never applied. The corrected figure, ~196 GB/month at 1,000 users, remains a model assumption rather than measured monthly transfer.
The log-volume correction matters more, because it was silently wrong in the
optimistic direction. Both the 50- and 100-user runs saturated at the query cap,
so logs_per_minute was an artefact of each test's duration: 498/min over 10
minutes against 333/min over 15. The apparent drop under higher load was purely
the longer window. The true 100-user volume is higher than reported.
What this stage does not claim
- That the anomaly was solved. It was not, and the documentation says so.
- That ~$2,104/month is a budget. It is a hypothesis contingent on tuning delivering an 8 vCPU database, which is contingent on a cause nobody has found.
- That 587 GB/month of egress. That figure is wrong and is corrected above.
- That the load was representative. It is a registration-and-login workflow, and the report says so rather than implying otherwise.
Interactive models
Explore the model below to inspect the implementation. This is a reconstruction from the project, not a live service.
Cloud estimate comparison and conditional tuning target
CostModelChart
Platform against database, from the calculator exports.
- aws · on-demand$9,240.09
platform $709.41 · database $8530.68 · 92.32%
bar width relative to $10936.86
- aws · one-year-reserved$9,061.24
platform $530.56 · database $8530.68 · 94.14%
bar width relative to $10936.86
- azure · on-demand$10,936.86
platform $1144.36 · database $9792.50 · 89.54%
bar width relative to $10936.86
- azure · one-year-reserved$10,721.56
platform $929.06 · database $9792.50 · 91.33%
bar width relative to $10936.86
- Cyan is the database. Every bar is the same shape, because the instance was sized from one unexplained measurement in all four cases.
client:visible
Stack and source figures
how it fits together
- load harness
-
-
Locust 2.42Headless registration-and-login workflow driving real traffic at three user levels.
-
- measurement
-
-
Prometheus 3.0 + Loki 3.2 + GrafanaP95 extracted from the Prometheus API over the exact test window. Log volume is marked unverified because the 50- and 100-user runs both saturated at Loki's 5,000-entry query cap. -
docker-stats-exporterReplaces cAdvisor, which cannot read kernel counters through Docker on WSL2.
-
- costing
-
-
AWS Pricing Calculator + Azure calculator exportsSaved calculator estimates make the cost inputs inspectable. Comparing them with the report exposed a platform subtotal discrepancy; they are not records of actual cloud spend.
-
- analysis
-
-
PythonThe analysis scripts behind the P95 series and the egress unit correction.
-
- DB CPU P95
- 3.40 vCPU At 100 users. Average 2.78, max 4.07. The max was deliberately not extrapolated.
- AWS total
- $9,240.09 /mo The calculator export's own total. The cost-analysis document states $9,420.16, which is $180.07 higher and reconciles to nothing in the file it cites.
- database share
- 92.32% $8,530.68 of $9,240.09. Against the document's $9,420.16 the same line is 90.56%, so the headline share moves by two points depending on the denominator.
- post-tuning
- ~$2,104 /mo A hypothesis, not a committed budget reduction. The root cause is unidentified.
- load scenarios
- 3 10, 50 and 100 users over 10, 10 and 15 minutes.
- commits
- 49
Full issue record · 4 entries
what broke, and what happened
- 2 fixed / 4
- 2 unresolved / 4
Every entry below says what broke and what became of it. An unresolved one is listed as unresolved rather than dropped.
- fixed
cAdvisor would not report container metrics at all in this environment
Docker on WSL2 isolates containers in a lightweight VM, which prevents cAdvisor from reading kernel-level statistics. A pristine Ubuntu VM was provisioned with Vagrant purely to isolate the cause, and cAdvisor still failed.
then: Replaced rather than debugged: `docker-stats-exporter` on port 9487 is now the scrape target in `monitoring/prometheus.yml`, which collects from the Docker socket instead of kernel counters.
- unresolved
PostgreSQL consumed far more CPU than the workload could explain
P95 3.40 vCPU at 100 users while the entire test generated only about 6.8 MB of backend egress.
then: Never explained. The measurement stands as recorded, and sizing conservatively on it is the decision: a 48-vCPU instance priced from a number nobody can account for.
- fixed
Two of the four hypotheses were ruled out by reading the application’s own source
`email` is declared `TEXT UNIQUE NOT NULL`, and `sql.Open("pgx", ...)` returns a persistent pool. These contradict the hypotheses of an absent email index or absent pool; they do not establish the CPU anomaly's cause.
then: Both were eliminated on inspection rather than by experiment, so two of the four hypotheses were dropped without a reproduction.
- unresolved
The root cause remains unidentified
What survives: default container database configuration, a measurement artefact in the exporter, and background maintenance overhead such as autovacuum and checkpointing.
then: Still unidentified, and carried forward as the number one post-migration risk rather than closed with a guess.