Skip to content
Latest

The platform watches its own cloud spend

The platform watches its own cloud spend.

What changed for people using Bellwether

  • The platform watches its own cloud spend. A daily check now reads how much building and testing the platform did in the last day, how many builds were started, whether any serving instance is idling at a cost and how many old images are kept, compares each against a fixed threshold, records a verdict, and fails loudly when it cannot read or cannot record one. A separate hourly alarm tells the owner when builds are being started far faster than normal, which is how an earlier runaway went unnoticed for days.
  • Cheaper by default, same safety. The heavy test runs after a merge and overnight moved to a smaller machine with their steps ordered to fit it; record-keeping changes no longer start a build at all; the overnight run happens twice a week instead of nightly; old images are pruned to the last ten plus a week; the console and Consult scale to zero when nobody is using them. None of this touches how orders are checked or placed.
  • What production already does is now written down as code. Several of these settings were first changed by hand during the incident; the infrastructure definitions now say exactly the same thing, so a later apply keeps them instead of quietly putting the old values back. The same goes for two monitoring fixes from the day before: the alert on the desk's job queue no longer counts healthy attempts as failures, and the desk service is sized from both of its queues so a busy hourly pass is not turned away.

What you may notice

  • The first request to the console or to Consult after a quiet spell can take a few seconds longer: both now start on demand.
  • The readiness page's cloud-spend row reads red until the daily check has run once after this release; that is expected and clears on its first run.

Known gaps / coming next

  • The change to what the desk reads before it trades (one blend number in its table and its signal tool, one 52-week high everywhere, a completed day's reading that does not move) was ready but is held back one release: it met a second, overlapping change that shipped first and the two are being reconciled.
  • The daily spend check, the image clean-up and the storm alarm's adoption into the infrastructure state are switched on by the owner right after this lands, in a fixed order with stop conditions.
  • A monthly read of the bill by line item and a quarterly infrastructure audit are scheduled work, not automation.

Release 0450936