Writing
What I have learned running this, written down before I forget it.
Most of these started as something that broke, or as an argument I kept having. I write them down because the second time I hit the same problem I would rather read than re-derive.
-
Running the models yourself
Sending every prompt to someone else's GPU is a decision, not a default. What it takes to serve models on hardware you own, how to know whether they are actually good, and what has to be true before an agent touches production.
-
One login instead of twelve
Every internal service wants its own user database. Running an identity provider means none of them get one — and offboarding becomes a single action instead of an archaeology project.
-
An overlay mesh, and the DNS trap underneath it
An overlay mesh gives every machine a stable address and an encrypted path to every other one. It also gives your internal names a way to silently resolve somewhere else.
-
Alert on symptoms, not on metrics
Most monitoring failures are not missing data. They are the wrong alert on good data — and the one class of outage nobody catches is the one where every metric is green.
-
Ceph without a vendor's price list
Ceph gives you replicated storage across ordinary machines instead of one expensive box with a support contract. It also gives you a set of failure modes that only appear when you are already unhappy.
-
Kubernetes, and when it earns its place
Kubernetes is not the next step up from Docker Compose — it is a trade, and most people are told about the benefits without being told the price. Here is what it actually bought, and what it cost.
-
A host you cannot rebuild is not running
Infrastructure as code is not a tool choice, it is a rule about where truth lives. The rule only works if you never break it — and the temptation to break it always arrives at three in the morning.
-
OpenTaberna, and writing the wiki first
An open-source ordering and fulfilment system built as four repositories with a published wiki — and the thing I would take to every project after it: the documentation is a repository, reviewed like code.
There is an Atom feed if you would rather not check back, and the full text of everything here is in one file at /llms-full.txt.