PhilippTheServer

testing

Tests that fail when the thing they describe breaks, including for infrastructure.

  1. Building a Headless Storefront Against the OpenTaberna API

    A headless storefront for OpenTaberna: a 37-line API client, TypeScript types generated from OpenAPI, and a compiler that catches field typos.

    frontend api-design testing

  2. Traefik's 60-Second readTimeout Covers the Whole Request Body

    "Expected filesize of 104857600 bytes but read 29491200": Traefik's readTimeout defaults to 60s and bounds the whole request body, so slow uploads get cut.

    kubernetes networking reliability testing

  3. Multicast Service Discovery on a Host With a WireGuard Interface

    "Required key not available" (ENOKEY) on multicast sendto: kernel WireGuard has no peer for 239.0.0.0/8. Filter interfaces on IFF_MULTICAST and IFF_POINTOPOINT.

    networking linux cpp embedded testing

  4. Documentation as a Repository: Publishing a Wiki.js Site from Reviewed Markdown

    Wiki.js Git storage for docs as code: publish reviewed Markdown and fail a pull request when src/ changes without docs/, checked with git diff.

    documentation testing ci-cd architecture

  5. Testing Infrastructure Code by Executing Its Real Expressions

    Ansible Templar in pytest: render the shipped combine(recursive=True) expression from defaults/main.yml instead of a shallow Python paraphrase.

    testing ansible ci-cd infrastructure-as-code

  6. Keeping Documentation Honest with an OpenAPI Snapshot Diff

    FastAPI app.openapi() snapshot test in pytest: fail CI when the live OpenAPI schema drifts from the committed openapi.snapshot.json.

    documentation testing ci-cd

  7. Self-Hosted LLM Inference: Serving, Benchmarking and Agent Guardrails

    Ollama OpenAI-compatible /v1/chat/completions: a repeatable benchmark on your own tasks and a dry-run plus --approve guardrail for infrastructure agents.

    llm agents testing security

  8. Verifying an Agent's Work Against Reality, Not Its Own Report

    git diff --quiet HEAD and a re-run test command in a separate CI job: verifying a coding agent's work from repository state instead of its own report.

    agents testing ci-cd

  9. Testing an Agent Harness Without Ever Calling the Model

    Unit testing a coding agent's tool-call permission layer with pytest and recorded fixtures: allow, deny or ask decisions with no model or API key.

    testing agents python

  10. Testing That a Canvas Diagram Actually Painted Pixels

    getImageData in Playwright to assert a canvas actually drew pixels, since jsdom has no canvas renderer and DOM assertions pass on a blank canvas.

    testing frontend

  11. DNS Wildcards and Empty Non-Terminals: The Answer That Is Not an Error

    A stale _acme-challenge TXT record makes a DNS wildcard return NODATA or NXDOMAIN: RFC 4592 closest encloser, empty non-terminals and a zone check.

    dns tls testing

  12. Separating 'Cannot Ever' From 'Currently Broken' in Sweep Automation

    Sweep automation: a 404 cannot tell "never supported" from "currently broken". An explicit cannot-ever allowlist with required reasons, and its tests.

    automation testing api-design observability

  13. A DNS Canary CronJob as a Regression Test for Split-Horizon Resolution

    CoreDNS Corefile rewrite regressions caught by a busybox nslookup CronJob that checks internal, external and NXDOMAIN names and fails the Job.

    kubernetes dns observability testing

  14. Sync Waves and the CRD-Before-Consumer Ordering Problem

    no matches for kind "Widget" in version: CRD and CR in one Argo CD sync. sync-wave "-1" fixes ordering; Prune=false stops a CRD prune wiping CRs.

    kubernetes gitops testing

  15. Alerting on Symptoms Instead of Metrics: Designing Alerts People Do Not Ignore

    probe_success == 0 via blackbox_exporter instead of CPU alerts, plus a dnspython check that treats NOERROR with an empty answer as down.

    observability alerting reliability testing

  16. A Multi-Service docker-compose That a New Contributor Can Actually Start

    depends_on condition: service_healthy: why a plain depends_on list races, with Postgres, Redis and Keycloak healthchecks, --import-realm and a smoke test.

    docker identity testing

  17. A 500 That Was a Misconfigured Client, Not an Auth Failure

    resp.json()["active"] raises KeyError when token introspection returns 401 invalid_client. Check the status code first and report a client config error.

    identity api-design testing

  18. Building the Dry-Run Path First, and Testing That It Sends Nothing

    if dry_run: return guards drift as side effects are added. Split the tool into a pure plan() and apply(), and test that dry run never calls the client.

    testing python automation infrastructure-as-code

  19. Scoring Tool Calls by Parsed Structure Instead of String Equality

    Scoring LLM tool calls: parse the arguments JSON first so key order and 5 vs 5.0 pass, wrong tools fail, and extra fields follow per-case rules.

    llm testing

  20. Why a Leaderboard Score Does Not Predict Your Workload

    LLM leaderboard scores do not predict a narrow workload. A 20 to 50 case task eval with a deterministic JSON schema scorer instead of an LLM judge.

    testing llm

  21. Making a Benchmark Deterministic Against a Server Tuned for Interactive Use

    LLM output still varies at temperature 0 with a fixed seed because batching changes float order. Repeat each prompt and compare mean and stdev.

    testing llm

  22. UUID Case Normalization as a Recurring Cross-Service Bug

    Uppercase vs lowercase UUID keys cause silent cache and database lookup misses between services. Normalise once at the boundary with str(uuid.UUID(value)).

    reliability architecture python testing

  23. A Layered Storage Benchmark: Isolating Device, Network and Protocol Bottlenecks

    fio, iperf3, rados bench, then fio on RBD: benchmarking device, network, Ceph protocol and full stack separately to find which layer caps throughput.

    ceph storage testing performance

  24. chmod -R and the setgid Bit: Why a Four-Digit Mode Is Not Enough

    chmod -R 775 clears the setgid bit of 2775 directories, so new files lose the shared group. Separate find -type d and -type f, and assert with stat -c %a.

    linux docker testing

  25. Ansible Role Idempotence: Why a changed=0 Run Is the Only Proof You Have

    Molecule "Idempotence test failed because of the following tasks": why command, shell and timestamped templates report changed, and how to fix it.

    ansible infrastructure-as-code testing

All topics · All writing