Your Free AI Server Will Fail Quietly. Five Gates to Make It Loud.

Your Free AI Server Will Fail Quietly. Five Gates to Make It Loud.

The model can be innocent. The server cannot.

Earlier this week I wrote a fail-closed checklist for AI-generated code. That list guards against the model writing something dangerous. This list guards against something duller: the server around it dying at 2 a.m. while the model stays online the whole time.

Nobody sees that failure until a user does.

The setup

I am testing MonkeyCode for a small side build: a log-summarizing API. The project gives you free model access and a free server option, which is exactly the toy setup I like. Ten lines of app logic. Zero dollars. One honest problem: free infrastructure is someone else’s best effort.

Disclosure: This article was prepared as part of MonkeyCode’s product outreach.

Before you judge, my plan was simple. I deliberately killed my own server to see where the stack would fail. Then I wrote gates that make each failure loud.

The kill test

Here is the failure sequence, reproduced on purpose.

1. The server process died. No restart policy.
2. Connections hit a dead socket. Nothing answered.
3. The client had no timeout and waited forever.
4. No health probe. No alert. No log line.
5. Four hours later, the model was still happy. The server was still dead. The tool was still broken.

The model was innocent the whole time. The harness was the guilty one. The problem was never intelligence. It was silence.

So here are five gates, ordered from cheapest to most annoying.

Gate 1: A kill switch that outlives the process

A crash bug can take down your app. It can also take down your ability to disable the app. So the switch lives outside the app.

KILL_FILE = "/tmp/disable-monkeycode"

@app.post("/summarize")
def summarize(logs: str):
    if os.path.exists(KILL_FILE):
        raise HTTPException(503, "disabled by operator")
    ...

Why a file and not a database row? Because the DB may be down when you need the switch most. A file survives restarts. You can touch it from cron. You can remove it by hand.

touch /tmp/disable-monkeycode   # fail closed
rm /tmp/disable-monkeycode      # reopen

Gate 2: A budget counter you control

Free endpoints rarely warn before they stop you. Sometimes they just return errors. So count every request yourself and stop early.

def spend(estimated_tokens: int):
    state = load_state()
    if state["tokens"] + estimated_tokens > MAX_TOKENS_PER_DAY:
        raise GateError("budget exhausted")
    state["tokens"] += estimated_tokens
    save_state(state)

The counter resets daily, lives in a JSON file, and blocks before the provider does. When in doubt, fail on the conservative side.

Gate 3: A timeout shorter than your patience

A hung call is worse than a failed call.

response = requests.post(MODEL_URL, json={"text": logs}, timeout=8)

Eight seconds. Then a clean 504. To be fair, this gate would not have saved the dead-socket outage above. It saves the other outage, the one where the endpoint hangs instead of dying. Free endpoints hang. It is practically their hobby.

Gate 4: An output canary that checks the reviewer

The model is your reviewer: it reads logs and writes a summary. Somebody has to review the reviewer.

data = response.json()
if not isinstance(data.get("summary"), str):
    return {"summary": "degraded: bad shape", "ok": False}

The exact shape check will vary. The principle will not: garbage must fail loudly, not flow downstream.

Gate 5: A health probe with a witness

A server that runs is not alive. A server that answers /healthz is.

@app.get("/healthz")
def healthz():
    return {"ok": True}

Now point a free uptime checker at it. Every minute, it calls this path. If the box dies, you get an email. This is Gate 5 because the first four already cost you one outage.

Decision table

Copy the table. Adjust the numbers to your risk appetite.

The five-minute deploy

On a free server, the whole ceremony looks like this.

git clone  && cd app
pip install fastapi uvicorn requests
touch /tmp/disable-monkeycode
uvicorn app:app --host 0.0.0.0 --port 8080 &
curl -i localhost:8080/summarize   # expect 503
rm /tmp/disable-monkeycode
curl -s localhost:8080/healthz     # expect {"ok":true}

Test the fail-closed path before you test the happy path. The happy path is only happy by accident.

Limitations

The JSON budget file is not atomic. Two simultaneous requests can race and overspend. For a solo tool, that is fine. For anything bigger, move the counter to SQLite.

The example API has no auth. That means anyone can spend your free tokens. Ten minutes of work: require a header token.

And none of these gates create a SLA. Free infrastructure is best-effort by definition. The goal is loud failure, not false confidence.

Who should skip this

Teams with compliance requirements. Products with paying customers. Anyone whose uptime appears in a contract. For you, this checklist is the minimum, not the gold standard.

Try it

If you want to break these gates on purpose, MonkeyCode’s free model access plus the free server option make a cheap lab. I broke mine within an hour. That hour taught me more than a week of reading docs.

Which gate did your last free-tier outage skip first? That is the failure field I am missing for the next version of this checklist.