The Heartbeat Pattern: The Cheapest Reliability Fix for Background Jobs

If you have background jobs like cron jobs, queues, or worker threads, they do not always fail loudly. Sometimes they just stop. AI will generate your background process code, add logging, and maybe even retries. But it will not automatically address the most important question: how do I know if this process is still alive?

The Problem

Crashes are obvious. They throw errors, trigger alerts, and show up in logs. Silent failures are the real danger. A worker stops processing, a cron job hangs, a queue consumer dies quietly. Nothing throws an error. No one notices until the downstream effects become visible, and by then the damage is done.

The Pattern

The fix is a heartbeat. Every long-running job writes a timestamp somewhere durable. It could be every minute, every five minutes, whatever interval you expect. A separate process checks those timestamps. If a timestamp is missing or stale, that is a failure.

At that point, fire an alert: send an email, send a text, or wake someone up with a service like PagerDuty.

Why AI Misses This

AI code generation tools will produce your background process code. They will add logging and, if you are lucky, retries. But they will not add heartbeat monitoring because they are focused on what the process does, not on how you verify it is still running. This is the kind of operational judgment that still needs a human to specify.

Key Takeaways

• Background jobs often fail silently rather than crashing with an error
• The heartbeat pattern is simple: each job writes a timestamp at regular intervals, and a separate process checks for staleness
• AI will generate your job code with logging and retries but will not add heartbeat monitoring unless you ask for it
• This pattern is cheap to implement and prevents the kind of silent failures that compound into real outages

Resources

• prodblueprint.dev -- The creator's library of production-ready prompts and patterns