Lead with the confirmed duplicate cost
Finance sees three charges for an inventory sync that should have run once. Begin the memo with the observed duplicate work, affected window, estimated and invoiced exposure, and business impact. Do not describe the calls as bots merely because they were automated; the known source is an internal background task.
Exclude the bot hypothesis explicitly
The classification labels Human, Bot, and Unknown are not applicable evidence for a scheduled job whose source and context are already known. Applying a visitor traffic block would miss the scheduler and could harm unrelated customers. The control belongs where concurrency is created.
Show the lock expiring while work continues
Lay out the job starts and finishes against the lock lifetime. If duration p95 is 11 minutes and the lock expires at five, a second scheduler can acquire the lock while the first job is still working. Three starts with the same job identity and overlapping execution windows establish an internal concurrency defect.
Verify runtime ownership rather than the configured TTL
Confirm the lock's actual semantics: acquisition must be atomic, ownership must be recorded, and release must not let one worker delete another worker's renewed lock. A configuration screenshot proves the intended TTL, not that the runtime lock behaved as designed.
| Memo field | Evidence | Example conclusion |
|---|---|---|
| Source/context | scheduler and job logs | inventory-sync / background |
| Runtime versus TTL | p95 duration and lock config | 11 min versus 5 min |
| Overlap | start/end timeline | three concurrent executions |
| External abuse indicators | edge and auth logs | no matching external trigger |
| Financial exposure | provider IDs and invoice | assigned amount, not estimate alone |
Name the control finance is approving
Propose a lock duration greater than a measured high percentile plus safety margin, or a lease that is renewed while the worker is healthy. Pair it with a stable job ID and deduplication at the paid-work boundary. Merely increasing the Cron interval can still overlap when a rare run exceeds it.
Add a stale-worker recovery rule so a crashed job does not hold the queue forever. Recovery should verify lease expiry and ownership, mark the old attempt abandoned, and reconcile any upstream request before starting a replacement. The memo should identify the engineering owner and due date.
Quantify avoided cost honestly
Calculate exposure from proven duplicate provider calls, not from all internal starts. If two overlapping workers exited before reaching the provider, they consumed compute time but not model usage. Separate confirmed invoice cost, estimated pending cost, and internal labor.
Forecast savings using observed frequency and duplicate usage, then label assumptions. A one-time incident does not support claiming the same saving every month. Finance should see the maximum credible exposure, central estimate, and evidence that will replace the estimate at invoice close.
Close with a test a non-engineer can verify
Run a test longer than the old five-minute TTL while triggering the scheduler again. Acceptance requires one active owner, no second provider request, one committed result, and a visible alert if the lease cannot renew. Attach the IDs and timeline rather than asking finance to trust a code change.
The final one-page memo should contain five blocks: symptom and amount, causal timeline, excluded bot hypothesis, approved control, and acceptance result. Link to the blog for deeper operational examples; keep this document short enough to support an actual approval.
Use the one-page cause-and-control memo from “Explain overlapping Cron locks to finance without calling every duplicate a bot” on a real first installation. Download AI Cost Circuit Breaker for free, begin in Monitoring, and move to enforcement only after the expected signals and rollback are verified.