Build -
We shipped the demo with a retired model, and fixed it the same day
The live triage demo went out calling an AI model that had already been retired. Here is what went wrong and what we changed.
When the live AI Test Triage demo first went out, it answered every request with 'Could not reach the AI right now.' The cause was one line: it called an AI model that Anthropic had retired back in February.
The site itself looked fine. The page loaded and the error message was polite. The failure only showed up in the server logs, and nothing alerted us. We caught it the same day it went out, switched to the current model, and the demo has worked since.
Three things we are keeping from this. A friendly fallback message can hide a real failure. 'Latest' model names can still be retired, so check retirement dates before shipping. And test the deployed demo, not just the code.
Written by Rushi Prajapati