代理模拟 = 人工智能的单元测试吗？
在传统软件中，我们编写单元测试以在问题到达用户之前捕捉回归错误。然而，在人工智能系统中——尤其是那些具有自主性的系统，这种模型就会失效。你可以测试输入和输出，使用评估工具，但代理在时间、工具、主控程序、API以及不可预测的用户输入中操作。失败模式并不明显，往往只在边缘案例中显现。我看到一种新兴的实践：代理模拟——结构化、可重复的场景，用于测试人工智能代理在复杂或长尾情况下的行为。
想想看：
如果上游工具在执行过程中失败怎么办？
如果用户在对话中改变意图怎么办？
如果代理的假设微妙地错误怎么办？
从自动驾驶汽车到人工智能代理，这些问题并不是一次性的测试。它们就像自动驾驶汽车的模拟：在受控环境中探索失败边界。自动驾驶汽车团队早已意识到，现实世界的数据是不够的。最稀有的事件往往是最重要的——你需要系统地生成并重播这些事件。相同的长尾分布也适用于大型语言模型代理。我们已经开始将场景测试视为开发循环的核心部分——对模拟进行版本控制，在持续集成中运行它们，并随着代理行为的变化而不断演进。这并不是关于完美覆盖，而是关于从“事后测试”转变为“通过模拟测试”，作为迭代代理开发的一部分。
我很好奇这里是否有其他人正在做类似的事情。你们是如何在几个提示和指标之外测试你们的代理的？我很想听听HN社区对代理可靠性和安全性的看法——不仅仅是在研究中，而是在实际部署中。
查看原文
In traditional software, we write unit tests to catch regressions before they reach users. In AI systems—especially agentic ones that model breaks down. You can test inputs and outputs, use evals, but agents operate over time, across tools, mcps, apis, and unpredictable user input. The failure modes are non-obvious and often emerge only in edge cases. I'm seeing an emerging practice: agent simulations—structured, repeatable scenarios that test how an AI agent behaves in complex or long-tail situations.<p>Think:
What if the upstream tool fails mid-execution?
What if the user flips intent mid-dialogue?
What if the agent’s assumptions were subtly wrong?<p>from self-driving cars to AI agents?
The above aren’t one-off tests. They’re like AV simulations: controlled environments to explore failure boundaries. Autonomous vehicle teams learned long ago that real-world data isn't enough. The rarest events are the most important—and you need to generate and replay them systematically. That same long-tail distribution applies to LLM agents. We’ve started treating scenario testing as a core part of the dev loop—versioning simulations, running them in CI, and evolving them as our agent behavior changes. It’s not about perfect coverage,it’s about shifting from “test after” to “test through simulation” as part of iterative agent development.
Curious if others here are doing something similar. How are you testing your agents beyond a few prompts and metrics? Would love to hear how the HN crowd is thinking about agent reliability and safety—not just in research, but in real-world deployments.