我建立了一个小实验,以检测人工智能编码助手何时出现偏差。
我最近在使用Cursor编程时遇到了一个常见问题。<p>最初,我们通常会给AI一些明确的约束,例如:<p>不修改数据库架构<p>不修改某些API<p>只允许修改前端逻辑<p>某些函数名称不能更改<p>在对话开始时,AI通常能够很好地遵循这些规则。<p>然而,随着对话的延长,比如达到40,000或50,000个标记,一个常见问题出现了:<p>AI逐渐“忘记”之前提到的限制。<p>例如:<p>最开始你说“不要修改数据库,”<p>但经过几轮对话后,它突然建议:<p>“我们可以通过修改数据库结构来解决这个问题。”<p>我之前问过其他人如何解决这个问题,有人建议:<p>在项目中写一个important.md或规则文件,以便AI每次都能读取。<p>这个方法确实有一定效果,但在实际开发中仍然会出现问题。<p>例如:<p>最开始你说“不要碰数据库A,”<p>但后来项目中添加了数据库B。<p>如果你没有及时更新markdown文件,AI可能会意外修改你不想更改的内容。<p>因此,我最近创建了一个小型实验工具,主要是为了应对AI编程助手中的“约束漂移”问题。
查看原文
I recently encountered a common problem while coding with Cursor.<p>Initially, we usually give the AI some explicit constraints, such as:<p>Don't modify the database schema<p>Don't modify certain APIs<p>Only allow modifications to the front-end logic<p>Some function names cannot be changed<p>At the beginning of the conversation, the AI generally follows these rules well.<p>However, as the conversation lengthens, for example, to 40,000 or 50,000 tokens, a common problem arises:<p>The AI gradually "forgets" the previously mentioned restrictions.<p>For example:<p>Initially you say "Don't modify the database,"<p>but after a few rounds, it suddenly suggests:<p>"We can solve this problem by modifying the database structure."<p>I've asked others how to solve this before, and some suggested:<p>Write an important.md or rule file in the project so the AI reads it every time.<p>This method does have some effect, but problems still arise in actual development.<p>For example:
Initially you say "Don't touch database A,"<p>but later database B is added to the project.<p>If you don't update the markdown file in time, the AI might accidentally modify things you didn't intend to change.<p>So I recently created a small experimental tool, mainly to solve the "constraint drift" problem in AI programming assistants.