应用层能否提高本地模型的输出质量？
你好，<p>我正在构建一个终端原生的代码生成工具，最近的一个更新是为那些不想将代码上传到第三方服务器的用户打包了一个本地模型（Qwen 2.5 Coder 7B，首次下载即可使用）。<p>用户对这一新增功能的初步反馈是积极的，但我对此有些怀疑：这个模型相对基础，质量与在线产品相比还有差距。<p>因此，我计划改进RAG（检索增强生成）能力，以便构建包含相关源文件片段的消息，增加规划调用，添加验证循环，可能还会进行多样本重排序等：这些都是常见的技术，如果实施得当，可以提高输出质量。<p>所以，我的问题是：我相信（希望？）通过实施这些改进，7B模型的质量可以大致提升到20B模型的水平，你认为这是可能的，还是觉得这将是徒劳无功，这种改进不会发生？<p>源代码在这里，如果你喜欢，可以给它一个星标： https://github.com/acrotron/aye-chat
查看原文
Hi -<p>I am building a terminal-native tool for code generation, and one of the recent updates was to package a local model (Qwen 2.5 Coder 7B, downloads on the first try) for those users who do not want their code uploaded to third-party servers.<p>Initial response from users to this addition was favorable - but I have my doubts: the model is fairly basic and does not compare in quality to online offerings.<p>So - I am planning to improve RAG capabilities for building a message with relevant source file chunks, add a planning call, add validation loop, maybe have a multi-sample with re-ranking, etc.: all those techniques that are common and when implemented properly - could improve quality of output.<p>So - the question: I believe (hope?) that with all those things implemented - 7B can be bumped approximately to quality of a 20B, do you agree that's possible or do you think it would be a wasted effort and that kind of improvement would not happen?<p>The source is here - give it a star if you like what you see: https://github.com/acrotron/aye-chat