StillMe – 一个开源的“透明RAG”框架

1作者: Anh_Nguyen_vn大约 2 个月前原帖
我正在构建 *StillMe*,一个开源的“透明RAG”框架,努力做到不假装自己知道一切。 与其仅仅提示一个大型语言模型(LLM)并寄希望于结果,StillMe 采取了以下措施: - 通过一个 *多层验证链* 处理所有答案(6个核心验证器 + 条件验证器,根据上下文最多可达13个) - 自动修复缺失的引用和虚构的“经验” - 记录 *系统级步骤*(RAG检索、验证器、时间分解) - 将“我不知道”视为一种 *一流的、诚实的状态*,并进行明确的认识追踪 --- ## StillMe 实际上做了什么 对于每个用户查询,StillMe: 1. 检测意图(哲学、技术或事实) 2. 相应地路由和过滤 RAG 上下文 3. 构建安全提示(考虑到令牌和语言) 4. 调用底层的 LLM(本地或云端) 5. 运行 *验证器链*: - `CitationRequired` → 添加 `[基础知识]` 或真实的网络/RAG 引用 - `EvidenceOverlap` → 检查答案与检索到的上下文(仅在上下文可用时) - `Ego-Neutrality` → 移除拟人化语言(“我感觉”,“我的经验”等) - `SourceConsensus` → 检测多个来源之间的矛盾(仅在有2个以上来源时) - `EthicsAdapter` → 避免不安全的建议,同时保持诚实 6. 记录结构化时间: - RAG 检索延迟 - LLM 推理延迟 - 验证与后处理 --- ## 真实的日志摘录(单一哲学问题) ```log StillMe 哲学查询跟踪(真实后端日志摘录) [INFO] 检测到哲学问题 — 正在过滤技术 RAG 文档 [INFO] 检索到3个基础知识文档(RAG缓存命中) [WARNING] 估计令牌超过安全限制 — 切换到最小哲学提示 [WARNING] 检测到缺失引用 — 自动修复为 [基础知识] [WARNING] Ego-Neutrality 验证器移除了拟人化术语:[‘trải nghiệm’] --- 延迟 --- RAG: 3.30s | LLM: 5.41s | 总计: 12.04s ``` --- ## 我为什么要构建这个 如今大多数 LLM 系统: - 隐藏它们的推理过程 - 草率处理引用 - 夸大信心 - 将“我不知道”视为失败 而 StillMe 则走向相反的方向: - *透明优先*:每个重大决策都有记录 - *认识诚实*:允许(并鼓励)说“我不知道” - *模型无关*:与本地和云端 LLM(DeepSeek、OpenAI、Ollama)兼容 - *无需微调*:所有行为在框架层面强制执行 --- ## 寻求反馈与贡献者 我是一名来自越南的独立开发者。StillMe 已经: - 作为后端 + 仪表板运行 - 集成了真实的学习管道(RSS、arXiv、维基百科 - 每4小时更新一次) - 使用了一个实时的 RAG 系统和基础文档 仓库链接:[https://github.com/anhmtk/StillMe-Learning-AI-System-RAG-Foundation](https://github.com/anhmtk/StillMe-Learning-AI-System-RAG-Foundation) 我希望能得到关于以下方面的反馈: - 验证器架构 - 更好的日志结构和可观察性 - 使项目更友好于贡献者 - 任何可以压力测试“诚实/透明”声明的想法 感谢阅读 — 如果有人感兴趣,我很乐意回答问题并分享更多日志、图表或内部信息。
查看原文
I&#x27;m building *StillMe*, an open-source &quot;transparent RAG&quot; framework that tries very hard to <i>not</i> pretend it knows everything.<p>Instead of just prompting an LLM and hoping for the best, StillMe:<p>- Runs all answers through a *multi-layer validator chain* (6 core validators + conditional validators, up to 13 total depending on context) - Auto-fixes missing citations and hallucinated &quot;experience&quot; - Logs *system-level steps* (RAG retrieval, validators, timing breakdown) - Treats &quot;I don&#x27;t know&quot; as a <i>first-class, honest state</i> with explicit epistemic tracking<p>---<p>## What StillMe actually does<p>For each user query, StillMe:<p>1. Detects intent (philosophical vs. technical vs. factual) 2. Routes and filters RAG context accordingly 3. Builds a safe prompt (token-aware, language-aware) 4. Calls the underlying LLM (local or cloud) 5. Runs a *ValidatorChain*: - `CitationRequired` → adds `[foundational knowledge]` or real web&#x2F;RAG citations - `EvidenceOverlap` → checks answer vs. retrieved context (only when context available) - `Ego-Neutrality` → removes anthropomorphic language (&quot;I feel&quot;, &quot;my experience&quot;, etc.) - `SourceConsensus` → detects contradictions between multiple sources (only when 2+ sources available) - `EthicsAdapter` → avoids unsafe suggestions while staying honest 6. Logs structured timing: - RAG retrieval latency - LLM inference latency - Validation &amp; post-processing<p>---<p>## A real log excerpt (single philosophical question)<p>```log StillMe philosophical query trace (real backend log excerpt)<p>[INFO] Philosophical question detected — filtering out technical RAG docs [INFO] Retrieved 3 foundational knowledge documents (RAG cache HIT) [WARNING] Estimated tokens exceed safe limit — switching to minimal philosophical prompt [WARNING] Missing citation detected — auto-patched with [foundational knowledge] [WARNING] Ego-Neutrality Validator removed anthropomorphic term: [&#x27;trải nghiệm&#x27;] --- LATENCY --- RAG: 3.30s | LLM: 5.41s | Total: 12.04s ```<p>---<p>## Why I&#x27;m building this<p>Most LLM systems today:<p>- Hide their reasoning - Hand-wave citations - Overstate confidence - Treat &quot;I don&#x27;t know&quot; as a failure<p>StillMe goes the other way:<p>- *Transparency-first*: every major decision is logged - *Epistemic honesty*: it&#x27;s allowed (and encouraged) to say &quot;I don&#x27;t know&quot; - *Model-agnostic*: works with local and cloud LLMs (DeepSeek, OpenAI, Ollama) - *No fine-tuning required*: all behavior is enforced at the framework layer<p>---<p>## Looking for feedback &amp; contributors<p>I&#x27;m a solo builder from Vietnam. StillMe is already:<p>- Running as a backend + dashboard - Integrated with a real learning pipeline (RSS, arXiv, Wikipedia - updates every 4 hours) - Using a live RAG system with foundational docs<p>Repo: https:&#x2F;&#x2F;github.com&#x2F;anhmtk&#x2F;StillMe-Learning-AI-System-RAG-Foundation<p>I would love feedback on:<p>- The validator architecture - Better ways to structure logs &amp; observability - Making the project more contributor-friendly - Any ideas to stress-test the &quot;honesty &#x2F; transparency&quot; claims<p>Thanks for reading — happy to answer questions and share more logs, diagrams, or internals if anyone&#x27;s curious.