0/6个主要航空航天文档门户已准备好使用人工智能代理。
我花了一些时间构建了AeroScore,这是一个评估主要文档门户网站对AI代理支持程度的评分引擎。简而言之,我在第一批评估中所评估的6个网站都没有做好充分准备。
该方法和标准基于AFDocs规范(一个针对代理准备文档的开源检查清单),并针对PDF使用量进行了特定扩展,因为航空航天领域对PDF的重视程度很高。每个网站在多个标准上得分,范围从0到100,包括llms.txt的存在、robots.txt等,详细信息可以在AFDocs仓库和AeroScore网页上找到。
主要发现:
- 0/6个网站有llms.txt
- 0/6个网站支持URL变体
- 最高得分为ICAO,得分为69/100
主要航空航天文档门户网站尚未为AI代理做好准备,这一点非常重要,因为这意味着该领域的效率没有得到充分发挥。AI代理可以在预期时间的几分之一内分析深度文档门户中的数据,这在许多领域都带来了革命性的变化。航空航天领域缺乏这一点意味着生产力的损失。
排行榜已上线,网址为:https://aeroscore.vercel.app
AFDocs仓库地址为:https://github.com/agent-ecosystem/afdocs
我是一名即将入学的本科航空航天工程专业学生,关注AI在我所在领域的应用及其潜力。这是AeroScore的第一个版本,尚不完善,我非常希望能对方法论得到反馈。在未来的版本中,我将使用改进的评分标准评估更多网站。
查看原文
I spent some time building AeroScore, a scoring engine that evaluates how well major documentation portals support AI agents. In short, none of the 6 that I evaluated in this first batch are sufficiently prepared.<p>The methodology and criteria are based on the AFDocs Spec (an open-source checklist for agent-ready documentation) and one specific extension for PDF-heaviness since aerospace as a field places so much importance on PDFs. Each site gets scored 0-100 across criteria like llms.txt presence, robots.txt, and many others that can be found both on the AFDocs repo and at the aeroscore webpage.<p>Key Findings:<p>-0/6 have an llms.txt<p>-0/6 support url variants<p>-the best score was ICAO with a 69/100<p>The fact that major aerospace documentation portals are not AI agent ready yet is important because it means the field is not being fully efficient. AI agents allow the analysis of data across deep documentation portals to occur in fractions of the expected time, and this has revolutionized many fields. The lack of this in aerospace means a loss of productivity.<p>The leaderboard is live at: https://aeroscore.vercel.app<p>The AFDocs repo is at: https://github.com/agent-ecosystem/afdocs<p>I'm an incoming undergraduate aerospace engineering student interested in how AI is and can be implemented in my field. This is v1 of aeroscore, it's pretty rough and I'd appreciate feedback on the methodology greatly. In future versions, I'll be scoring more sites with a refurbished rubric.