展示HN：BlazeRules – 用于流数据的YAML规则引擎，支持每秒处理高达500万条记录
<a href="https://github.com/purijs/blazerules" rel="nofollow">https://github.com/purijs/blazerules</a><p>我最初想要在C++中制作一个亚毫秒级的日志解析器，但最终发展成了一个可嵌入的决策引擎，可以对传入的数据执行YAML定义的规则。规则通过先将数据重投影为列式格式（如果尚未如此）在传入数据上以向量化的方式执行。根据负载大小和规则复杂性，性能从每秒20万条记录提升到超过每秒百万条记录，吞吐量平均在200 MiB/s到3 GiB/s之间。<p>规则也可以是SQL表达式，或者是ONNX模型（数值型），支持窗口操作和其他多种操作。<p>它可与DuckDB相媲美，但更适用于流数据和实时决策。
查看原文
<a href="https://github.com/purijs/blazerules" rel="nofollow">https://github.com/purijs/blazerules</a><p>I initially wanted to make a sub-millisecond log parser in C++ but that blew into a embeddable decision engine, that can run YAML defined rules on incoming data.
The rules are executed in a vectorized format on incoming data by reprojecting into a columnar format first, if it's not already. Depending on the payload size and rules complexity, the performance goes from 200K records/s to more than million records/sec, in terms of througput this would be around 200 MiB/s to 3 GiB/s on average.<p>Rules can be sql expressions too, or onnx models (numeric), window ops and quite a few more operations are supported.<p>It's comparable to DuckDB but for streaming data and on the fly decisions.