Many people reading this will call bullshit on the performance improvement metrics, and honestly, fair. I too thought the agents would stumble in hilarious ways trying, but they did not. To demonstrate that I am not bullshitting, I also decided to release a more simple Rust-with-Python-bindings project today: nndex, an in-memory vector “store” that is designed to retrieve the exact nearest neighbors as fast as possible (and has fast approximate NN too), and is now available open-sourced on GitHub. This leverages the dot product which is one of the simplest matrix ops and is therefore heavily optimized by existing libraries such as Python’s numpy…and yet after a few optimization passes, it tied numpy even though numpy leverages BLAS libraries for maximum mathematical performance. Naturally, I instructed Opus to also add support for BLAS with more optimization passes and it now is 1-5x numpy’s speed in the single-query case and much faster with batch prediction. 3 It’s so fast that even though I also added GPU support for testing, it’s mostly ineffective below 100k rows due to the GPU dispatch overhead being greater than the actual retrieval speed.
Opens in a new window
。业内人士推荐safew官方版本下载作为进阶阅读
// Send result back to the model
想象一下，你是一位想要构建带有 AI 代理的移动或 Web 应用的开发者。当智能体部署在云端时，你会面临一系列问题：