Browsing Tag
LLM
168 posts
vLLM vs Ollama: Production Serving 2026
Compare vLLM and Ollama for LLM serving in 2026 — architecture, verified performance under concurrency, and a decision…
RAG vs. Direct Context: I Tested Both on Real Documents, Here’s What Broke
A hands-on test of BGE-M3 + Qwen3 (RAG vs. direct-context answering) on a real research paper and a…
瀏覽器跑 LLM 的實際體感與骨感限制
直接在瀏覽器跑 LLM，既兼顧隱私又不用複雜的GPU設定，太完美了吧？ 從 WebLLM 到 Transformers.js，前端社群有一群反骨仔吹起一股「邊緣 LLM」的熱潮，可是，當真正將模型落地到使用者的瀏覽器時，第一個面對的考驗就是，WebGPU 真的有比 WASM 快嗎？ 這陣子實測的結論比想像中更加戲劇化， 500M 以下的微型模型，WASM 反而快了 12%，但是對於 3B 以上的大模型，WASM 直接變成悲劇，直接讓Chrome撞的頭破血流（Chrome…
Giving a fleet of AI agents one shared memory — when each agent runs a different model
Most agent frameworks give each agent its own context window and call it memory. That works right up…
Your Sandbox Has a Hole in It, and the AI Agent Found It
Two of the most sophisticated AI labs on earth ran safety evaluations on their own frontier agents, and…
Lesson 4b – Validation: Testing the gate itself
The last lesson was about validating what a model hands you. The story behind it: a set of…
A Framework-Agnostic Testing Methodology for AI Agents (61 sources, 58 test blocks, OWASP Agentic Top 10)
How do you actually test an AI agent? Not “does it respond,” but: does it route to the…
Same DeepSeek V4 Flash, Different Agent: Why the Runtime Changes the Result
Same DeepSeek V4 Flash. Different runtime. Very different long-task outcomes. My local sample is bounded: Codex + Flash…
Run and Compare AI Evaluations with a CLI for Developers and Coding Agents
TL;DR: This walkthrough shows how developers and coding agents can use Quantiles, an open-source AI evaluation platform licensed…
The $3.2 Million Question: Calculating the True Cost of AI Vendor Lock-In
The $3.2 Million Question: Calculating the True Cost of AI Vendor Lock-In Your chosen AI platform might seem…
