Open-weight large language models have undergone a structural transition: from under 2% of routed token traffic to roughly half in eighteen months, with the capability delta to the closed frontier compressing to ~5 points on standardized intelligence indices at a 6–100x lower per-token cost. This is a live, hands-on hack session, not a lecture: insight briefings are interleaved with working demonstrations on real hardware and real codebases throughout, and attendees receive the complete toolchain installers, model manifests, and demo repositories — to reproduce every exercise on their own machines. The session proceeds in three parts.
The model landscape (briefing). We survey the state-of-the-art open-weight families GLM-5.2, Qwen3, Kimi K2.6, DeepSeek V4, and MiniMax M3 through the lens of architecture and strategy rather than leaderboard rank: mixture-of-experts sparsity (e.g., 80B total / 3B active parameters), million-token context windows, licensing regimes from MIT to attribution-encumbered, and the benchmark-skepticism discipline required to evaluate vendor-reported claims.
The inference substrate (live demos). We stand up inference across the full deployment spectrum in real time: loading an 80B-parameter MoE model on-device via LM Studio (MLX-accelerated, with live memory and throughput readouts), CLI-driven serving with Ollama, NPU-targeted deployment with AMD's Lemonade, and hosted open-weight APIs unified by the OpenAI-compatible endpoint as the de facto interoperability layer. We derive the break-even economics governing tier selection — amortized infrastructure cost versus token volume and steadiness and the compliance calculus (data egress, DORA/HIPAA/ITAR, sovereignty) that increasingly dominates the decision.
The agentic harness layer (live demos). We execute the same long-horizon coding task a failing test suite and a live bug across three harnesses in front of the audience: 's ZCode (an agentic development environment purpose-tuned for GLM-5.2), 's Droid (provider-agnostic, BYOK), and, as the capstone, an agent loop pointed at the local endpoint stood up in Part II a fully local, zero-egress agentic coding workflow, demonstrated end to end. The result is an empirical argument that the harness and the model are now orthogonal design decisions.
The session closes with an interactive exercise: audience-volunteered workloads are routed live through a five-question per-workload decision framework whose output is a model portfolio rather than a vendor selection. Attendees leave with reproducible setups, cost models, and an evaluation methodology applicable within the week.
Read More →
