88 内存约束下LLM推理调度的cμ规则的渐近最优性

Asymptotic Optimality of cμ-Rule for LLM Inference Scheduling under Memory Constraint

LLM调度理论math.OC cs.AI

Zhenghao Li、Shengbo Wang、Nian Si

👉 从排队论角度建模KV cache受限下的prefill/decode调度，证明cμ规则渐近最优

大语言模型（LLM）推理，即预训练模型接收输入提示并顺序生成输出 token 的过程，随着基于 LLM 的应用的广泛部署而变得日益重要。随着推理工作负载持续扩大，GPU 内存已成为关键瓶颈：在智能体与推理工作负载中常见的长输入提示和生成序列会导致大量的键值（KV）缓存占用。本文从排队论视角研究由此产生的在线调度问题。请求以随机方式到达，具有异构的提示长度和输出长度，经历预填充和自回归解码，并竞争有限的 KV 缓存容量。与经典排队模型不同，处理步骤的持续时间同时取决于预填充计算和 KV 缓存 I/O，因此服务能力由计算、内存访问和内存占用共同决定。我们建立了一个流体模型，将系统简化为一个多类单服务器队列，其有效服务速率刻画了计算、内存访问和内存容量。该流体分析导出了一个 cμ 优先级规则，其中有效服务速率 μ 由计算速度、内存带宽和内存容量共同决定。我们证明该规则在流体模型中最小化累积持有成本，并且对于有限工作负载，还最小化完成时间并最大化全程输出 token 吞吐量。随后我们为原始有限系统构造了一个可行的调度策略，并证明其缩放后的动态收敛到最优流体策略。最后，在使用 NVIDIA H800 的真实 LLM 服务引擎上的实验表明，所提出优先级规则的实际实现相对于现有调度策略，在保持高吞吐量的同时大幅降低了平均延迟。

Large language model (LLM) inference, the process by which a pre-trained model receives an input prompt and generates output tokens sequentially, has become increasingly important with the widespread deployment of LLM-based applications. As inference workloads continue to scale, GPU memory has emerged as a critical bottleneck: the long input prompts and generation sequences common in agentic and reasoning workloads lead to substantial key-value (KV) cache usage. In this paper, we study the resulting online scheduling problem from a queueing perspective. Requests arrive stochastically with heterogeneous prompt and output lengths, undergo prefill and autoregressive decode, and compete for a finite KV-cache capacity. Unlike classical queueing models, the duration of a processing step depends on both prefill computation and KV-cache I/O, so service capacity is jointly determined by computation, memory access, and memory occupancy. We develop a fluid model that reduces the system to a multiclass single-server queue with effective service rates capturing computation, memory access, and memory capacity. This fluid analysis yields a cμ priority rule, where the effective service rate μ is jointly determined by computational speed, memory bandwidth, and memory capacity. We prove that this rule minimizes cumulative holding cost in the fluid model and, for a finite workload, also minimizes completion time and maximizes full-run output-token throughput. We then construct a feasible scheduling policy for the original finite system and prove that its scaled dynamics converge to the optimal fluid policy. Finally, experiments on a real LLM serving engine using an NVIDIA H800 show that practical implementations of the proposed priority rule substantially reduce average latency while maintaining high throughput relative to existing scheduling policies.

📄 arXiv | ⬇ PDF | arXiv:2611.04807 · math.OC

88 是什么决定了循环语言模型的服务速度？按精确性分级的测量与优化

What Sets the Serving Speed of Looped Language Models? Measurements and Optimizations Graded by Exactness

循环模型服务cs.DC

Dezhen Lu、Runguo Li、Zixiang Ni、Yuchen Tang、Xingquan Kong、Jiaxing Yang、Jiarui Guo、Lingxiang Gao

👉 测量并优化循环语言模型serving，小kernel融合与KV/前缀缓存复用提速1.6-3.2倍

循环语言模型对每个 token 多次应用同一组层，因此服务引擎每循环一次就要支付一次每层开销；我们在一个开源循环模型引擎中测量了 Ouro 和 Huginn-0125 的服务性能，并将我们的优化按从逐位完全一致的改动到需要精度检查的改动进行分级。在 batch 为 1 时，融合所有循环提交后 Ouro 的吞吐量仍为 0.99 倍；每个 token 的 5689 个小内核才是真正的成本，在舍入边界后融合它们可将吞吐量提升 1.11-1.59 倍，同时 1014/1014 个请求保持完全一致。近似融合并没有更快，并且在 87 个问题上未达到 GSM8K 的一点规则（61 和 62 正确，而 64 正确），因此我们没有发布它。Huginn 发布的默认配置避免了死 KV 槽、复用了前缀缓存页，服务速度比基础引擎快 1.77-3.16 倍；Ouro 达到 1.65-2.15 倍，智能体会话达到 1.91-3.21 倍，而 Huginn 的噪声和前缀复用改动在所有 1319 个 Huginn 测试问题上保持在一点以内；Huginn 的打包预填充在相同问题上为 -0.61 点，仅在其点估计上满足该规则。按位置键控的噪声使 CPU 缓存复用变得精确，但仍有 128 个 GPU 全命中中的 20 个不同，而循环融合延迟了预填充准入，64 图缓存则防止了急切回退。在预填充-解码分离下，禁用对等错误处理时 ExactLoop 达到基础吞吐量的最高 3.3 倍，启用时在三次运行中的两次达到 3.1 倍。其传输预算按容纳一个请求的在途 KV 设定，将提交尾部从先前预算下约一秒缩短到 16 毫秒（基础：2.8 秒），消除了引擎间隔否则会施加的轮询轮次。

Looped language models apply one block of layers many times per token, so a serving engine pays every per-layer overhead once per loop; we measure serving for Ouro and Huginn-0125 in an open-source looped-model engine and grade our optimizations from bitwise-identical changes to those that require an accuracy check. At batch 1, fusing all loop submissions leaves Ouro throughput at 0.99x; the 5689 small kernels per token are the real cost, and fusing them behind rounding fences raises throughput 1.11-1.59x while 1014/1014 requests remain identical. An approximate fusion is no faster and misses a one-point GSM8K rule on 87 questions (61 and 62 correct versus 64), so we do not ship it. Huginn's released defaults avoid dead KV slots, reuse prefix-cache pages, and serve 1.77-3.16x faster than the base engine; Ouro reaches 1.65-2.15x and agent sessions 1.91-3.21x, while Huginn's noise and prefix-reuse changes stay within one point on all 1319 Huginn test questions; Huginn's packed prefill, -0.61 points on the same questions, meets that rule only on its point estimate. Position-keyed noise makes CPU cache reuse exact but still leaves 20 of 128 GPU full hits different, while loop fusion delays prefill admission and a 64-graph cache prevents eager fallbacks. Under prefill-decode disaggregation ExactLoop reaches up to 3.3x the base throughput with peer error handling disabled, and 3.1x in two of three runs with it enabled. Its transfer budget, sized to hold one request's KV in flight, cuts the commit tail from about one second under the earlier budget to 16 ms (base: 2.8 s), removing the polling rounds that the engine's interval would otherwise impose.

💬 16 pages, 3 figures, 10 tables. Appendix included

📄 arXiv | ⬇ PDF | arXiv:2611.05150 · 分布式/并行/集群计算

88 ForgeKernel：追赶并超越专家与智能体的自演化内核

ForgeKernel: Self-Evolving Kernels Catching and Surpassing Experts and Agents

自进化GPU内核cs.DC

Yaojian Chen、Dazhi Peng、Yiyun Zheng、Zhui Zhu、Shengye Xiang、Leshan Li、Zhen Li、Xianglong Deng 等 16 人

👉 跨运行积累知识并持续演化内核架构的 GPU kernel 生成系统，直接服务推理算子优化。

专门构建的内核智能体能够编写高性能 GPU 内核，但现有系统普遍缺乏跨运行的知识积累和架构演化。每次运行都从相同的初始知识开始，后续迭代很大程度上是在搜索早期固定的架构内进行优化。我们提出 ForgeKernel，一个内核生成系统，它跨运行积累并复用知识，同时支持内核架构的持续演化。它分析高性能参考实现的硬件行为，并将经过验证的架构决策提炼为关于硬件机制的可复用知识，而非模仿源代码。随后它封存一个通过正确性和性能验证的内核，并将其用作下一次优化运行的参考，从而使架构决策得以重新审视。在 NVIDIA Hopper 和 Blackwell GPU 上的 27 个算子和 29 个输入配置中，ForgeKernel 在 25 个配置上达到或超过其专家参考，几何平均性能提升 10%。在 Ascend 910C 上，它在 CANN-Bench 第一赛季决赛排行榜全部 8 个四级算子的公开用例集上超越各算子的公开最佳成绩，几何平均提升 25.1%，而其全套件提交在 A3 排行榜上排名第一。截至 2026-10-30，由 ForgeKernel 生成的公开提交在全部三个 CANN-Bench 2.0 解决方案排行榜（Ascend 910C、950DT 和 950PR）以及 SOL-ExecBench FlashInfer-Bench 赛道上均排名第一。

Purpose-built kernel agents can write high-performance GPU kernels, but existing systems generally lack cross-run knowledge accumulation and architectural evolution. Each run starts from the same initial knowledge, and later iterations largely optimize within an architecture fixed early in the search. We present ForgeKernel, a kernel-generation system that accumulates and reuses knowledge across runs while enabling continued evolution of kernel architectures. It analyzes the hardware behavior of a high-performance reference and distills validated architectural decisions into reusable knowledge of hardware mechanisms rather than imitating source code. It then seals a kernel that passes correctness and performance validation and uses it as the reference for the next optimization run, enabling architectural decisions to be revisited. Across 27 operators and 29 input configurations on NVIDIA Hopper and Blackwell GPUs, ForgeKernel matches or beats its expert reference on 25 configurations, with a geometric-mean performance gain of 10%. On Ascend 910C, it outperforms the per-operator public best on the public case set of all 8 level-4 operators in the CANN-Bench Season 1 final leaderboard, with a geometric-mean gain of 25.1%, while its full-suite submission ranks first on the A3 leaderboard. As of 2026-10-30, public submissions generated by ForgeKernel rank first on all three CANN-Bench 2.0 solution leaderboards (Ascend 910C, 950DT, and 950PR) and on the SOL-ExecBench FlashInfer-Bench track.

💬 16 pages, 8 figures, including appendices

📄 arXiv | ⬇ PDF | arXiv:2611.04998 · 分布式/并行/集群计算

86 面向可扩展LLM推理解码的工作负载静态光电路交换

Workload-Static Optical Circuit Switching For Scalable LLM Inference Decode

光交换解码cs.DC cs.AI cs.AR cs.NI

Maximilian Thürmer、Nicolas Scotto di Perto、Eloise Stein、Natanijel Vasic、Mikaeil Bagherli、Zhenguo Wu、Benjamin Klenk、Holger Fröning 等 9 人

👉 面向decode的静态光电路交换，与模型并行协同设计，提升MoE长上下文交互性

LLM 推理解码执行细粒度的集合通信，其延迟直接限制了每用户的交互性，然而纵向扩展网络仍由为训练设计的电子分组交换机搭建。我们观察到，对于给定的模型部署和并行化方式，解码的通信结构是固定的，并提出工作负载静态光电路交换（OCS）：电路在每次部署时配置一次，与模型并行以及拓扑感知的集合通信调度协同设计，从而将空闲的加速器端口转化为更少的通信阶段或更高的带宽。我们使用 GXNET 评估该设计，GXNET 是一个基于图的工作负载模拟器，耦合了以硬件测量为依据的解析模型和 flit 级网络模型。在服务于 5T 参数 MoE 模型、上下文最多达 5M token 的 576 加速器纵向扩展域中，全 OCS 网络相比 4:1 超订的分组交换网络将交互性提升最多 65%，能效提升最多 119%。

LLM inference decode executes fine-grained collectives whose latency directly limits per-user interactivity, yet scale-up fabrics are still built from electronic packet switches designed for training. We observe that, for a given model deployment and parallelization, the communication structure of decode is fixed, and propose workload-static optical circuit switching (OCS): circuits are configured once per deployment, co-designed with model parallelism and with topology-aware collective schedules that turn spare accelerator ports into fewer communication phases or higher bandwidth. We evaluate this design with GXNET, a graph-based workload emulator coupled to analytical and flit-level network models grounded in hardware measurements. In a 576-accelerator scale-up domain serving 5T-parameter MoE models with up to 5M tokens of context, all-OCS fabrics improve interactivity by up to 65% and energy efficiency by up to 119% over 4:1 oversubscribed packet-switched fabrics.

💬 39 pages, 10 main text, 29 appendix, 28 figures

📄 arXiv | ⬇ PDF | arXiv:2611.05104 · 分布式/并行/集群计算

85 LeanSparse：面向原生稀疏注意力 LLM 的内存高效服务

LeanSparse: Memory-Efficient Serving for Native Sparse-Attention LLMs

稀疏注意力服务cs.DC

Zaifeng Pan、Chris Wu、Xinwei Qiang、Zhengding Hu、Zhongkai Yu、Yufei Ding

👉 原生稀疏注意力LLM服务系统，降低decode常驻与prefill临时显存，提升并发

稀疏注意力限制了每个查询读取的历史范围，但服务长上下文仍然需要大量 GPU 内存。现有的 KV 卸载系统将索引器键的完整历史保留在 GPU 上，限制了解码并发度。预填充的临时内存需求可能进一步延迟准入，即使剩余内存足以支持同一请求后续的解码。我们提出 LeanSparse，一个同时降低解码持久内存占用和预填充临时内存需求的服务系统。我们的设计利用了两点观察：预填充一次一层地消费历史数据，而解码可以在比较原始索引器分数之前识别出有希望的候选。在预填充期间，LeanSparse 通过两个可复用的 GPU 缓冲区流式传输各层历史，将下一层加载与正在进行的模型计算重叠。在解码期间，由粗到细的索引搜索紧凑表示，并仅针对候选细化检索原始索引器键。独立的索引器键缓存和 KV 缓存跨生成步骤复用数据。我们在 SGLang 中实现了 LeanSparse，并在 DeepSeek-V4-Flash 上进行了评估。在 950K 上下文长度下，LeanSparse 支持的并发请求数是 HiSparse 的 2.95×，解码吞吐达到 HiSparse 的 2.14×。在多轮 Open-SWE 轨迹上，相对于 ECHO，它将首 token 时间中位数从 21.6 秒降低到 0.6 秒，并将会话完成时间中位数降低了 17.7%。

Sparse attention limits the history read by each query, but serving long contexts still requires substantial GPU memory. Existing KV offloading systems retain the full history of indexer keys on GPU, limiting decode concurrency. Prefill's temporary memory demand can further delay admission even when enough memory remains for the same request's subsequent decode. We present LeanSparse, a serving system that reduces both decode's persistent memory use and prefill's temporary memory demand. Our design exploits two observations: prefill consumes historical data one layer at a time, and decode can identify promising candidates before comparing their original indexer scores. During prefill, LeanSparse streams layer histories through two reusable GPU buffers, overlapping next-layer loading with ongoing model computation. During decode, coarse-to-fine indexing searches compact representations and retrieves original indexer keys only for candidate refinement. Separate indexer key and KV caches reuse data across generation steps. We implement LeanSparse in SGLang and evaluate it on DeepSeek-V4-Flash. At 950K context length, LeanSparse supports 2.95× as many concurrent requests and achieves 2.14× decode throughput over HiSparse. On multi-turn Open-SWE traces, it reduces median time to first token from 21.6 to 0.6s and median session completion time by 17.7% relative to ECHO.

📄 arXiv | ⬇ PDF | arXiv:2611.04671 · 分布式/并行/集群计算

84 通过交错并发请求加速基于卸载的 MoE 推理

Accelerating Offloading-Based MoE Inference by Interleaving Concurrent Requests

MoE卸载调度cs.DC cs.LG

Hongseung Yu、In Gim、Kyunghan Lee、Lin Zhong

👉 发现专家缓存下预取有害，用并发请求run-ahead调度替代预取加速MoE卸载推理

专家卸载使大型混合专家（MoE）语言模型能够在本地运行，但传统的批处理会在获取缺失专家时使 GPU 停顿。一种流行的补救措施称为预取，即预测很快将需要哪些专家，并利用空闲链路带宽获取它们。然而，不精确的预测会在获取不必要的专家上浪费带宽和 GPU 内存。我们的关键发现是，当 GPU 内存中存在专家缓存时，预取可能是有害的。这是因为随着缓存包含更多必要专家，专家预测变得更加不精确。我们的关键洞见是，恰当的请求调度胜过预取。基于这一洞见，我们提出 Bevy，一个基于并发请求前瞻推理的 MoE 推理系统。专家已就绪的请求可以提前运行到后续层，而其他请求则在等待获取，从而揭示它接下来需要的专家。Bevy 随后可以在计算就绪请求的同时获取由此识别出的缺失专家，将计算与获取重叠，而无需预测专家选择。然而，单独运行请求会重复读取 GPU 内存中的共享权重并增加计算时间。为限制这一成本，Bevy 的自平衡调度器在每一层对就绪请求进行批处理，并优先处理滞后的请求，随着专家可用性的变化调整它们的分组。在两个本地平台上，Bevy 的解码吞吐最高达到 llama.cpp 的 3.46×，以及最快预取基线的 1.55×。

Expert offloading enables large mixture-of-experts (MoE) language models to run locally, but conventional batching stalls the GPU while missing experts are fetched. A popular remedy, known as prefetching, is to predict what experts will be needed soon and fetch them using idle link bandwidth. Imprecise prediction, however, wastes bandwidth and GPU memory on fetching unnecessary experts. Our key finding is that prefetching can be harmful when there is an expert cache in GPU memory. This is because as the cache includes more necessary experts, expert prediction becomes less precise. Our key insight is that proper request scheduling beats prefetching. Building on this insight, we present Bevy, an MoE inference system based on runahead inference for concurrent requests. A request whose experts are available can run ahead to later layers while other requests wait for fetches, revealing the experts it needs next. Bevy can then fetch the missing experts so identified while computing ready requests, overlapping computation and fetching without predicting expert selections. Running requests separately, however, repeats reads of shared weights in GPU memory and increases compute time. To limit this cost, Bevy's self-balancing scheduler batches ready requests at each layer and prioritizes lagging requests, adapting their grouping as expert availability changes. Across two local platforms, Bevy delivers up to 3.46× the decode throughput of llama.cpp and up to 1.55× that of the fastest prefetching baseline.

💬 12 pages, 13 figures

📄 arXiv | ⬇ PDF | arXiv:2611.04794 · 分布式/并行/集群计算

82 通过查询路由在数据中心创造功率灵活性

Creating Power Flexibility in Datacenters via Query Routing

查询路由调功cs.AI

Shivam Patel、Bingqing Chen、Jared Fernandez、Guannan Qu、Gauri Joshi

👉 用查询路由在模型实例间分配请求，实现数据中心功率灵活性且不延迟请求

大语言模型推理不断增长的能源需求凸显了数据中心通过提供电力需求灵活性来支持电网安全运行的必要性，同时还要维持交互式工作负载的服务。本工作研究查询路由作为一种实现该能力的机制，通过在不同语言模型实例之间动态分配查询而不推迟请求。我们开发了一个通用分析框架，将查询到达率和序列长度分布与 token 迭代时间、请求驻留时间、服务容量以及模型实例的功耗联系起来。利用该框架，我们刻画了可行的查询路由分配及相应的电力需求灵活性，从而能够选择路由策略以实现期望的功耗行为。我们在四个 GPU 平台上对 1B 到 70B 参数的语言模型验证了该框架，在估计可实现的功率灵活性方面获得了 2.8% 的平均绝对百分比误差。我们在一个 10,000 块 H100 GPU 数据中心的真实轨迹驱动仿真上评估了我们的方法，展现出高达总功耗 16% 的灵活性，约 10 秒内即可实现 90% 的期望功率削减。这些结果确立了查询路由作为数据中心电力需求灵活性的一种有效机制，并证明了我们提出的框架在数据中心级规划和编排中具有更广泛的适用性。

The growing energy requirements of large language model inference highlight the need for datacenters to support safe grid operation by providing power demand flexibility, while maintaining service for interactive workloads. This work investigates query routing as a mechanism to enable such capability by dynamically allocating queries across different language model instances without deferring requests. We develop a general analytical framework that connects query arrival rates and sequence length distributions to token iteration time, request residence time, serving capacity, and power consumption of model instances. Leveraging this framework, we characterize the feasible query routing allocations and the corresponding power demand flexibilities, enabling the selection of routing strategies to achieve desired power consumption behavior. We validate our framework across four GPU platforms on language models ranging from 1B to 70B parameters, obtaining 2.8% mean absolute percentage error in estimating achievable power flexibility. We evaluate our approach on a real trace-driven simulation of a 10,000 H100 GPUs datacenter, exhibiting flexibility up to 16% of total power consumption, with 90% of the desired power reduction achieved within approximately 10 seconds. These results establish query routing as an effective mechanism for datacenter power demand flexibility and demonstrate the broader applicability of our proposed framework to datacenter level planning and orchestration.

📄 arXiv | ⬇ PDF | arXiv:2611.05148 · 人工智能

78 LoopVQ：面向循环 Transformer 的亚 1 比特 KV 缓存量化

LoopVQ: Sub 1-bit KV Cache Quantization for Looped Transformers

循环模型KV量化cs.LG cs.AI cs.CL

Minsoo Cheong、Sungjoo Yoo

👉 针对looped transformer沿循环轴冗余，联合多步通道做亚1bit KV cache向量量化

循环 Transformer 是标准 Transformer 在语言建模方面的一种参数高效替代方案。通过在多个循环步骤中重复应用相同的层，近期的循环语言模型达到了数倍于其规模的非循环模型的性能。虽然这种架构有效减少了参数内存，但每个循环步骤仍存储自己的键和值，因此键值（KV）缓存随循环步骤数线性增长，并成为主要的内存瓶颈。我们观察到循环模型的 KV 缓存沿循环轴高度冗余，且各循环步骤对量化的敏感度差异很大。基于这些观察，我们提出 LoopVQ，一种面向循环 Transformer 的 KV 缓存向量量化方法，它联合编码多个循环步骤的通道，并微调码本以匹配全精度模型的输出。在每元素 0.5 比特下，LoopVQ 在 Ouro 和 Huginn 模型上的通用任务、长思维链推理和长上下文检索任务中优于面向标准 LLM 的现有 KV 缓存向量量化方法。其设计与这些方法正交，与 TaSQ 结合后，即使在每元素 0.25 比特下也能保留大部分 BF16 精度。我们还在 SGLang 中实现了 LoopVQ，在 32K token 上下文下它几乎将解码延迟减半，并在 1.4B 和 2.6B 的 Ouro 模型上相比 BF16 基线将峰值服务吞吐量分别提升最多 4.3× 和 6.1×。

Looped transformers are a parameter-efficient alternative to standard transformers for language modeling. By reapplying the same layers across multiple loop steps, recent looped language models match the performance of non-looped models several times their size. While this architecture effectively reduces parameter memory, each loop step still stores its own keys and values, so the key-value (KV) cache grows linearly with the number of loop steps and becomes the main memory bottleneck. We observe that the KV cache of looped models is highly redundant along the loop axis and that the loop steps differ substantially in their sensitivity to quantization. Based on these observations, we propose LoopVQ, a KV cache vector quantization method for looped transformers that codes channels of multiple loop steps jointly, and fine-tunes the codebooks to match the output of the full-precision model. At 0.5 bits per element, LoopVQ outperforms existing KV cache vector quantization methods for standard LLMs on general, long chain-of-thought reasoning, and long-context retrieval tasks with Ouro and Huginn models. Its design is orthogonal to these methods, and combined with TaSQ, it retains much of the BF16 accuracy even at 0.25 bits per element. We also implement LoopVQ in SGLang, where it nearly halves the decode latency at a 32K-token context and raises the peak serving throughput by up to 4.3× and 6.1× on the 1.4B and 2.6B Ouro models compared to the BF16 baseline.

📄 arXiv | ⬇ PDF | arXiv:2611.05304 · 机器学习

78 在 Lean 中形式化 PTX 以证明 GPU 内核正确性

Formalizing PTX in Lean to Prove GPU Kernel Correctness

PTX形式化验证cs.AI

Luk Koska、Tom Koska、Ben Koska

👉 在Lean中形式化PTX，验证WGMMA GEMM、FlashAttention与MoE流水线kernel正确性

我们提出了一种对 PTX 进行数学形式化的方法，该方法允许对深度学习 GPU 内核中常见的模式进行建模与推理，包括获取-释放语义、张量核心、代理、异步操作、多内核流水线以及数据相关的控制流。我们在 Lean 中实现了该方法，并用它证明了 WGMMA GEMM 内核、FlashAttention 以及一个三内核 MoE 流水线的简单实现的正确性。

We present an approach to mathematically formalizing PTX, an approach that allows modeling and reasoning about patterns common in GPU kernels for Deep Learning, including acquire-release, tensor cores, proxies, async operations, multi-kernel pipelines and data-dependent control flow. We implement it in Lean and use it to prove the correctness of simple implementations of a WGMMA GEMM kernel, FlashAttention, and a 3-kernel MoE pipeline.

📄 arXiv | ⬇ PDF | arXiv:2611.04447 · 人工智能

78 EvoCache：利用公共缓存锚点加速浏览器大语言模型交付

EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors

浏览器LLM缓存cs.DC

Zhiyu Wang、Rajkumar Buyya

👉 浏览器端LLM分发系统，用公共缓存锚点跨发布者复用模型文件，端侧推理加速

大语言模型（LLM）可以直接在浏览器中运行，但首次使用通常需要下载数百 MiB 甚至数 GiB 的模型文件。同一个 LLM 常被不同发布者重新打包；这些文件共享大量内容，但字节差异使整文件缓存无法复用它们。我们提出 EvoCache，一个由浏览器管理的 LLM 交付系统。EvoCache 使用由浏览器预先存储、与浏览历史无关的公共缓存锚点，无损重建站点所请求的发布者版本。站点仅指定目标，而浏览器选择锚点并执行重建，二者均对站点隐藏。对 1,000 个公共仓库的审计发现，100 个最流行的 LLM 中有 61 个由多个发布者提供，且 98.1% 的重叠版本无法被整文件缓存复用。在 23 对跨发布者模型包上，EvoCache 实现中位流量减少 77.3%，并优于所有对比方案。在 240 次真实端到端浏览器实验中，相对于普通完整下载，EvoCache 将网络流量减少 78.9%，将页面就绪时间减少 64.2%。对于其他站点留下的缓存数据，EvoCache 还通过对缓存命中和未命中使用相同的返回时间，保护用户的跨站缓存隐私。

Large Language Models (LLMs) can run directly in a browser, but first use typically requires downloading model files that are hundreds of MiB or even several GiB in size. The same LLM is often repackaged by different publishers; these files share substantial content, yet byte differences prevent whole-file caches from reusing them. We propose EvoCache, a browser-managed LLM delivery system. EvoCache uses public cache anchors that are stored in advance by the browser and independent of browsing history to reconstruct losslessly the publisher version requested by a site. A site specifies only the target, while the browser selects the anchor and performs reconstruction, keeping both hidden from the site. An audit of 1,000 public repositories finds that 61 of the 100 most popular LLMs are offered by multiple publishers and that 98.1% of overlapping versions cannot be reused by whole-file caching. Across 23 cross-publisher model-package pairs, EvoCache achieves a median traffic reduction of 77.3% and outperforms every comparator. Across 240 real end-to-end browser experiments, EvoCache reduces network traffic by 78.9% and page-ready time by 64.2% relative to ordinary full downloads. For cached data left by other sites, EvoCache also protects users' cross-site cache privacy by using the same return time for cache hits and misses.

📄 arXiv | ⬇ PDF | arXiv:2611.04638 · 分布式/并行/集群计算

72 FourSight：通过序列过滤与稀疏通道补偿实现稳定的 FP4 强化学习后训练

FourSight: Stable FP4 RL Post-Training via Sequence Filtering and Sparse Channel Compensation

FP4 RL后训练🔄 训练技术·可迁移cs.AI cs.LG

Yining Wei、Tianyuan Wu、Gu Gong、Yuechen Tao、Yinghao Yu、Xiaowen Chu、Wei Wang

👉 NVFP4 RL后训练的序列过滤与稀疏通道补偿，缓解训练推理失配；可迁移至低比特推理

强化学习（RL）后训练正日益面向低比特部署，其中 NVFP4 权重与激活量化（W4A4）已成为当前实践中的主流格式。然而，将 NVFP4 集成到 RL 中会降低 rollout 质量，并放大跨引擎的训练-推理不匹配（TIM），从而使训练不稳定。我们表明，这些失败是集中出现的，而非均匀分布的——严重不匹配的尾部 token 集中在少数离群序列中，而策略权重基本保持完好，量化误差则集中在少数高方差激活通道中。基于这一观察，我们提出 FourSight，一种算法-系统协同设计，包含两个有针对性的解决方案：TIM 感知的序列过滤，将漂移的轨迹从策略更新中排除；以及稀疏精度补偿，仅将方差最高的通道保留为 BF16，其余部分保持原生 NVFP4。我们进一步设计了一个融合的 SpliceGEMM 内核，带有专门的预填充与解码数据通路，以最小化开销。FourSight 在多种模型和引擎上保持了接近 BF16 的精度和强大的 FP4 部署质量，同时将达到目标精度所需的步数改善了 1.76-2.60 倍，并且在高达 120B 规模的模型上，步时与 FP4 的差距保持在 2.3% 以内。

Reinforcement learning (RL) post-training increasingly targets low-bit deployment, with NVFP4 weight-and activation quantization (W4A4) becoming the state-of-the-practice format. However, integrating NVFP4 into RL degrades rollout quality and amplifies cross-engine training-inference mismatch (TIM), destabilizing training. We show that these failures are concentrated rather than uniform - severely mismatched tail tokens concentrate in a few outlier sequences, while policy weights remain largely intact and quantization error concentrates in a few high-variance activation channels. Guided by this, we present FourSight, an algorithm-system co-design with two targeted solutions: TIM-aware sequence filtering that excludes drifted trajectories from policy updates, and sparse precision compensation that keeps only top-variance channels in BF16 while retaining native NVFP4 elsewhere. We further design a fused SpliceGEMM kernel with specialized prefill and decode datapaths to minimize overhead. FourSight sustains near-BF16 accuracy and strong FP4 deployment quality across models and engines, while improving step-to-accuracy by 1.76-2.60x and staying within 2.3% of FP4 step time up to 120B-scale models.

💬 18 pages, 13 figures

📄 arXiv | ⬇ PDF | arXiv:2611.04887 · 人工智能

72 Perseus：面向同态模型推理的全栈系统

Perseus: A Full-Stack System for Homomorphic Model Inference

同态加密推理cs.CR cs.AI

Alessandro Zirilli、Davide Marincione、Evgenios M. Kornaropoulos、Giuseppe Ateniese、Emanuele Rodolà

👉 开源全栈FHE模型推理库，后端密钥再生、数据打包与非线性算子优化

大语言模型（LLM）使用户将私有数据发送给以明文处理这些数据的模型提供商变得异常便捷。全同态加密（FHE）有望通过在加密数据上直接计算来保障远程推理的安全，但其速度一直慢得令人难以承受。使其走向实用的努力仍然零散，现有框架大多为闭源，这阻碍了进展的累积。我们提出 Perseus，一个免费开源的 Python 软件库，支持加密模型推理的全栈流程，从密码学后端与数据打包，到加密非线性运算与端到端模型执行。Perseus 内置了新颖的优化，在栈的每一层各有一项：在 (i) 后端层面进行密钥切换密钥的内核内再生，在 (ii) 数据层面采用 Cachemir 打包的复数变体，以及在 (iii) 模型层面采用同时感知打包与量级的最先进自举放置器。Perseus 在多种模型与模态上实现了最先进的延迟，与明文模型的精度和困惑度相差在 3% 以内。值得注意的是，在 Llama-3-8B 上，其单 token 解码速度比任何其他实现快 3.96×。我们首次发起了针对加密 GPT-2 的公开竞速，以便进展能够累积。

Large Language Models (LLMs) have made it exceptionally convenient for users to send their private data to model providers, which process them in plaintext. Fully Homomorphic Encryption (FHE) promises to secure remote inference by computing directly on encrypted data, yet it has remained prohibitively slow. The effort to make it practical remains fragmented, with most existing frameworks being closed-source, which keeps progress from compounding. We present Perseus, a free and open Python software library that supports the full stack of encrypted model inference, from the cryptographic backend and data packing, to encrypted nonlinearities and end-to-end model execution. Perseus ships with built-in novel optimizations, one at each level of the stack: in-kernel regeneration of key-switching keys at the (i) backend level, a complex variant of Cachemir packing at the (ii) data level, and a state-of-the-art bootstrap placer that is both packing- and magnitude-aware at the (iii) model level. Perseus achieves state-of-the-art latency across multiple models and modalities, within 3% of the plaintext models' accuracy and perplexity. Remarkably, it is 3.96× faster at single-token decoding than any other implementation on Llama-3-8B. We initiate, for the first time, a public speedrun on encrypted GPT-2, so that progress can compound.

💬 4 pages, 5 tables

📄 arXiv | ⬇ PDF | arXiv:2611.04488 · cs.CR

72 GSPIM：面向支持 PIM 的边缘设备的时间局部性感知 4DGS 加速器

GSPIM: A Temporal-Locality-Aware 4DGS Accelerator for PIM-Enabled Edge Devices

4DGS边缘加速器cs.AR

Zirui Ma、Zhihua Fan、Haibin Wu、Dongrui Fan、Xiaochun Ye、Wenming Li

👉 面向PIM边缘设备的4D高斯泼溅加速器，利用时序局部性做近存过滤压缩，降低LPDDR5带宽

4D 高斯泼溅（4DGS）能够为动态场景实现高质量的实时新视角合成，在边缘端支持自由视点视频、移动 XR 和自主系统。使用显式 4D 分布、关键帧插值或锚点绑定的方法都存在两个 GPU 数据移动瓶颈。时间过滤在模型记录加载之后进行，浪费了 LPDDR5 带宽于非活跃数据。帧隔离执行还忽略了相邻帧之间高度重叠的活跃集合和深度关系，当工作集超出片上缓存容量时会重新加载共享属性。为解决这些瓶颈，我们提出了 GSPIM，一种面向支持 PIM 的边缘设备的时间局部性感知 4DGS 加速器。多态 PIM 将共享的近内存数据通路适配到特定于表示的字段布局和选择过程。它在 GPU 访问完整属性之前将有效数据过滤并压缩为连续输入，减少无效传输。多帧渲染数据流在时间窗口内复用活跃属性和稳定的深度关系，减少跨帧的重复工作。混合数据流调度将近内存数据准备与 GPU 渲染重叠，以减少等待。评估结合了针对 Jetson AGX Orin 64 GB 校准的 GPU-PIM 模型与所添加电路的 1z DRAM 物理实现。在三种 4DGS 方法和六个动态场景上，GSPIM 相比完全优化的逐帧 GPU 基线实现了 4.15 倍的几何平均加速和 4.99 倍的几何平均能效提升。最大场景级 PSNR 差异为 0.011 dB，在 32 个 LPDDR5 裸片上增加的物理面积为 18.7172 平方毫米。

4D Gaussian Splatting (4DGS) enables high-quality real-time novel-view synthesis for dynamic scenes, supporting free-viewpoint video, mobile XR, and autonomous systems at the edge. Methods using explicit 4D distributions, keyframe interpolation, or anchor binding share two GPU data-movement bottlenecks. Temporal filtering follows model record loading, wasting LPDDR5 bandwidth on inactive data. Frame-isolated execution also overlooks highly overlapping active sets and depth relationships across adjacent frames, reloading shared attributes when working sets exceed on-chip cache capacity. To address these bottlenecks, we present GSPIM, a temporal-locality-aware 4DGS accelerator for PIM-enabled edge devices. Polymorphic PIM adapts a shared near-memory datapath to representation-specific field layouts and selection procedures. It filters and compacts valid data into contiguous inputs before the GPU accesses complete attributes, reducing ineffective transfers. Multi-Frame Rendering Dataflow reuses active attributes and stable depth relationships within a temporal window, reducing repeated work across frames. Hybrid Dataflow Scheduling overlaps near-memory data preparation with GPU rendering to reduce waiting. Evaluation combines a GPU-PIM model calibrated to Jetson AGX Orin 64 GB with a 1z DRAM physical implementation of the added circuits. Across three 4DGS methods and six dynamic scenes, GSPIM achieves a 4.15x geometric-mean speedup and a 4.99x geometric-mean energy-efficiency improvement over a fully optimized frame-by-frame GPU baseline. The maximum scene-level PSNR difference is 0.011 dB, and the added physical footprint is 18.7172 square millimeters across 32 LPDDR5 dies.

💬 15 pages, 23 figures, Accepted by HPCA 2027. Code repo: https://github.com/MAdrid1011/GSPIM

📄 arXiv | ⬇ PDF | arXiv:2611.04804 · 硬件架构

72 LatentTriad：用于恶意智能体技能检测的行动前风险表征

LatentTriad: Pre-Action Risk Representations for Malicious Agent Skill Detection

恶意技能检测cs.AI

Zhe Yu、Yi Liu、Ying Zhang、Simin Chen

👉 利用prefill隐状态检测agent恶意技能，零输出token零工具调用，属agent安全与推理侧检测。

大语言模型智能体越来越多地被扩展以可复用技能，但不可信的技能内容可能诱导出超出用户任务范围的操作。检测这种上下文风险需要的不仅仅是可疑模式匹配，而动态分析又会带来技能执行的成本。我们发现，预填充阶段形成的隐藏状态包含关于任务-操作一致性的可读信息，在受控评估中包括跨留出表达形式的情况。匹配的操作状态干预也会影响后续的边界状态，并将固定的风险读数推向供体标签。基于这些发现，我们提出 LatentTriad，它结合了上下文边界特征、组调整的操作方向，以及对良性状态关系的偏离。所有分支共享一次预填充：检测不生成任何输出 token、不调用任何工具、不执行任何技能代码。在来自五个来源、七种骨干模型的 5,830 个技能包上，LatentTriad 达到 99.0-99.2% 的汇总 F1 和 93.0-94.6% 的来源宏平均 F1。一个单独测量的紧凑实现以 259-284 毫秒的中位数完成预填充和检测。这些结果将行动前风险的可读性与功能影响联系起来，实现了无需执行的技能筛查。

Large language model agents are increasingly extended with reusable skills, but untrusted skill content can induce operations outside the user's task. Detecting this contextual risk requires more than suspicious-pattern matching, while dynamic analysis incurs the cost of skill execution. We find that hidden states formed during prefill contain readable information about task-operation consistency, including across held-out expression forms in controlled evaluations. Matched Operation-state interventions also affect later Boundary states and shift fixed risk readouts toward the donor label. Based on these findings, we introduce LatentTriad, which combines contextual Boundary features, group-adjusted Operation directions, and departures from benign state relationships. All branches share one prefill: detection generates no output tokens, invokes no tools, and executes no skill code. Across 5,830 skill packages from five sources and seven backbones, LatentTriad achieves 99.0-99.2% pooled F1 and 93.0-94.6% source-macro F1. A separately measured compact implementation completes prefill and detection in a median of 259-284 ms. These results connect pre-action risk readability and functional influence to execution-free skill screening.

💬 Preprint. Under review. 44 pages, 16 figures, 37 tables. 9 pages of main text

📄 arXiv | ⬇ PDF | arXiv:2611.05088 · 人工智能

72 少想一点，局部地：面向推理压缩的局部偏好学习

Think Less, Locally: Local Preference Learning for Reasoning Compression

推理压缩cs.AI cs.CL

Muyao Yuan、Liyan Xu、Weizhan Zhang、Muyan Jiao、Jiangnan Li、Jinchao Zhang、Jie Zhou

👉 局部偏好学习判断何时停止推理，降低CoT推理成本，可迁移至推理加速

大型推理模型在挑战性任务上取得了强劲性能，但其长思维链往往在已获得足够信息后仍继续推敲，从而带来可观的推理成本。现有的推理压缩方法主要作用于推理轨迹，较少关注在当前状态下是否需要进一步推理这一局部决策。我们提出 LoCo，一种用于推理压缩的局部偏好学习方法。LoCo 从模型自身的轨迹中检测候选的过度思考状态，冻结相应的推理前缀，从相同状态重新生成更短且经过验证的续写，并用 DPO 在由此得到的状态条件偏好对上训练。这一形式教会模型何时继续推理、何时得出结论，而非简单地鼓励整体更短的轨迹。我们完全从自生成的数学轨迹构建训练数据，并在五个数学和三个代码生成基准上评估 LoCo。LoCo 在数学上将补全长度减少 41.7%–45.4%，在代码上减少 23.4%–36.3%，同时保持或提升准确率，并在所有代码基准上取得最佳 AUC。LoCo 在数学上持续改善准确率-效率权衡，并且尽管仅在数学上训练，也能有效迁移到代码生成，展示了局部偏好学习用于推理压缩的跨域通用性。代码和模型将在 https://github.com/MuyaoYuan/LoCo 公开发布。

Large reasoning models achieve strong performance on challenging tasks, but their long chain-of-thought often incurs substantial inference costs by continuing to deliberate after sufficient information has been obtained. Existing reasoning compression methods primarily operate on reasoning trajectories, paying less attention to the local decision of whether further reasoning is necessary given the current state. We propose LoCo, a local preference learning method for reasoning compression. LoCo detects candidate overthinking states from the model's own traces, freezes the corresponding reasoning prefixes, regenerates shorter verified continuations from the same states, and trains on the resulting state-conditioned preference pairs with DPO. This formulation teaches the model when to continue reasoning and when to conclude, rather than simply encouraging shorter trajectories overall. We construct training data entirely from self-generated mathematical traces and evaluate LoCo on five mathematical and three code generation benchmarks. LoCo reduces completion length by 41.7%--45.4% on mathematics and 23.4%--36.3% on code, while maintaining or improving accuracy and achieving the best AUC across all code benchmarks. LoCo consistently improves the accuracy-efficiency trade-off on mathematics and transfers effectively to code generation despite math-only training, demonstrating the cross-domain generality of local preference learning for reasoning compression. The code and model will be publicly released at https://github.com/MuyaoYuan/LoCo.

📄 arXiv | ⬇ PDF | arXiv:2611.04814 · 人工智能

72 EvoCG：面向更快共轭梯度的自进化精度控制

EvoCG: Self-Evolving Precision Control for Faster Conjugate Gradient

混合精度CG求解cs.DC

Jiaqi Zhai、Haozhi Han、Kaiyi Huang、Wenju Zhao、Yifeng Chen、Yunquan Zhang、Ting Cao、Yunxin Liu 等 11 人

👉 RL在线控制tile精度+混合精度SpMV核，加速CG求解器，属GPU算子/精度优化

现代 GPU 在低精度下的吞吐量远高于 FP64，但要在共轭梯度（CG）求解器中利用这一点却很困难，因为对降低精度的容忍度会随 Krylov 状态而演变。我们提出 EvoCG，一种混合精度 CG 求解器，它将分块（tile）精度视为一种在线控制动作，而非固定设置：迭代循环中的强化学习策略根据不断演变的求解器状态为每个分块选择精度，并通过与 Tile-CSC 统一混合精度 SpMV 内核以及低开销的循环内运行时的协同设计而获得收益。在来自八个科学领域的 47 个对称正定（SPD）矩阵上，EvoCG 是整体上最快的求解器，相对于 cuSPARSE、PETSc、Ginkgo、Mille-FP64 和 Mille-Mixed 的几何平均加速比分别为 2.52 倍、3.06 倍、2.48 倍、1.93 倍和 6.99 倍。在相同底层实现上，它相对于 EvoCG-FP64 将求解时间缩短了 1.28 倍，而迭代次数仅增加 11.4%、开销仅增加 1.09%，用 FP64 重新计算的最终真实残差范数与 FP64 的结果几乎相同，并且无需重新训练即可扩展到 Jacobi 预条件 CG，相对于 EvoCG-FP64 进一步获得 1.15 倍的几何平均加速。

Modern GPUs offer far higher throughput in low precision than in FP64, but exploiting this in conjugate gradient (CG) solvers is difficult because tolerance to reduced precision evolves with the Krylov state. We present EvoCG, a mixed-precision CG solver that treats tile-wise precision as an online control action rather than a fixed setting: a reinforcement-learning policy in the iteration loop selects per-tile precision from the evolving solver state, made profitable by co-design with a Tile-CSC unified mixed-precision SpMV kernel and a low-overhead in-loop runtime. Across 47 SPD matrices from eight scientific domains, EvoCG is the fastest solver overall, with geometric-mean speedups of 2.52x, 3.06x, 2.48x, 1.93x, and 6.99x over cuSPARSE, PETSc, Ginkgo, Mille-FP64, and Mille-Mixed. On the same substrate, it improves time-to-solution over EvoCG-FP64 by 1.28x at only 11.4% more iterations and 1.09% overhead, with FP64-recomputed final true-residual norms nearly identical to those of FP64, and extends to Jacobi-preconditioned CG without retraining, yielding a further 1.15x geometric-mean speedup over EvoCG-FP64.

💬 Accepted in the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP'27)

📄 arXiv | ⬇ PDF | arXiv:2611.04469 · 分布式/并行/集群计算

70 对抗性推测推理提升长程智能体

Adversarial Speculative Reasoning Improves Long Horizon Agents

投机解码噪声cs.AI

Hongyin Luo、Wei Fang、Lijie Yang、Owen Stepan、Daniel Suit、Jeff Krapf、Dana Wensberg、Jack O'Brien

👉 揭示投机解码验证窗口数值噪声影响长程agent推理，通过控制噪声提升准确率

推测解码一直被视为一种纯系统优化，可在不改变模型行为的情况下加速大语言模型解码。在这种观点下，训练更好草稿模型的日益增长的努力仅集中在接受率上。我们认为这一观点对长程智能体推理并不成立。在使用不同推测解码方法测量前沿开放权重模型在复杂智能体任务上的表现时，我们发现数值噪声在验证窗口内增长，思维过程的草稿接受结果会在 30 步后改变模型的输出分布，从而以随机方向显著影响长程智能体的推理准确率。在本工作中，我们通过控制此类噪声来提升目标模型的推理能力。虽然我们无法改变由目标模型验证的整体 token 分布，但我们可以通过训练草稿模型被拒绝来以最高精度编码关键推理。我们在智能体轨迹中识别这些点，并训练草稿模型偏好对抗性偏好。在推理时，训练后的草稿模型与目标模型恰好在这些决策点分道扬镳，验证重新开始，从而使关键思维 token 由目标模型采样并在其噪声最小的条件下编码。这种方法使关键推理在长程上得到更好的索引并更持久。在复杂智能体编码基准上，我们的方法在目标模型或其输出分布不变的情况下提高了端到端成功率。我们的结果将推测解码机制从纯加速器重新定义为上下文精度的主动分配器，通过训练托管参数的 0.06% 至 0.28% 来提升长程智能体的准确率。

Speculative decoding has been treated purely as a systems optimization that speeds up large language model decoding without changing the model behavior. Under this view, the growing effort to train better draft models has been focused solely on acceptance rate. We argue that this view does not hold true for long-horizon agent reasoning. Measuring frontier open-weight models on complex agentic tasks with different speculative decoding approaches, we found that numerical noise grows within verification windows, and the draft acceptance results of the thinking process can change the output distribution of the model after 30 steps, thus noticeably influencing the reasoning accuracy of long-horizon agents in random directions. In this work, we improve the reasoning capability of target models by controlling such noise. While we cannot change the overall token distribution verified by the target model, we can encode critical reasoning with highest precision by training the draft model to be rejected. We identify these points in agent traces and train the draft model to favor adversarial preference. At inference the trained draft and the target part ways precisely at these decision points, verification restarts, so that critical thinking tokens are sampled from the target model and encoded under its least noisy conditions. This approach makes critical reasoning better indexed and more durable over a long horizon. On complex agentic coding benchmarks, our approach raises end-to-end success rate with no change to the target model or its output distribution. Our results recast the speculative decoding mechanism from a pure accelerator into an active allocator for context precision that improves the accuracy of long-horizon agents by training 0.06% to 0.28% of hosted parameters.

📄 arXiv | ⬇ PDF | arXiv:2611.05345 · 人工智能

68 利用 Transformer 注意力中的偏置

Exploiting Biases in Attention for Transformers

注意力偏置剪枝cs.LG

Charles Herr、Gabriele Spadaro、Enzo Tartaglione

👉 显式建模token绝对重要性偏置，剪枝注意力图降低FLOPs与推理延迟

已知视觉 Transformer 中的注意力是稀疏的。特别地，我们分析了注意力矩阵各列与最终计算之间相关性的不均衡，并表明这一现象随网络深度增加而愈发明显。这可以解释为：token 对注意力过程具有一种绝对重要性，且与查询无关。受此启发，我们提出了一种注意力公式的重新表述，通过向查询-键点积添加一个依赖输入的偏置，使这一现象显式化。我们表明，这一新项为每个 token 赋予一个标量值，指示其在注意力过程中对其他 token 的预期影响，因而可作为剪枝注意力图的代理指标。在多种 Transformer 模型规模上，这使我们能够在 FLOPs 和推理延迟两方面大幅降低注意力的计算成本，同时保留大部分原始分类准确率。我们已在 https://github.com/EIDOSLAB/AttentionWithBias 公开代码与模型。

Attention is known to be sparse in vision transformers. In particular, we analyze the unequal relevance of attention-matrix columns to the final computation and show that this phenomenon becomes increasingly pronounced with network depth. This can be interpreted as saying that tokens have an absolute importance for the attention process, independently of queries. Inspired by this, we propose a reformulation of the attention formula that makes this phenomenon explicit by adding an input-dependent bias to the query-key dot product. We show that this new term gives each token a scalar value indicating its anticipated influence on other tokens during the attention process, and can thus serve as a proxy for pruning attention maps. Across multiple transformer model sizes, this allows us to substantially reduce the computational cost of attention, in terms of both FLOPs and inference latency, while retaining most of the original classification accuracy. We publicly released the code and models at https://github.com/EIDOSLAB/AttentionWithBias.

💬 23 pages, 21 figures, accepted at WACV 2027

📄 arXiv | ⬇ PDF | arXiv:2611.04657 · 机器学习

68 VIGOR：面向多模态离散扩散语言模型幻觉缓解的视觉增益排序

VIGOR: Visual Gain Ordering for Hallucination Mitigation in Multimodal Discrete Diffusion Language Models

扩散模型解码顺序cs.CV cs.AI

Junzhe Chen、Tian Qin、Eugenie Shi、Tianshu Zhang、Qiang Ju、Lijie Wen

👉 多模态离散扩散LLM的免训练推理策略，按反事实视觉证据重排解掩码顺序以抑制幻觉，属解码加速/质量优化。

多模态离散扩散语言模型（dLLM）通过迭代掩码预测生成回复，为自回归大型视觉语言模型提供了一种有前景的替代方案。然而，它们仍然存在幻觉问题，产生的输出在语言先验下看似合理，但缺乏充分的图像依据。我们认为，多模态 dLLM 中的幻觉不仅是 token 预测问题，也是一个过早承诺问题：在迭代去噪过程中，一些掩码位置被过早地解除掩码，因为其置信度更多由语言先验而非视觉证据驱动，而这些早期承诺会偏置后续生成。为解决这一问题，我们提出视觉增益排序（VIGOR），一种无需训练、即插即用的推理策略，它根据反事实视觉证据对掩码位置重新排序。在每个去噪步骤中，VIGOR 比较每个候选 token 在原始图像条件前向与额外视觉消融前向下的置信度，并优先处理那些预测得到图像更强支持的位置。在 LLaDA-V 和 Lumina-DiMOO 上的实验表明，VIGOR 在各项基准上持续减少幻觉并提升多模态推理性能。代码见 https://github.com/THU-BPM/Vigor。

Multimodal discrete diffusion language models (dLLMs) generate responses through iterative mask prediction and offer a promising alternative to autoregressive large vision-language models. However, they still suffer from hallucination, producing outputs that are plausible under language priors but insufficiently grounded in the image. We argue that hallucination in multimodal dLLMs is not only a token prediction problem, but also a premature commitment problem: during iterative denoising, some masked positions are unmasked too early because their confidence is driven more by language priors than by visual evidence, and these early commitments can bias later generation. To address this issue, we propose Visual Gain Ordering (VIGOR), a training-free and plug-and-play inference strategy that reorders masked positions according to counterfactual visual evidence. At each denoising step, VIGOR compares the confidence of each candidate token under the original image-conditioned pass and an additional visual-ablation pass, and prioritizes positions whose predictions are more strongly supported by the image. Experiments on LLaDA-V and Lumina-DiMOO show that VIGOR consistently reduces hallucination and improves multimodal reasoning performance across benchmarks. Code is available at https://github.com/THU-BPM/Vigor.

💬 Accepted to NeurIPS 2026

📄 arXiv | ⬇ PDF | arXiv:2611.05066 · cs.CV

68 迈向早退的最优控制基础——转轨与 ResNet

Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets

早退最优控制理论cs.LG eess.SY

Jens Püttschneider、Simon Heilig、Asja Fischer、Timm Faulwasser

👉 用最优控制理论分析早退网络turnpike现象，给出层数上界与过思考洞见

通过神经网络的前馈传播对训练和推理至关重要。为权衡效率与精度，已有研究提出将隐藏状态的中间输出视为训练期间的数据相关正则化，并最终在推理期间视为早退。在本工作中，我们利用最优控制理论分析针对输出层和中间层计算的损失函数的联合优化。我们证明，以这种联合方式训练的 ResNet 表现出所谓的转轨现象，这意味着达到规定损失水平所需的层数有一个与整体网络深度无关的上界。这在不同深度的早退网络的内部表示之间诱导出一种相似性。我们将转轨与过度思考现象联系起来，并为源自 softmax 概率的早退准则（特别是熵和最大概率）的成功提供理论洞见。在一系列实验中，我们展示了转轨的存在与效应，以及由此产生的不同深度 ResNet 在早退性能上的相似性。

The feed-forward propagation through neural networks is crucial for training and inference. To trade off efficiency and accuracy, it has been proposed to consider intermediate outputs of hidden states as data-dependent regularization during training and, ultimately, as early-exits during inference. In this work, we analyze the joint optimization of loss functions computed for the output layer and intermediate layers using optimal control theory. We prove that ResNets trained in this joint fashion exhibit the so-called turnpike phenomenon, this implies that the number of layers required to reach a prescribed loss level is bounded from above independently of the overall network depth. This induces a similarity property across the internal representations of early-exiting networks of varying depths. We connect the turnpike to the overthinking phenomenon, and provide theoretical insights into the success of early-exit criteria derived from softmax probabilities, specifically entropy and max-probability. In a range of experiments, we demonstrate the existence and effect of turnpikes and the resulting similarity in the early-exit performance for ResNets of varying depths.

💬 Accepted at NeurIPS 2026

📄 arXiv | ⬇ PDF | arXiv:2611.05120 · 机器学习

66 LoopQ：对量化视觉-语言-动作模型的公平闭环评估

LoopQ: Fair, Closed-Loop Evaluation of Quantized Vision-Language-Action Models

VLA量化评测cs.RO

Syed Umer Bukhari、Magnus Almgren、Ahmed Ali-Eldin

👉 闭环基准公平比较VLA模型PTQ方法，跨H100/L4/Jetson评测可复现性

面向视觉-语言-动作（VLA）模型的训练后量化（PTQ）有望实现更小、更快的 VLA 部署。然而，在分析现有 VLA 量化工作时，我们意识到很难推断所提出的量化方法与现有的、类似的、或在许多情况下更简单的 PTQ 方法相比的性能。我们提出 LoopQ，一个闭环基准，在同一检查点、层和引擎上比较多种量化策略，仅改变方法和内核。我们在四个 VLA 上评估 RTN、AWQ、GPTQ、ActQuant 和 QuantVLA，使用超过 180,000 个 LIBERO 和 LIBERO-plus 回合，在相同层上与 QVLA 报告的结果进行比较，并基准测试四个公开的 VLA 量化发布版本能否被复现。我们在 NVIDIA H100、L4 和 Jetson AGX Thor 上对这些方法进行基准测试。我们的结果表明，虽然量化可靠地节省内存（真实低比特模型可节省 2-3 倍），但 VLA 专用的 PTQ 方法表现不如更简单的通用基线。在 4 比特下，RTN、AWQ 和 GPTQ 达到或超过 ActQuant 和 QuantVLA，普通 RTN 与校准方法相当，普通 AWQ 与 QVLA 自我报告的成功率相当。标准 LIBERO 掩盖了确实存在的损失：在 LIBERO 上无显著损失的配置在成对的 LIBERO-plus 扰动下损失 1.5-4.3 分。速度方面的表现更弱。在 H100 上批大小为 1 时，没有任何量化后的 pi0.5 或 OpenVLA-OFT 配置比其 16 比特模型更快（在库内核上慢 1.06-1.58 倍）；只有使用 int8 内核的 L4 有增益（4-14%），Thor 在每个 4 比特配置上都更慢，而仅引擎本身就会使延迟变化 2.1 倍。性能剖析解释了原因：pi0.5 的量化骨干是计算受限的，而其内存受限的动作专家（占 GPU 时间的 63% 和参数的 13%）仍保持 16 比特。

Post-training quantization (PTQ) for vision-language-action (VLA) models promises smaller and faster VLA deployments. However, when analyzing existing VLA quantization works, we realised that it is hard to reason about the performance of the suggested quantization method compared to existing, similar, or in many cases, simpler PTQ methods. We present LoopQ, a closed-loop benchmark that compares multiple quantization policies on the same checkpoint, layers and engine, varying only the method and the kernels. We evaluate RTN, AWQ, GPTQ, ActQuant and QuantVLA on four VLAs with more than 180,000 episodes of LIBERO and LIBERO-plus, compare against QVLA's reported results on the same layers, and benchmark whether the four public VLA-quantization releases can be reproduced. We benchmark the methods on an NVIDIA H100, an L4, and a Jetson AGX Thor. Our results show that while quantization reliably saves memory, 2-3x for real low-bit models, VLA-specific PTQ methods underperform compared to simpler generic baselines. At 4 bits, RTN, AWQ and GPTQ match or exceed ActQuant and QuantVLA, plain RTN matches the calibrated methods, and plain AWQ matches the success QVLA reports for itself. Standard LIBERO hides the losses that do exist: configurations with no significant LIBERO loss lose 1.5-4.3 points under paired LIBERO-plus perturbations. Speed is a weaker story. At batch 1 on the H100, no quantized pi0.5 or OpenVLA-OFT configuration is faster than its 16-bit model (1.06-1.58x slower on library kernels); only the L4 with int8 kernels gains (4-14%), the Thor is slower on every 4-bit configuration, and the engine alone changes latency by 2.1x. Profiling explains why: pi0.5's quantized backbone is compute-bound, while its memory-bound action expert, 63% of GPU time and 13% of the parameters, stays in 16 bits.

📄 arXiv | ⬇ PDF | arXiv:2611.05093 · cs.RO

66 BtsPruner：面向大型视觉语言模型的角色感知视觉 token 剪枝

BtsPruner: Role-Aware Visual Token Pruning for Large Vision-Language Models

视觉token剪枝cs.CV cs.AI

Zihao Qi、Zhibo Zhang、Miaoxin Zhang、Jincheng Cai、Zengqiang Yan

👉 免训练角色感知视觉token剪枝，按局部细节与全局代表角色保留互补token

大型视觉语言模型（LVLM）通常要处理数百或数千个视觉 token，导致推理期间产生大量的计算与内存开销。现有的免训练剪枝方法通常使用语言模型注意力、视觉注意力或特征相似度来估计 token 重要性。然而，这些方法以统一的标准对待视觉 token，忽略了 token 在视觉编码过程中可能扮演不同角色。在本工作中，我们研究了 LVLM 中视觉 token 角色的演化，并观察到随着编码的推进，token 逐渐从局部细节载体转变为全局引用的代表。基于这一观察，我们提出 BtsPruner，一个免训练的角色感知视觉 token 剪枝框架，在固定预算下保留互补的 token 角色。具体而言，BtsPruner 首先基于注意力集中度的演化识别 token 角色转变点。然后，它从较晚的编码阶段发现全局引用的代表，并从较早阶段递归地检索局部细节 token，以恢复互补的细粒度证据。在 LLaVA-1.5、LLaVA-NeXT、Qwen-VL、InternVL 和 CogVLM 上的大量实验表明，BtsPruner 在激进的 token 削减下实现了更优的性能保持。在 LLaVA-1.5-7B 上，BtsPruner 仅使用 576 个视觉 token 中的 32 个（5.6%）就保持了稠密模型 91.6% 的性能。在高分辨率的 LLaVA-NeXT-7B 设定下，它实现了最高 5.95 倍的 CUDA 延迟加速。

Large Vision-Language Models (LVLMs) often process hundreds or thousands of visual tokens, resulting in substantial computational and memory overhead during inference. Existing training-free pruning methods typically estimate token importance using language-model attention, visual attention, or feature similarity. However, these methods treat visual tokens with uniform criteria and overlook that tokens may serve different roles during visual encoding. In this work, we investigate the evolution of visual token roles in LVLMs and observe that tokens progressively transition from local detail carriers to globally referenced representatives as encoding proceeds. Based on this observation, we propose BtsPruner, a training-free role-aware visual token pruning framework that preserves complementary token roles under a fixed budget. Specifically, BtsPruner first identifies the token-role transition point based on the evolution of attention concentration. It then discovers globally referenced representatives from later encoding stages and recursively retrieves local detail tokens from earlier stages to recover complementary fine-grained evidence. Extensive experiments across LLaVA-1.5, LLaVA-NeXT, Qwen-VL, InternVL, and CogVLM demonstrate that BtsPruner achieves superior performance retention under aggressive token reduction. On LLaVA-1.5-7B, BtsPruner preserves 91.6% of the dense model's performance using only 32 of 576 visual tokens (5.6%). In the high-resolution LLaVA-NeXT-7B setting, it achieves up to a 5.95x CUDA-latency speedup.

📄 arXiv | ⬇ PDF | arXiv:2611.04537 · cs.CV

82 ToolSpace：为长程智能体扩展以工具为中心的环境合成

ToolSpace: Scaling Tool-Centric Environment Synthesis for Long-Horizon Agents

工具环境合成cs.CL

Guanting Dong、Zhichao Sun、Xinyu Yang、Rui Wang、Xin Deng、Jiajie Jin、Zhixiong Yang、Junhao Huang 等 25 人

👉 以工具为中心从真实MCP执行轨迹挖掘依赖并聚类合成环境，扩展长程agent训练环境，系统贡献明确。

大型语言模型被部署为长程智能体，它们将跨 MCP 服务的多种工具组合成可执行的工作流。现有的 MCP 环境扩展方法是数据驱动的：它们首先合成一个数据库和场景，然后从中派生工具和任务。因此，工具接口可能与真实接口不匹配，工具组合并非基于执行，任务也很少跨越多个环境。我们提出 ToolSpace，一个以工具为中心、反转这一顺序的框架。ToolSpace 从真实 MCP 服务和开放智能体语料的执行轨迹中相邻调用之间挖掘数据、状态和重试依赖关系，使用确定性规则而非 LLM 判断，并将得到的加权工具组合图聚类为环境。数据库仅为状态变更型环境合成，而查询类和无状态工具则重放记录的返回值。随后 ToolSpace 执行带回滚的状态感知随机游走，并将每条链反编译为带有基于断言的检查的自然语言任务，仅保留能够从初始状态成功重放的样本。应用于 AutomationBench 和 tau3-Bench Banking 时，该流水线生成更难的 Pro 版本，7 个前沿模型在其上最多下降 38.2 分。在七个长程工具使用基准上，ToolSpace 将 Qwen3.6-35B-A3B 的平均分从 23.0 提升至 41.0（使用 RL 时为 43.2），将 GLM-5.2 从 44.6 提升至 56.6，与 Claude Opus 5 持平。消融实验表明，经执行见证的边明显优于经 LLM 判断的边，且性能随数据和环境规模持续提升。

Large language models are deployed as long-horizon agents that compose diverse tools across MCP services into executable workflows. Existing MCP environment-scaling methods are data-centric: they first synthesize a database and scenario, then derive tools and tasks from them. As a result, tool interfaces may not match real ones, tool compositions are not grounded in execution, and tasks rarely span more than one environment. We propose ToolSpace, a tool-centric framework that reverses this order. ToolSpace mines data, state, and retry dependencies from adjacent calls in execution traces of real MCP services and open agent corpora, using deterministic rules rather than LLM judgment, and clusters the resulting weighted tool-composition graph into environments. Databases are synthesized only for state-mutating environments, while query and stateless tools replay recorded returns. ToolSpace then performs state-aware random walks with rollback and decompiles each chain into a natural-language task with assertion-based checks, keeping only samples that replay successfully from the initial state. Applied to AutomationBench and tau3-Bench Banking, the pipeline produces harder Pro versions on which 7 frontier models drop by up to 38.2 points. Across seven long-horizon tool-use benchmarks, ToolSpace raises Qwen3.6-35B-A3B from 23.0 to 41.0 average score (43.2 with RL) and GLM-5.2 from 44.6 to 56.6, on par with Claude Opus 5. Ablations show that execution-witnessed edges clearly outperform LLM-judged ones, and performance keeps improving with data and environment scale.

📄 arXiv | ⬇ PDF

78 JHiMA-WebRL：面向长时程 Web 任务的联合分层多智能体强化学习框架

JHiMA-WebRL: A Joint Hierarchical Multi-Agent Reinforcement Learning Framework for Long-Horizon Web Tasks

分层Web智能体cs.AI

Shanka Subhra Mondal、Jiayun Wang

👉 联合训练任务分解器与执行器的分层多智能体RL，提升长程网页导航

多模态 Web 智能体已展现出通过图形用户界面（GUI）与网站交互的良好能力，然而现实中的 Web 任务往往需要长时程、多步规划，这对单体策略仍然具有挑战性。此类智能体必须联合推断高层计划、在中间状态间导航、在延长的交互历史中跟踪进度，并执行基于视觉的动作。尽管先前工作已探索了带有独立规划器的模块化多智能体系统，但规划器通常不与执行器联合更新。我们研究显式分离并联合训练规划与执行是否为长时程 Web 导航提供了更强的归纳偏置。我们提出 JHiMA-WebRL，一个面向 Web 智能体的联合分层多智能体强化学习框架，将任务分解器与执行器耦合。为训练分解器，我们使用一个强教师模型从多模态 GUI 轨迹中构建结构化子任务监督，并使用这些标注对分解器和以子任务为条件的执行器进行微调。然后，我们将组相对策略优化（GRPO）扩展为在交互式 WebArena 环境中通过分层 rollout 分组和角色特定优势来联合优化分解器和执行器。在 WebArena-Lite-v2 上，由 7B 执行器和 7B 分解器组成的 JHiMA-WebRL-14B 实现了 37.2% 的平均成功率，而 ScaleCUA-32B 智能体在相同训练任务上用 RL 微调并在相同协议下评估，其成功率为 24%。消融实验表明，在 RL 期间联合更新分解器优于仅更新执行器。这些结果表明，使用较小模型的联合学习角色分解可以成为长时程 Web 任务中扩展单体智能体的有竞争力的替代方案。

Multimodal web agents have shown promising capabilities for interacting with websites through graphical user interfaces (GUIs), yet realistic web tasks often require long-horizon, multi-step planning that remains challenging for monolithic policies. Such agents must jointly infer high-level plans, navigate through intermediate states, track progress over extended interaction histories, and execute visually grounded actions. Although prior work has explored modular multi-agent systems with a separate planner, the planner is typically not updated jointly with the actor. We investigate whether explicitly separating and jointly training planning and execution provides a stronger inductive bias for long-horizon web navigation. We introduce JHiMA-WebRL, a joint hierarchical multi-agent reinforcement learning framework for web agents that couples a task decomposer with an actor. To train the decomposer, we use a strong teacher model to construct structured subtask supervision from multimodal GUI trajectories, and use these annotations to fine-tune both the decomposer and the subtask-conditioned actor. We then extend Group Relative Policy Optimization (GRPO) to jointly optimize the decomposer and actor in the interactive WebArena environment through hierarchical rollout grouping and role-specific advantages. On WebArena-Lite-v2, JHiMA-WebRL-14B, consisting of a 7B actor and a 7B decomposer, achieves an average success rate of 37.2%, compared with 24% for a ScaleCUA-32B agent that we fine-tune with RL on the same training tasks and evaluate under the same protocol. Ablations show that jointly updating the decomposer during RL outperforms updating the actor alone. These results suggest that jointly learned role factorization with smaller models can be a competitive alternative to scaling monolithic agents for long-horizon web tasks.

📄 arXiv | ⬇ PDF

78 一个助手，多种记忆：当召回不足以支撑时，对真实世界群组协助的基准测试

One Assistant, Many Memories: Benchmarking Real-World Group Assistance When Recall Is Not Enough

群聊记忆基准cs.CL cs.AI

Zhanghao Hu、Linhai Zhang、Qian Zhao、Qi Zhu、Yuan Hua、Di Liang、Jiasheng Si、Xin Zhao 等 11 人

👉 真实群聊助手记忆基准，8万消息50群，评测多成员信息整合与冲突跟踪

记忆的价值不仅在于召回，还在于利用记忆提供恰当的协助。近期的基准测试在个性化与智能体任务中考察记忆使用，但很少评估没有固定答案的协助，也很少评估跨多个参与者的整合。群聊使这两项挑战更加严峻：助手必须整合来自多个成员的信息，随时间跟踪修订与冲突，并为群组生成一个回复。然而，现有的群组记忆基准测试依赖合成对话，并继承了通用记忆基准测试中受限的评估方式，侧重于针对固定答案的召回或在封闭集合上做决策。我们提出 GroupAssistBench，这是首个基于真实世界群组对话、并配有已部署群组助手的记忆基准测试，覆盖 50 个群组中的 80,299 条消息，成本超过 5.3 万美元。在相同的历史记录上，GroupAssistBench 用参考答案评估记忆召回（L1），并用没有固定答案的服务请求评估记忆使用（L2），涵盖单一、分布式和冲突记忆（M1--M3）。请求经验证确实需要群组历史，记忆被锚定到其来源，L2 使用量身定制的评分标准，并带有源自真实用户反馈的惩罚项。在对具备完整上下文的前沿模型、检索方法和记忆系统的评估中，最强的记忆系统在召回上可与完整上下文匹敌，但在使用上远远落后，尤其是在分布式和冲突记忆方面。两项新增机制——将检索到的记忆相互关联，以及将记住的事实绑定为回复约束——共同带来了一致的提升，表明群组记忆系统必须改进使用，而不仅仅是召回。

The value of memory lies not only in recall but in using it to provide appropriate assistance. Recent benchmarks test memory use in personalisation and agentic tasks, but seldom evaluate assistance without fixed answers or integration across multiple participants. Group chats make both challenges more acute: an assistant must integrate information from multiple members, track revisions and conflicts over time, and produce one response for the group. Yet existing group memory benchmarks rely on synthetic conversations and inherit constrained evaluation from general memory benchmarks, focusing on recall against fixed answers or decisions over a closed set. We introduce GroupAssistBench, the first memory benchmark from real-world group conversations with a deployed group assistant, covering 80,299 messages in 50 groups and costing over 53K. Over the same histories, GroupAssistBench evaluates memory recall (L1) with reference answers and memory use (L2) with service requests without fixed answers across single, distributed, and conflicting memories (M1--M3). Requests are verified to require group history, memories are anchored to their sources, and L2 uses tailored rubrics with penalties derived from real user feedback. Across evaluations of frontier models with full context, retrieval methods, and memory systems, the strongest memory system rivals full context on recall but falls far behind on use, especially for distributed and conflicting memories. Together, two additions that relate retrieved memories and bind remembered facts as response constraints bring consistent gains, showing that group memory systems must improve use, not recall alone.

📄 arXiv | ⬇ PDF

78 Meta²：为长时程智能体的在线 harness 演化预计算可复用知识

Meta²: Precomputing Reusable Knowledge for Online Harness Evolution in Long-Horizon Agents

在线harness进化cs.AI

Lingzhi Yuan、Yiguang Cai、Weize Liu、Siqi Yang、Feijiang Han、Yongkyun Lee、Udari Madhushani Sehwag、Furong Huang

👉 离线学习可复用Meta-Skill文本知识，在线指导长程agent的harness演化，提升任务适应。

Harness 优化通过改进冻结模型周围的提示、工具、记忆和钩子来提升长时程智能体的表现。离线方法可以从已完成的轨迹中学习，但会部署一个共享的 harness，这可能过拟合训练任务，且无法对新运行过程中显现的证据作出反应。在线演化可以通过演化器在一次运行之前、期间或之后进行适应，但可能不得不从头推理每一次干预，或依赖在测试时通常不可用的验证器反馈。我们提出 Meta²，它将离线优化提升一个层次：不是优化一个共享的 harness，而是学习用于 harness 演化的可复用知识，并在每个任务中在线应用。在离线阶段，Meta² 通过对比同一任务智能体在三种配置下的训练 rollout 来学习一个文本化的元技能 M：无演化器、由当前版本 M 引导的演化器，以及被赋予仅在训练中可用的特权验证器信息的演化器。轨迹和结果上的对比跨任务聚合，以将 M 精炼为关于何时以及如何演化、何时保持 harness 不变以及应避免哪些干预的可复用指导。在在线阶段，演化器将冻结的 M 与新出现的轨迹证据相结合，以决定是保持（KEEP）harness 不变，还是通过任务局部编辑来演化（EVOLVE）它。每次编辑都应用于同一次运行的剩余部分，而不会重置其环境或进度。因此，演化指导可跨任务迁移，而 harness 编辑保持任务局部，且演化器始终无法获得留出的验证器反馈。在两个长时程基准 Terminal-Bench 2.1 和 DeepSWE 1.1 上，Meta² 使用 GPT-5.6-Terra 在所评估的方法中取得了最高性能，分别达到 81.00% 和 44.44%。消融实验支持了对比式演化指导的价值以及保持在线 harness 编辑任务局部的价值。

Harness optimization improves long-horizon agents by refining the prompts, tools, memory, and hooks around a frozen model. Offline methods can learn from completed trajectories, but deploy one shared harness that may overfit training tasks and cannot react to evidence revealed during a new run. Online evolution can adapt before, during, or after one run through an evolver, but may have to reason about every intervention from scratch or rely on verifier feedback that is usually unavailable at test time. We introduce Meta², which moves offline optimization one level up: rather than optimizing a shared harness, it learns reusable knowledge for harness evolution and applies it within each task online. Offline, Meta² learns a textual Meta-Skill M by contrasting training rollouts of the same task agent under three configurations: without an evolver, with an evolver guided by the current version of M, and with an evolver given privileged verifier information available only in training. Contrasts in trajectories and outcomes are aggregated across tasks to refine M into reusable guidance on when and how to evolve, when to leave the harness unchanged, and which interventions to avoid. Online, an evolver combines the frozen M with emerging trajectory evidence to decide whether to KEEP the harness unchanged or EVOLVE it through a task-local edit. Each edit applies to the rest of the same run without resetting its environment or progress. Thus, evolution guidance transfers across tasks while harness edits remain task-local, and held-out verifier feedback is never available to the evolver. On two long-horizon benchmarks, Terminal-Bench 2.1 and DeepSWE 1.1, Meta² achieves the highest performance among evaluated methods with GPT-5.6-Terra, reaching 81.00% and 44.44%, respectively. Ablations support the value of contrastive evolution guidance and keeping online harness edits task-local.

📄 arXiv | ⬇ PDF

78 Signals：AI 智能体的持续诊断

Signals: Continuous Diagnostics of AI Agents

Agent诊断cs.AI

Amey Rambatla、Vatsal Goel、Sagar Gaur、Siddhant Loya、Anurag Mishra、Asif Mohammad

👉 从agent轨迹自动识别45类问题并给修复建议，agent可观测性系统，F1达91%

即使 AI 智能体的工具和基础设施运行正常，它们也可能做出错误决策或浪费资源。其非确定性行为使这些问题难以复现和诊断。我们开发了 Signals，用于从智能体轨迹中识别此类问题并给出具体的修复建议。该方法围绕一个包含八大类、45 种问题类型的分类体系构建，并为每种类型定义了定义与证据要求。该方法区分已发生的失败、风险暴露和低效情况，将相关发现归组为称为 Signals 的工程问题，并依据支撑证据评估其严重程度。每个 Signal 都包含开发者可采取的行动以及用于评估变更的检查项。后续执行可以为已有 Signal 补充证据，或揭示新的问题。我们在公开的智能体执行记录上评估了该方法，在将 Signals 与参考问题匹配时达到 91.43% 的 F1，比使用相同模型和分类体系的单遍 LLM 对照高出约 18 个百分点。

AI agents can make incorrect decisions or waste resources even when their tools and infrastructure work normally. Their nondeterministic behavior makes these problems difficult to reproduce and diagnose. We developed Signals to identify such problems from agentic traces and suggest concrete remediations. We built the method around a taxonomy of 45 problem types across eight families, with definitions and evidence requirements for each type. The method distinguishes realized failures, risk exposures, and inefficiencies, groups related findings into engineering issues called Signals, and assesses their severity from the supporting evidence. Each Signal includes actions a developer can take and checks for evaluating the change. Further executions can add evidence to an existing Signal or reveal a new problem. We evaluated the method on public agent executions and achieved 91.43% F1 when matching Signals to reference issues, about 18 percentage points higher than a one-pass LLM control that used the same model and taxonomy.

📄 arXiv | ⬇ PDF