Qinzhe Hu

Home / Publications

Publications

Qinzhe Hu is shown in bold; * marks equal contribution. Click a figure to enlarge it.

Architecture of Universal Test-Time Training: an expanded uTTT block with a shared fast-weight pool (uTTT-MoE) or a shared dense fast-weight operator (uTTT-Dense) across blocks.

Universal Test-Time Training

Qinzhe Hu*,Zefan Cai*,Ziqiao Ma,Hao Tan,Junjie Hu
* Equal contribution
COLM 2026 Workshop on Efficient Reasoning SpotlightUnder review at ICLR 2027

Shares test-⁠time-⁠training fast weights across all layers instead of keeping one private state per layer, which unifies time and depth into a single recurrence and improves both long-⁠context retrieval and novel view synthesis at equal state size.

Architecture of Universal Test-Time Training: an expanded uTTT block with a shared fast-weight pool (uTTT-MoE) or a shared dense fast-weight operator (uTTT-Dense) across blocks.

arXivWebsiteCodePoster
Deterministic versus generative readout of a fast-weight memory: a squared-loss read returns the conditional mean, which can fall between plausible values; the generative read samples one of them.

Test-Time Generation

Zefan Cai,Qinzhe Hu,Yuchen Zhu,Hao Tan,Tianyuan Zhang,Sai Bi,Haoyi Qiu,Yicong Hong,Haozhe Zhao,Jason Kuen,Haoliang Wang,Ruiyi Zhang,Ryan A. Rossi,Wanrong Zhu,Molei Tao,Wen Xiao,Junjie Hu,Jiuxiang Gu
Under review at ICLR 2027

Reads a fast-⁠weight memory by generating from it instead of regressing to one answer, so when a key has several stored values the memory can recall each of them rather than their average; this improves long-⁠context retrieval and novel view synthesis at the same state size.

Deterministic versus generative readout of a fast-weight memory: a squared-loss read returns the conditional mean, which can fall between plausible values; the generative read samples one of them.

PDFPoster
Transformer versus LastKV: a standard Transformer keeps a per-layer KV history; LastKV keeps only the last layer's KV of completed chunks and lets every layer of later chunks read it.

Last Layer KV Cache is All You Need

Zefan Cai,Sai Bi,Lin Zhang,…,Qinzhe Hu,…,Jiuxiang Gu,Junjie Hu,Hao Tan
Under review at ICLR 2027

Lets every layer read the last layer's KV cache from earlier chunks, which unifies time and depth into a single recurrence for attention. With long-⁠term KV storage cut to roughly a single layer's worth, it still matches or beats standard Transformers on language modeling, novel view synthesis and video generation, and leads the compared architectures on long-⁠context retrieval and most state-⁠tracking tasks.

Transformer versus LastKV: a standard Transformer keeps a per-layer KV history; LastKV keeps only the last layer's KV of completed chunks and lets every layer of later chunks read it.

Overview of TF-MoE: the mel-band-split Conformer backbone, alternating frequency- and time-wise MoE Conformer blocks, and the sparse MoE feed-forward module with a time/band-wise router.

TF-MoE: Time-Frequency Mixture-of-Experts for Efficient Speech Separation

Qinzhe Hu,Chenda Li,Wangyou Zhang,Shujie Liu,Yan Lu,Yanmin Qian
INTERSPEECH 2026 Oral

A sparse Mixture-⁠of-⁠Experts framework that routes experts over time frames and mel bands, raising the capacity of speech-⁠separation models with almost no extra inference cost, so it outperforms models that need far more compute.

Overview of TF-MoE: the mel-band-split Conformer backbone, alternating frequency- and time-wise MoE Conformer blocks, and the sparse MoE feed-forward module with a time/band-wise router.

arXiv
Overview of SmartThinker's advantage calculation: trajectory sampling, length-reward calculation from the estimated optimal length, and a dynamic reward coefficient that never penalizes correct trajectories.

SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning

Chenzhi Hu,Qinzhe Hu,Yuhang Xu,Junyi Chen,Ruijie Wang,Shengzhong Liu,Jianxin Li,Fan Wu,Guihai Chen
ICML 2026

An RL method for efficient reasoning that lets the model think only as long as each question needs: during training it estimates the best reasoning length per question and pulls over-⁠long answers toward it, cutting reasoning length by up to half while improving accuracy on benchmarks such as AIME25.

Overview of SmartThinker's advantage calculation: trajectory sampling, length-reward calculation from the estimated optimal length, and a dynamic reward coefficient that never penalizes correct trajectories.

arXivCodePoster