Tencent’s AngelSpec, an open-weight framework, optimizes LLM inference through specialized multi-token prediction and block-parallel speculative decoding.
Tencent open sourced AngelSpec, a training framework designed for speculative-decoding draft models covering both multi-token prediction and block-parallel architectures. Seven pre-trained draft model checkpoints were released on Hugging Face and ModelScope under open-weight terms for use with Tencent’s Hunyuan models. AngelSpec treats workload domain differences as a core design parameter, using autoregressive drafters for open-ended conversational traffic and block-parallel diffusion drafters for syntax-constrained code and mathematics tasks.
AngelSpec addresses training and inference discrepancies in recurrent multi-token prediction by employing a shared-parameter scheme that unrolls predictions during training and uses generated predictions rather than reference tokens. To prevent draft training from degrading target output quality, the target model backbone and output head remain frozen while training on generated rollouts. These training techniques increased average accepted token length on Hunyuan models, while deeper token position acceptance rates improved significantly.
AngelSpec introduces DFly, an enhanced block-diffusion drafting architecture featuring a hybrid target-conditioning backbone that combines a shared layer with per-layer residual fusion weights. DFly incorporates a small sequential head following the parallel backbone to prevent acceptance decay without incurring heavy computational costs.
On testing benchmarks, DFly achieved a higher mean accepted sequence length than earlier drafting architectures. On larger Hunyuan target models, DFly delivered up to a more than two-fold inference speedup over standard autoregressive decoding across varying concurrency levels.
AngelSpec utilizes a disaggregated system architecture where inference worker processes capture internal state data and transfer it to training nodes via high-speed network connections. Long-context training is enabled through sequence parallelism across long contexts, supported by document-aware packing mechanisms. Training efficiency is further enhanced by compiling rollout operations into optimized kernels to keep memory consumption near that of a standard processing pass.
