Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing

Chongbo Zhao1,* Jiangming Wang1,* Xilai Wang2 Xinyu Wang3 Jingyi Tang4 Chunjie Hao5 Pengjie Song6 Yue Ma7,†
1Sun Yat-sen University 2South China University of Technology 3Tsinghua University 4Shandong University 5Nankai University 6Hunan University 7The Hong Kong University of Science and Technology
* Equal contribution.    † Corresponding author.

We propose training-free video editing in the discrete token space of visual autoregressive models, using probability-guided token replacement to eliminate the need for lossy inversion.

GitHub
Showcase of Edit-VAR
Overview

Abstract

Text-guided video editing modifies target content while preserving the appearance and temporal coherence of unedited regions. Training-based approaches provide strong control but demand substantial data and computation. Training-free methods fall into inversion-free and inversion-based paradigms. Inversion-free approaches avoid trajectory recovery, but their source-preserving guidance can limit editing strength and leave semantic changes incomplete. Inversion-based approaches recover a latent trajectory before regeneration, where approximation errors can accumulate and cause source-content drift and temporal inconsistency. We introduce Edit-VAR, the first training-free and inversion-free framework for text-guided video editing with a pretrained visual autoregressive video model. Edit-VAR directly encodes the source video into multi-scale discrete tokens and performs probability-guided conditional token replacement for source preservation. Attention-guided token-wise and scale-aware modulation selectively relaxes source constraints over edit-relevant positions and generation stages, while Scale-Decoupled Generation, implemented as late-scale constraint release, regenerates motion-consistent details and reduces texture fragmentation. Residual-guided token pruning further exploits redundancy at the final two high-resolution scales to reduce inference cost. Extensive experiments and a blind user study demonstrate that Edit-VAR outperforms existing training-free video editing methods overall in editing fidelity, source preservation, temporal coherence, and inference efficiency.
Motivation

Why Discrete Token Space?

A persistent trade-off between editability and source fidelity in existing training-free methods, and our solution.

✗

Existing Training-Free Methods

Inversion-based: map the source onto a latent trajectory then regenerate under the target prompt — approximation errors accumulate across frames, causing source-content drift and temporal inconsistency.
Inversion-free: construct source-to-target transformations without trajectory recovery, but source-preserving guidance can restrict semantic deviation, resulting in incomplete edits.

✓

Edit-VAR (Ours)

Directly encodes the source video into a coarse-to-fine hierarchy of discrete tokens — no iterative trajectory inversion needed. Editing is performed by selectively preserving or replacing encoded tokens, avoiding inversion-induced drift while retaining direct control over source fidelity.

Approach

Framework

Framework Overview
Figure 2. Overview of our framework. In Phase 1, the source video is encoded and a forward pass under the source prompt caches per-token probabilities and cross-attention maps. In Phase 2, a forward pass under the edit prompt performs probability-guided token replacement steered by the attention-guided spatial modulation field, which assigns low γ to edited regions and high γ to preserved regions. Source caching stops at scale Sstop; beyond that, the editing pass generates freely.
Spatial Modulation
Figure 3. Spatial modulation visualization. Given a source frame and an edit-target word, the cross-attention map localizes the editing region, which is then mapped to a per-token gamma field. High attention (editing target) → low γ (allow editing); low attention (background) → high γ (preserve).
Results

Editing Cases

Evaluation

Comparison

Qualitative comparison with training-free and training-based video editing methods.

Analysis

Ablation Study

Validating the contribution of each component.