# vid2vid-long: Consistent Video-to-Video Translation via Explicit Correspondences

> Plain-text version of the vid2vid-long project page, for LLM agents. It holds the content of both versions of the page (TL;DR and Overview) in one file. For all details, read the paper. Text marked [placeholder] is not written yet.

## Metadata

- Title: Consistent Video-to-Video Translation via Explicit Correspondences
- Short name: vid2vid-long
- Venue: ECCV 2026 (European Conference on Computer Vision)
- Authors:
  - Gaurav Parmar (Carnegie Mellon University, Adobe Research) - https://gauravparmar.com/
  - Zhengqi Li (Adobe Research) - https://zhengqili.github.io/
  - Richard Zhang (Adobe Research) - https://richzhang.github.io/
  - Jun-Yan Zhu (Carnegie Mellon University) - https://www.cs.cmu.edu/~junyanz/
  - Srinivasa Narasimhan (Carnegie Mellon University) - https://www.cs.cmu.edu/~srinivas/
  - Eli Shechtman (Adobe Research) - https://research.adobe.com/person/eli-shechtman/
  - Yotam Nitzan (Adobe Research) - https://yotamnitzan.github.io/
- Paper: [placeholder]

## Page versions

The HTML page shows one version at a time. The URL hash selects the version:

- no hash - landing cover: title, authors, the example videos, and links to the versions below
- #tldr - TL;DR, about 15 seconds
- #overview - Overview, about 5 minutes
- "Deep dive" is not a version of the page: it will be a link to the paper on arXiv. The paper is not on arXiv yet [placeholder: the arXiv URL]

## TL;DR

We turn a text-to-video model into a fast, causal video-to-video model that stays consistent over long durations.

- Shown on the page: this sentence, then one input video (RGB, with its depth as a small inset) next to a 2 x 2 grid of outputs. All videos play in sync for the full 60 seconds. A clock shows the time of the videos. The key words "fast", "causal" and "long durations" have an underline.
- Example: one input video with its depth, one text prompt for each output. Each output keeps the scene consistent for the full 60 seconds.
  - Input RGB video (60 s, 832x480): videos/gazebo_input_rgb.mp4
  - Input depth video (60 s, 832x480): videos/gazebo_input_depth.mp4
  - Output 1: videos/gazebo_out_sunset.mp4
    Prompt: "As the sun sets, the gazebo is enveloped in the warm, golden hues of twilight, with the lattice casting dramatic patterns on the grass as fireflies begin to dance."
  - Output 2: videos/gazebo_out_snow.mp4
    Prompt: "Wrapped in a blanket of snow, the gazebo appears almost ethereal, its lattice work dusted white, standing out against the dark, bare branches of the surrounding trees."
  - Output 3: videos/gazebo_out_blossom.mp4
    Prompt: "In full spring bloom, the gazebo is surrounded by pink cherry blossoms, with petals drifting through the air and settling on the roof and the grass under a clear blue sky."
  - Output 4: videos/gazebo_out_rain.mp4
    Prompt: "Under a heavy night rain, the gazebo glistens in the glow of nearby lamps, its roof streaming with water while puddles on the path reflect the dark, stormy sky."

## Overview

### 01 Problem

Autoregressive distillation methods (Self-Forcing, arXiv:2506.08009; LongLive, arXiv:2509.22622; Rolling Forcing, arXiv:2509.25161) make video diffusion models fast and streaming, but their causal attention covers only a context window of recent frames. Once earlier content leaves the window, the model cannot condition on it, and revisited regions are generated inconsistently.

- Shown on the page, as an animated sequence in the same format as section 02: the output of the baseline plays along a filmstrip (frames at 0:02, 0:05, 0:08, 0:12 and 0:15). A bracket marked "Context window" shows the context window of the model (about 5 s) and moves with the video. The example is a 16 s video of a dog made of lego bricks that runs in a desert: the dog runs out of the frame at about 0:05 and comes back at about 0:11. When it comes back, its first visit is outside the window, so the model gives it different colors: the crop at 0:02 shows a green head, and the crop at 0:15 shows a brown, yellow and white head ("A different dog").
- The baseline video: videos/problem_baseline.mp4. It is our model with the correspondence turned off, so it has only its context window. Prompt: "a dog made out of lego bricks running in the desert". From 0:05 to 0:11, while the dog is away, the prompt is "an empty desert landscape with sand dunes". Depth strength 0.75. The input is a storyboard video (sb1_dog_seed5).

### 02 Main insight

Unlike text-to-video generation, video-to-video translation has an input video. Patches that correspond across input frames tell the model which earlier outputs to reuse, even far outside the context window.

If a spot is at the same position in the input video at two different moments, the same position in the output video must look the same at both moments too. So the input video tells the model which earlier outputs to reuse.

- Shown on the page, as an animated sequence: (1) the input video plays as one live video that glides along a filmstrip at the top; each frame of the strip (one every 12 s, 0:05 to 0:53) stays behind as the video passes it; (2) a box marks the same roof spot in the two revisit frames (the camera sees the same view at 0:05 and 0:53); the rest of each frame stays faded, and lines from the corners of each box go to an enlarged crop of the spot below it; the two crops are marked "Matching input regions"; (3) the output video plays the same way, along a filmstrip at the bottom; (4) the same boxes open in the output, with crops between the strips marked "Matching output regions". An arrow marked "imply" goes from the input label to the output label, so the figure reads: matching input regions imply matching output regions. The sequence starts when the reader reaches the figure, plays once and then stays on the final figure.

### 03 Method

We keep a compressed history of input-output patch pairs. For each new block, every input patch retrieves the outputs of its nearest stored inputs, and a correspondence cross-attention layer conditions the model on them.

The page tells the method as a story in 7 steps, for one moment of the example video. On the page each step has only a short phrase (listed after the steps); the full explanation of each step is here:

1. Setup: Consider one moment of a long video. We are at 0:53, and the next block of input frames comes in.
2. Setup: The model already made the output for all earlier blocks. It kept a compressed history of them: pairs of an input patch (the key) and the output patch that the model made for it (the value).
3. Read: We patchify and flatten the new input: we cut it into patches, and each patch is one token of the model.
4. Read: Each input patch is a query. For each query, we find the closest K matches among the keys of the history bank. The page shows K = 1 for illustration: the closest match of the roof patch is the same roof, seen at 0:05.
5. Read: The value of that pair is the output patch that the model made for this roof at 0:05. We retrieve it. Each token retrieves its own k values in this way.
6. Generate: Now the model makes the output, with the depth as control. Correspondence cross-attention lets each token attend to its retrieved patches, so the roof looks as it did at 0:05.
7. Write: Last, we update the history. A pair goes in only if its input patch is novel. The roof is already there, so we skip it. The history grows with new content, not with video length.

- The short phrases on the page, step by step: (1) "Let's consider the input at t = 0:53." (2) "We maintain a history bank of unique key-value pairs from the past." (3) "Input is patchified and flattened." (4) "For each input query, we find the closest K matches in the history bank." with the footnote "K = 1 for illustration." below the figure (5) "We retrieve its value." (6) "The model conditions on the retrieved patches to generate the output." (7) "New key-value pairs are added to the history bank, only if they are unique."
- Shown on the page: one scene with "Back" and "Next" buttons. Nothing plays by itself; step 1 only fades in when the reader reaches the scene. Each step adds one thing to the scene, with a transition, and shows its short phrase on the scene, next to that thing: (1) a time line with "now 0:53", the input frame and its depth frame; (2) the text, then the empty history bank, then the pairs of real patches (input patch -> output patch), faded; (3) the input frame breaks into a 5 x 3 grid of patches, and the patches go into one column of small tokens next to the frame; (4) the text, then one token (the roof) grows in its place in the column as the query and the other tokens fade, then a line goes to the closest match in the bank (the roof at 0:05), then a larger copy of that pair comes out of the bank; (5) the key of that copy fades, and its value is marked as retrieved; (6) the video diffusion model with correspondence cross-attention, a line from the depth frame to the model (the depth control), and the output frame come in; (7) the text, then two new pairs of this block come together next to the bank (each key from the token column, each value from the output frame), then the pair that is already in the bank (the roof) fades with the label "Redundant", then the pair that is new goes into the empty slot of the bank. Input patches (keys) are teal; output patches (values) are orange.
- The pairs in the history on the page are an illustration, not the history of a real run.

### 04 Results

Our evaluation targets long-range consistency: we construct videos in which scenes repeat, and report the reconstruction error at each revisit. We apply our method to three different distillation methods: Self-Forcing (arXiv:2506.08009), LongLive (arXiv:2509.22622), and Rolling Forcing (arXiv:2509.25161). Our correspondence mechanism consistently improves long-context reconstruction across all three. The base model used is VACE 1.3B.

Chart: long-context reconstruction error, measured with LPIPS (lower is better), without and with our correspondence mechanism (Table 3 of the paper, LPIPS columns).

| Benchmark | Distillation method | Without correspondence | With correspondence | Change |
|---|---|---|---|---|
| VACE-Bench | Self-Forcing | 0.548 | 0.343 | -37% |
| VACE-Bench | LongLive | 0.511 | 0.272 | -47% |
| VACE-Bench | Rolling Forcing | 0.427 | 0.240 | -44% |
| DL3DV | Self-Forcing | 0.616 | 0.451 | -27% |
| DL3DV | LongLive | 0.534 | 0.345 | -35% |
| DL3DV | Rolling Forcing | 0.445 | 0.305 | -32% |

- Shown on the page: a bar chart with one panel for each benchmark. Each method has a pair of bars: the bar "without correspondence" is solid and faded, and the bar "with correspondence" has diagonal lines. An arrow goes from the first bar to the second, with the decrease in percent. The two exact values show on hover and on focus.
- Below the chart: the sentence "Below, we show a gallery of results generated with our method.", then the gallery.
- The gallery: all 20 results of the supplement (58 videos, 832x480, in videos/gallery/). On the page it is a viewer that shows one result at a time: the input video with its control video as a small inset, an arrow, the output video, and the text prompt below. The reader picks a result with "Back" / "Next" or with a strip of 20 small pictures. The page loads only the three videos of the result that shows. There are two column labels ("Input" and "Output") and no group titles; the groups below are only for this file.
  - DL3DV, depth control:
    - Result 1. Prompt: "The lush tropical garden glows under the soft, diffused light of an overcast sky, casting gentle shadows on the vibrant greenery and making the colors appear richer and more vivid." Input: videos/gallery/dl3dv_depth_input_videos_0492bc707e62fa7a_loop.mp4 Control: videos/gallery/dl3dv_depth_ctrl_processed_0492bc707e62fa7a_depthv2.mp4 Output: videos/gallery/dl3dv_depth_out_0492bc707e62fa7a_depthv2_s0.5_cap0.mp4
    - Result 2. Prompt: "As the sun sets, the gazebo is enveloped in the warm, golden hues of twilight, with the lattice casting dramatic patterns on the grass as fireflies begin to dance." Input: videos/gallery/dl3dv_depth_input_videos_0a78c25f77c1ba1d_loop.mp4 Control: videos/gallery/dl3dv_depth_ctrl_processed_0a78c25f77c1ba1d_depthv2.mp4 Output: videos/gallery/dl3dv_depth_out_0a78c25f77c1ba1d_depthv2_s0.5_cap2.mp4
    - Result 3. Prompt: "Wrapped in a blanket of snow, the gazebo appears almost ethereal, its lattice work dusted white, standing out against the dark, bare branches of the surrounding trees." Input: videos/gallery/dl3dv_depth_input_videos_0a78c25f77c1ba1d_loop.mp4 Control: videos/gallery/dl3dv_depth_ctrl_processed_0a78c25f77c1ba1d_depthv2.mp4 Output: videos/gallery/dl3dv_depth_out_0a78c25f77c1ba1d_depthv2_s0.75_cap4.mp4
    - Result 4. Prompt: "A single colorful chair with a vibrant, iridescent design sits on a shiny floor in a room that appears to be a classroom or meeting space, featuring multiple tables and chairs arranged around them." Input: videos/gallery/dl3dv_depth_input_videos_44c36f1b5346479d_loop.mp4 Control: videos/gallery/dl3dv_depth_ctrl_processed_44c36f1b5346479d_depthv2.mp4 Output: videos/gallery/dl3dv_depth_out_44c36f1b5346479d_depthv2_s0.75_cap_nosty.mp4
    - Result 5. Prompt: "The classroom glows warmly during the golden hour, casting long shadows through the windows and softening the fluorescent lights." Input: videos/gallery/dl3dv_depth_input_videos_484c0aca40c1087f_loop.mp4 Control: videos/gallery/dl3dv_depth_ctrl_processed_484c0aca40c1087f_depthv2.mp4 Output: videos/gallery/dl3dv_depth_out_484c0aca40c1087f_depthv2_cap0.mp4
    - Result 6. Prompt: "In the gentle moonlight of a clear night, the greenhouse is softly illuminated by the glow of distant streetlights filtering through the glass, casting long shadows and lending a dreamlike quality to the arrangement of plants and decor." Input: videos/gallery/dl3dv_depth_input_videos_493816813d2d6d24_loop.mp4 Control: videos/gallery/dl3dv_depth_ctrl_processed_493816813d2d6d24_depthv2.mp4 Output: videos/gallery/dl3dv_depth_out_493816813d2d6d24_depthv2_s0.5_cap4.mp4
    - Result 7. Prompt: "Under the soft glow of a golden hour sunset, the red lanterns cast warm, elongated shadows across the shelves of vibrant merchandise, bathing the space in a cozy, inviting light." Input: videos/gallery/dl3dv_depth_input_videos_5a060a54bcf64dcd_loop.mp4 Control: videos/gallery/dl3dv_depth_ctrl_processed_5a060a54bcf64dcd_depthv2.mp4 Output: videos/gallery/dl3dv_depth_out_5a060a54bcf64dcd_depthv2_s0.5_cap0.mp4
  - DL3DV, scribble control:
    - Result 8. Prompt: "Snowflakes gently fall, covering the inscriptions in a delicate layer of white, while the stark contrast of the stone against the snow enhances the solemnity of the setting." Input: videos/gallery/dl3dv_scribble_input_videos_d219b7e1a06d24d5_loop.mp4 Control: videos/gallery/dl3dv_scribble_ctrl_processed_d219b7e1a06d24d5_scribble.mp4 Output: videos/gallery/dl3dv_scribble_out_d219b7e1a06d24d5_scribble_cap4.mp4
    - Result 9. Prompt: "A cozy living room features a beige sofa adorned with patterned pillows, a matching armchair draped with a blanket, and a small table with a decorative cushion on a white carpet, all illuminated by natural light streaming through the window blinds." Input: videos/gallery/dl3dv_scribble_input_videos_afaad3e729e8f497_loop.mp4 Control: videos/gallery/dl3dv_scribble_ctrl_processed_afaad3e729e8f497_scribble.mp4 Output: videos/gallery/dl3dv_scribble_out_afaad3e729e8f497_scribble_cap_nosty.mp4
  - DL3DV, flow control:
    - Result 10. Prompt: "The classroom glows warmly during the golden hour, casting long shadows through the windows and softening the fluorescent lights." Input: videos/gallery/dl3dv_flow_input_videos_484c0aca40c1087f_loop.mp4 Control: videos/gallery/dl3dv_flow_ctrl_processed_484c0aca40c1087f_flow.mp4 Output: videos/gallery/dl3dv_flow_out_484c0aca40c1087f_flow_cap0.mp4
    - Result 11. Prompt: "Under the dim, flickering light of a cloudy afternoon, the room feels somber and mysterious, with shadows playing over the tiled surfaces and the stained-glass windows appearing muted and melancholic." Input: videos/gallery/dl3dv_flow_input_videos_6c876758f641c503_loop.mp4 Control: videos/gallery/dl3dv_flow_ctrl_processed_6c876758f641c503_flow.mp4 Output: videos/gallery/dl3dv_flow_out_6c876758f641c503_flow_cap1.mp4
  - Minecraft:
    - Result 12. Prompt: "The meadow glows under the soft, golden hues of a late afternoon sunset, casting long shadows and a warm, serene ambiance." Input: videos/gallery/minecraft_input_left_right_seed_15.mp4 Control: videos/gallery/minecraft_ctrl_processed_left_right_seed_15_src_video_depthv2.mp4 Output: videos/gallery/minecraft_out_left_right_seed_15_depthv2_s0.5_cap0.mp4
    - Result 13. Prompt: "Harsh sunlight pierces through the scattered clouds, casting sharp shadows on the sand and highlighting the vibrant greens of the surrounding vegetation." Input: videos/gallery/minecraft_input_left_right_seed_18.mp4 Control: videos/gallery/minecraft_ctrl_processed_left_right_seed_18_src_video_depthv2.mp4 Output: videos/gallery/minecraft_out_left_right_seed_18_depthv2_cap4.mp4
    - Result 14. Prompt: "The rolling hills are bathed in the soft, warm glow of a golden hour sunset, casting long shadows across the lush green landscape." Input: videos/gallery/minecraft_input_left_right_seed_2.mp4 Control: videos/gallery/minecraft_ctrl_processed_left_right_seed_2_src_video_depthv2.mp4 Output: videos/gallery/minecraft_out_left_right_seed_2_depthv2_s0.75_cap0.mp4
    - Result 15. Prompt: "A heavy blanket of gray clouds dulls the vibrant greens, casting long shadows across the meadow as raindrops bead on the petals of wildflowers, and the river reflects the somber, overcast sky." Input: videos/gallery/minecraft_input_left_right_seed_25.mp4 Control: videos/gallery/minecraft_ctrl_processed_left_right_seed_25_src_video_depthv2.mp4 Output: videos/gallery/minecraft_out_left_right_seed_25_depthv2_cap1.mp4
    - Result 16. Prompt: "Snowflakes gently cover the green field, transforming the wildflowers into delicate snow-capped treasures, while the tree line is outlined in white, creating a serene winter wonderland under a pale, diffused daylight." Input: videos/gallery/minecraft_input_left_right_seed_32.mp4 Control: videos/gallery/minecraft_ctrl_processed_left_right_seed_32_src_video_depthv2.mp4 Output: videos/gallery/minecraft_out_left_right_seed_32_depthv2_s0.75_cap4.mp4
    - Result 17. Prompt: "A blanket of fog hugs the ground, obscuring the lower parts of the hill while the upper sections peek through, creating an ethereal landscape where only the tops of the tallest flowers and shrubs are visible." Input: videos/gallery/minecraft_input_left_right_seed_49.mp4 Control: videos/gallery/minecraft_ctrl_processed_left_right_seed_49_src_video_depthv2.mp4 Output: videos/gallery/minecraft_out_left_right_seed_49_depthv2_cap2.mp4
  - Single-object videos (storyboard):
    - Result 18. Prompt: "a dog made out of lego bricks running in the desert" Input: videos/gallery/storyboard_input_videos_sb1_dog_seed5.mp4 Control: videos/gallery/storyboard_ctrl_videos_sb1_dog_seed5_depth.mp4 Output: videos/gallery/storyboard_out_sb1_dog_seed5_depthv2_cap2.mp4
    - Result 19. Prompt: "a bird flying near a nest" Input: videos/gallery/storyboard_input_videos_sb2_bird_seed1.mp4 Control: videos/gallery/storyboard_ctrl_videos_sb2_bird_seed1_depth.mp4 Output: videos/gallery/storyboard_out_sb2_bird_seed1_depthv2_cap_nosty.mp4
    - Result 20. Prompt: "a lego rocket flying in outer space" Input: videos/gallery/storyboard_input_videos_sb4_rocket_seed1.mp4 Control: videos/gallery/storyboard_ctrl_videos_sb4_rocket_seed1_depth.mp4 Output: videos/gallery/storyboard_out_sb4_rocket_seed1_depthv2_cap0.mp4

## Citation

```bibtex
@inproceedings{parmar2026consistent,
  title={Consistent Video-to-Video Translation via Explicit Correspondences},
  author={Parmar, Gaurav and Li, Zhengqi and Zhang, Richard and Zhu, Jun-Yan and Narasimhan, Srinivasa and Shechtman, Eli and Nitzan, Yotam},
  booktitle={European Conference on Computer Vision},
  pages={39--57},
  year={2026},
  organization={Springer}
}
```