Question about EK100 action anticipation evaluation

[KirillRed]

Hello! Thank you for releasing the code for this amazing work. I'm currently working on reproducing the action anticipation results on the EPIC-Kitchens-100 dataset, and while tracing the evaluation pipeline I came across something that I wanted to ask about.

In the eval.py file, the variable val_anticipation_point defaults to [0.0, 0.0]:

vjepa2/evals/action_anticipation_frozen/eval.py

Line 96 in 204698b

val_anticipation_point = args_data.get("val_anticipation_point", [0.0, 0.0])

As val_anticipation_point does not appear to be overridden in any of the released action anticipation configs, it seems that [0.0, 0.0] is the value used during evaluation.

However, in the epickitchens.py file, the input clip is generated as

vjepa2/evals/action_anticipation_frozen/epickitchens.py

Lines 131 to 134 in 204698b

	ap = random.uniform(*self.anticipation_point)
	af = int(sf * ap + (1 - ap) * ef - aframes)
	indices = np.arange(af - nframes, af, fstp).astype(np.int64)

where sf and ef are the action start and end frames, respectively, and aframes corresponds to the anticipation time.

With val_anticipation_point = [0.0, 0.0], we always have ap = 0, which gives

af = ef - aframes

As a result, the sampled clip ends 1 second before the action end, rather than 1 second before the action start, which seems different from the standard EPIC-Kitchens action anticipation protocol described in the paper.

From my understanding, it seems that val_anticipation_point would instead need to be [1.0, 1.0] so that the observation clip ends 1 second before the action start.

Could you please let me know whether my understanding is correct, or if I'm overlooking something in the evaluation pipeline?

Thank you very much!

[mnjihun]

Thanks @KirillRed — we reached the same reading independently and can add numbers (audited at 204698b, still the current HEAD).

Code. eval.py (L94–97) defaults val_anticipation_point to [0.0, 0.0], and no released EK100 config overrides it. With ap = 0, the sampler (epickitchens.py, L132) computes

af = int(sf * ap + (1 - ap) * ef - aframes)   # ap=0 → af = stop_frame − 1s

Paper. Appendix 13.1 (arXiv:2506.09985) defines ap = 0 as anchoring at the first frame of the action — context ending 1 s before the action starts, as §6 also describes. The released sampler implements the reverse; a swapped interpolation order in af would explain the mismatch, though we may be misreading the intent.

Effect on EK100 val: 82.8% of samples anchor inside the labeled action (median exposure within the 4 s context: 0.96 s); only 17.2% are strictly anticipatory.

One-line experiment — released ViT-g/384 probe + released inference config, full val (9,296 train-filtered segments), the only functional diff being one added line:

val_anticipation_point	joint top-5 acc	class-mean R@5
[0.0, 0.0] (released default → end − 1 s)	64.0%	37.9% — within 2 pp of the reported 39.7
[1.0, 1.0] (start − 1 s)	28.5%	13.5%

— consistent with the probe itself being trained end-anchored (train_anticipation_point ∈ [0, 0.25] supervising the same row).

We implemented a strictly-before-start EK100 path (targets shifted one row within each video; overlapping pairs removed → 62,896 train / 8,915 val) and would be glad to contribute it as a PR. Context: we hit this adapting V-JEPA 2 to strict next-action prediction on Ego4D GoalStep, where actions are longer (median 12.6 s) and end-anchoring would overlap the target for ~99.9% of samples.