Earth-observation foundation models (EOFMs) are commonly evaluated using lightweight probes over frozen representations, so one encoding pass can support repeated downstream training across tasks and regions. Yet geographic generalization is typically summarized by a single in-distribution to out-of-distribution (ID-OOD) gap, with a probe trained on source regions and evaluated on held-out regions. This summary is sensitive to evaluation design and cannot tell deployers how to mitigate geographic transfer failures. We introduce ARVA (Agricultural Robustness across Varied Areas), an evaluation protocol for frozen EOFM representations. On the same target-region test examples, ARVA compares probes trained with no, limited, or only target-region labels while varying split construction, reporting per-region performance, and including a raw-input control. Replacing one quarter of a fixed source-label budget with target-region labels improves every evaluated representation and raw-input setting by 1.6-14.8 points. Split construction changes performance by as much as 21 points, and near-identical ID-OOD gaps can correspond to more than twofold differences in the gain from target labels. Benchmark averages can also conceal near-zero regional performance, while learned representations do not consistently outperform raw inputs under the same probe procedure. We release ARVA as a reusable benchmark with standardized geographic splits, cached embeddings, and an evaluation harness for reproducing the label-access, calibration, split-sensitivity, and worst-region results.