RAND proportional open-weight evaluation proposal

The authors propose four approaches: testing without external safeguards, examining robustness of model safeguards, assessing capability amplification and approximating feasible worst-case misuse. They also say weaker models may warrant less exhaustive testing.

Applying such a proposal to mandatory release decisions would raise questions about proportional costs, reproducibility and appeals. The paper discloses operating income and philanthropic support without allocating a named grant to it.

The practical dispute is what counts as the system being evaluated. A hosted product includes controls maintained by its provider; released weights can be used in other configurations. This proposal tries to account for that difference. If authorities adopted such tests as release conditions, the important questions would include the modification budget, the assumed user’s resources, the reproducibility of results and the remedy for a disputed assessment. Those choices determine how much burden falls on a developer and whether a test measures realistic additional risk.

Assessment updated 2026-09-14