[damienen]

Fixes #418

Problem

With a dict gold answer, question_scorer raised instead of returning a score:

• A prediction of {} makes calc_recall average an empty list. The NaN reaches the alignment step, which raises ValueError: matrix contains invalid numeric entries.
• A prediction like [{"sender": "USPS", ...}, "maybe FedEx"] passes the type(pred[0]) == dict check in evaluate_dicts, then .get is called on the string and raises AttributeError.

Change

• calc_recall returns 0 when there is nothing to compare.
• evaluate_dicts requires every element of a list prediction to be a dict, and scores 0 otherwise, as it already did when the first element was not a dict.

Testing

Ran the real question_scorer with the dict gold from the dev set in a Docker sandbox with numpy and scipy, on main at 27a90ce815098f43414e93491e06907af468e45b.

• Without the change (exit 1): {} raises ValueError, the mixed list raises AttributeError, the matching answer scores 1.0.
• With the change (exit 0): {} scores 0.0, the mixed list scores 0, the matching answer still scores 1.0.

tests/assistantbench/test_evaluation.py needs gymnasium and the full browsergym install, so I did not run it.