validation

Use mouse to pan and zoom

Use mouse to pan and zoom

Shared validation helpers

FUNCTION	DESCRIPTION
ensure_unique_metric_result_keys	Ensure that all metric specs resolve to unique final result keys.

Classes

Functions

ensure_unique_metric_result_keys

ensure_unique_metric_result_keys(metric_specs: Sequence[MetricSpec]) -> None

Ensure that all metric specs resolve to unique final result keys.

PARAMETER	DESCRIPTION
metric_specs	Metric specs to validate. TYPE: Sequence[MetricSpec]

RAISES	DESCRIPTION
DuplicateMetricResultKeyError	If duplicate final result keys are detected.

Source code in src/evallm/application/validation.py

def ensure_unique_metric_result_keys(metric_specs: Sequence[MetricSpec]) -> None:
    """Ensure that all metric specs resolve to unique final result keys.

    Args:
        metric_specs (Sequence[MetricSpec]): Metric specs to validate.

    Raises:
        DuplicateMetricResultKeyError: If duplicate final result keys are detected.
    """
    counts: Counter[str] = Counter(
        spec.result_key if spec.result_key is not None else spec.name
        for spec in metric_specs
    )
    if len(duplicates := [key for key, count in counts.items() if count > 1]) > 0:
        raise DuplicateMetricResultKeyError(
            duplicate_keys=duplicates,
            metric_names=[spec.name for spec in metric_specs],
        )