Rank-Based Evaluation

Implementation of ranked based evaluator.

class RankBasedMetricResults(mean_rank, mean_reciprocal_rank, hits_at_k, adjusted_mean_rank)[source]

Results from computing metrics.

Includes results from:

  • Mean Rank (MR)

  • Mean Reciprocal Rank (MRR)

  • Adjusted Mean Rank (AMR; [berrendorf2020])

  • Hits @ K

class RankBasedEvaluator(ks=None, filtered=True, automatic_memory_optimization=True, **kwargs)[source]

A rank-based evaluator for KGE models.

Calculates:

  • Mean Rank (MR)

  • Mean Reciprocal Rank (MRR)

  • Adjusted Mean Rank (AMR; [berrendorf2020])

  • Hits @ K

Initialize rank-based evaluator.

Parameters

  • ks (Optional[Iterable[Union[int, float]]]) – The values for which to calculate hits@k. Defaults to {1,3,5,10}.

  • filtered (bool) – Whether to use the filtered evaluation protocol. If enabled, ranking another true triple higher than the currently considered one will not decrease the score.