RankBasedEvaluator
- class RankBasedEvaluator(ks=None, filtered=True, **kwargs)[source]
Bases:
pykeen.evaluation.evaluator.Evaluator
A rank-based evaluator for KGE models.
Calculates the following metrics:
Mean Rank (MR) with range \([1, \infty)\) where closer to 0 is better
Adjusted Mean Rank (AMR; [berrendorf2020]) with range \((0, 2)\) where closer to 0 is better
Adjusted Mean Rank Index (AMRI; [berrendorf2020]) with range \([-1, 1]\) where closer to 1 is better
Mean Reciprocal Rank (MRR) with range \((0, 1]\) where closer to 1 is better
Hits @ K with range \([0, 1]\) where closer to 1 is better.
- berrendorf2020(1,2)
Berrendorf, et al. (2020) Interpretable and Fair Comparison of Link Prediction or Entity Alignment Methods with Adjusted Mean Rank.
Initialize rank-based evaluator.
- Parameters
ks (
Optional
[Iterable
[Union
[int
,float
]]]) – The values for which to calculate hits@k. Defaults to {1,3,5,10}.filtered (
bool
) – Whether to use the filtered evaluation protocol. If enabled, ranking another true triple higher than the currently considered one will not decrease the score.kwargs – Additional keyword arguments that are passed to the base class.
Methods Summary
finalize
()Compute the final results, and clear buffers.
process_head_scores_
(hrt_batch, true_scores, ...)Process a batch of triples with their computed head scores for all entities.
process_tail_scores_
(hrt_batch, true_scores, ...)Process a batch of triples with their computed tail scores for all entities.
Methods Documentation
- process_head_scores_(hrt_batch, true_scores, scores, dense_positive_mask=None)[source]
Process a batch of triples with their computed head scores for all entities.
- Parameters
hrt_batch (
LongTensor
) – shape: (batch_size, 3)true_scores (
FloatTensor
) – shape: (batch_size)scores (
FloatTensor
) – shape: (batch_size, num_entities)dense_positive_mask (
Optional
[FloatTensor
]) – shape: (batch_size, num_entities) An optional binary (0/1) tensor indicating other true entities.
- Return type
- process_tail_scores_(hrt_batch, true_scores, scores, dense_positive_mask=None)[source]
Process a batch of triples with their computed tail scores for all entities.
- Parameters
hrt_batch (
LongTensor
) – shape: (batch_size, 3)true_scores (
FloatTensor
) – shape: (batch_size)scores (
FloatTensor
) – shape: (batch_size, num_entities)dense_positive_mask (
Optional
[FloatTensor
]) – shape: (batch_size, num_entities) An optional binary (0/1) tensor indicating other true entities.
- Return type