OGBEvaluator

class OGBEvaluator(filtered=False, **kwargs)[source]

Bases: SampledRankBasedEvaluator

A sampled, rank-based evaluator that applies a custom OGB evaluation.

Initialize the evaluator.

Parameters:

  • evaluation_factory – the factory with evaluation triples

  • additional_filter_triples – additional true triples to use for filtering; only relevant if not explicit negatives are given. cf. pykeen.evaluation.rank_based_evaluator.sample_negatives()

  • num_negatives – the number of negatives to sample; only relevant if not explicit negatives are given. cf. pykeen.evaluation.rank_based_evaluator.sample_negatives()

  • head_negatives – shape: (num_triples, num_negatives) the entity IDs of negative samples for head prediction for each evaluation triple

  • tail_negatives – shape: (num_triples, num_negatives) the entity IDs of negative samples for tail prediction for each evaluation triple

  • kwargs – additional keyword-based arguments passed to pykeen.evaluation.rank_based_evaluator.RankBasedEvaluator.__init__()

Raises:

ValueError – if only a single side’s negatives are given, or the negatives are in wrong shape

Methods Summary

evaluate(model, mapped_triples[, ...])

Run evaluate_ogb() with this evaluator.

Methods Documentation

evaluate(model, mapped_triples, batch_size=None, slice_size=None, device=None, use_tqdm=True, tqdm_kwargs=None, restrict_entities_to=None, restrict_relations_to=None, do_time_consuming_checks=True, additional_filter_triples=None, pre_filtered_triples=True, targets=('head', 'tail'))[source]

Run evaluate_ogb() with this evaluator.

Return type:

MetricResults

Parameters:

  • model (Model) –

  • mapped_triples (LongTensor) –

  • batch_size (int | None) –

  • slice_size (int | None) –

  • device (device | None) –

  • use_tqdm (bool) –

  • tqdm_kwargs (Mapping[str, str] | None) –

  • restrict_entities_to (Collection[int] | None) –

  • restrict_relations_to (Collection[int] | None) –

  • do_time_consuming_checks (bool) –

  • additional_filter_triples (None | LongTensor | List[LongTensor]) –

  • pre_filtered_triples (bool) –

  • targets (Collection[Literal['head', 'relation', 'tail']]) –