AnchorTokenizer

class AnchorTokenizer(selection=None, selection_kwargs=None, searcher=None, searcher_kwargs=None)

Bases: Tokenizer

Tokenize entities by representing them as a bag of anchor entities.

The entities are chosen by shortest path distance.

Initialize the tokenizer.

Parameters:

• selection (Union[str, AnchorSelection, Type[AnchorSelection], None]) – the anchor node selection strategy.

• selection_kwargs (Optional[Mapping[str, Any]]) – additional keyword-based arguments passed to the selection strategy

• searcher (Union[str, AnchorSearcher, Type[AnchorSearcher], None]) – the component for searching the closest anchors for each entity

• searcher_kwargs (Optional[Mapping[str, Any]]) – additional keyword-based arguments passed to the searcher

Methods Summary

__call__(mapped_triples, num_tokens, ...)

Tokenize the entities contained given the triples.

Methods Documentation

__call__(mapped_triples, num_tokens, num_entities, num_relations)

Tokenize the entities contained given the triples.

Parameters:

• mapped_triples (LongTensor) – shape: (n, 3) the ID-based triples

• num_tokens (int) – the number of tokens to select for each entity

• num_entities (int) – the number of entities

• num_relations (int) – the number of relations

Return type:

Tuple[int, LongTensor]

Returns:

shape: (num_entities, num_tokens), -1 <= res < vocabulary_size the selected relation IDs for each entity. -1 is used as a padding token.