MetisAnchorTokenizer

class MetisAnchorTokenizer(num_partitions=2, device=None, **kwargs)

Bases: AnchorTokenizer

An anchor tokenizer, which first partitions the graph using METIS.

We use the binding by torch_sparse. The METIS graph partitioning algorithm is described here: http://glaros.dtc.umn.edu/gkhome/metis/metis/overview

Initialize the tokenizer.

Parameters:

• num_partitions (int) – the number of partitions obtained through Metis.

• device (Union[str, device, None]) – the device to use for tokenization

• kwargs – additional keyword-based parameters passed to AnchorTokenizer.__init__(). note that there will be one anchor tokenizer per partition, i.e., the vocabulary size will grow respectively.

Methods Summary

__call__(mapped_triples, num_tokens, ...)

Tokenize the entities contained given the triples.

Methods Documentation

__call__(mapped_triples, num_tokens, num_entities, num_relations)

Tokenize the entities contained given the triples.

Parameters:

• mapped_triples (LongTensor) – shape: (n, 3) the ID-based triples

• num_tokens (int) – the number of tokens to select for each entity

• num_entities (int) – the number of entities

• num_relations (int) – the number of relations

Return type:

Tuple[int, LongTensor]

Returns:

shape: (num_entities, num_tokens), -1 <= res < vocabulary_size the selected relation IDs for each entity. -1 is used as a padding token.