MetisAnchorTokenizer
- class MetisAnchorTokenizer(num_partitions=2, device=None, **kwargs)[source]
Bases:
AnchorTokenizer
An anchor tokenizer, which first partitions the graph using METIS.
We use the binding by
torch_sparse
. The METIS graph partitioning algorithm is described here: http://glaros.dtc.umn.edu/gkhome/metis/metis/overviewInitialize the tokenizer.
- Parameters
num_partitions (
int
) – the number of partitions obtained through Metis.device (
Union
[str
,device
,None
]) – the device to use for tokenizationkwargs – additional keyword-based parameters passed to
AnchorTokenizer.__init__()
. note that there will be one anchor tokenizer per partition, i.e., the vocabulary size will grow respectively.
Methods Summary
__call__
(mapped_triples, num_tokens, ...)Tokenize the entities contained given the triples.
Methods Documentation