Tokenizer
- class Tokenizer[source]
Bases:
object
A base class for tokenizers for NodePiece representations.
Methods Summary
__call__
(mapped_triples, num_tokens, ...)Tokenize the entities contained given the triples.
Methods Documentation
- abstract __call__(mapped_triples, num_tokens, num_entities, num_relations)[source]
Tokenize the entities contained given the triples.
- Parameters
- Return type
- Returns
shape: (num_entities, num_tokens), -1 <= res < vocabulary_size the selected relation IDs for each entity. -1 is used as a padding token.