Representation
- class Representation(max_id, shape=64, normalizer=None, normalizer_kwargs=None, regularizer=None, regularizer_kwargs=None, dropout=None, unique=None)[source]
Bases:
Module
,ExtraReprMixin
,ABC
A base class for obtaining representations for entities/relations.
A representation module maps integer IDs to representations, which are tensors of floats.
max_id defines the upper bound of indices we are allowed to request (exclusively). For simple embeddings this is equivalent to num_embeddings, but more a more appropriate word for general non-embedding representations, where the representations could come from somewhere else, e.g. a GNN encoder.
shape describes the shape of a single representation. In case of a vector embedding, this is just a single dimension. For others, e.g.
pykeen.models.RESCAL
, we have 2-d representations, and in general it can be any fixed shape.We can look at all representations as a tensor of shape (max_id, *shape), and this is exactly the result of passing indices=None to the forward method.
We can also pass multi-dimensional indices to the forward method, in which case the indices’ shape becomes the prefix of the result shape: (*indices.shape, *self.shape).
Initialize the representation module.
- Parameters:
max_id (
int
) – The maximum ID (exclusively). Valid Ids reach from 0, …, max_id-1shape (
Union
[int
,Sequence
[int
]]) – The shape of an individual representation.normalizer (
Union
[str
,Callable
[[FloatTensor
],FloatTensor
],Type
[Callable
[[FloatTensor
],FloatTensor
]],None
]) – A normalization function, which is applied to the selected representations in every forward pass.normalizer_kwargs (
Optional
[Mapping
[str
,Any
]]) – Additional keyword arguments passed to the normalizerregularizer (
Union
[str
,Regularizer
,Type
[Regularizer
],None
]) – An output regularizer, which is applied to the selected representations in forward passregularizer_kwargs (
Optional
[Mapping
[str
,Any
]]) – Additional keyword arguments passed to the regularizerdropout (
Optional
[float
]) – The optional dropout probabilityunique (
Optional
[bool
]) – whether to optimize for calculating representations for same indices only once. This is only useful if the calculation of representations is either significantly more expensive than an index-based lookup and duplicate indices are expected, e.g., when using negative sampling and large batch sizes
Attributes Summary
Return the device.
Methods Summary
forward
([indices])Get representations for indices.
Iterate over components for
extra_repr()
.Apply constraints which should not be included in gradients.
Reset the module's parameters.
Attributes Documentation
Methods Documentation
- forward(indices=None)[source]
Get representations for indices.
Note
depending on
Representation.unique
, this implementation will use an optimization for duplicate indices. It is generally only recommended if computing individual representation is expensive, e.g., since it involves message passing, or a large encoder networks, but discouraged for cheap lookups, e.g., a plain embedding lookup.- Parameters:
indices (
Optional
[LongTensor
]) – shape: s The indices, or None. If None, this is interpreted astorch.arange(self.max_id)
(although implemented more efficiently).- Return type:
FloatTensor
- Returns:
shape: (
*s
,*self.shape
) The representations.