LowRankRepresentation¶
- class LowRankRepresentation(*, max_id, shape, num_bases=3, weight_initializer=<pykeen.utils.compose object>, **kwargs)[source]¶
Bases:
Representation
Low-rank embedding factorization.
This representation reduces the number of trainable parameters by not learning independent weights for each index, but rather having shared bases among all indices, and only learn the weights of the linear combination.
\[E[i] = \sum_k B[i, k] * W[k]\]Initialize the representations.
- Parameters:
max_id (
int
) – the maximum ID (exclusively). Valid Ids reach from 0, …, max_id-1shape (
Union
[int
,Sequence
[int
]]) – the shape of an individual base representation.num_bases (
int
) – the number of bases. More bases increase expressivity, but also increase the number of trainable parameters.weight_initializer (
Callable
[[FloatTensor
],FloatTensor
]) – the initializer for basis weightskwargs – additional keyword based arguments passed to
pykeen.nn.representation.Embedding
, which is used for the base representations.
Attributes Summary
Return the number of bases.
Methods Summary
approximate
(other, **kwargs)Construct a low-rank approximation of another representation.
Reset the module's parameters.
Attributes Documentation
Methods Documentation
- classmethod approximate(other, **kwargs)[source]¶
Construct a low-rank approximation of another representation.
Note
While this method tries to find a good approximation of the base representation, you may lose all (useful) inductive biases you had with the original one, e.g., from shared tokens in
pykeen.representation.NodePieceRepresentation
.- Parameters:
other (
Representation
) – the other representationkwargs – additional keyword-based parameters passed to
LowRankRepresentation.__init__()
. Must not contain max_id nor shape, which are determined by other
- Return type:
- Returns:
a low-rank approximation obtained via (truncated) SVD