LowRankRepresentation

class LowRankRepresentation(*, max_id, shape, num_bases=3, weight_initializer=<pykeen.utils.compose object>, **kwargs)[source]

Bases: Representation

Low-rank embedding factorization.

This representation reduces the number of trainable parameters by not learning independent weights for each index, but rather having shared bases among all indices, and only learn the weights of the linear combination.

\[E[i] = \sum_k B[i, k] * W[k]\]

Initialize the representations.

Parameters:

  • max_id (int) – the maximum ID (exclusively). Valid Ids reach from 0, …, max_id-1

  • shape (Union[int, Sequence[int]]) – the shape of an individual base representation.

  • num_bases (int) – the number of bases. More bases increase expressivity, but also increase the number of trainable parameters.

  • weight_initializer (Callable[[FloatTensor], FloatTensor]) – the initializer for basis weights

  • kwargs – additional keyword based arguments passed to pykeen.nn.representation.Embedding, which is used for the base representations.

Attributes Summary

num_bases

Return the number of bases.

Methods Summary

approximate(other, **kwargs)

Construct a low-rank approximation of another representation.

reset_parameters()

Reset the module's parameters.

Attributes Documentation

num_bases

Return the number of bases.

Return type:

int

Methods Documentation

classmethod approximate(other, **kwargs)[source]

Construct a low-rank approximation of another representation.

Note

While this method tries to find a good approximation of the base representation, you may lose all (useful) inductive biases you had with the original one, e.g., from shared tokens in pykeen.representation.NodePieceRepresentation.

Parameters:

  • other (Representation) – the other representation

  • kwargs – additional keyword-based parameters passed to LowRankRepresentation.__init__(). Must not contain max_id nor shape, which are determined by other

Return type:

ForwardRef

Returns:

a low-rank approximation obtained via (truncated) SVD

reset_parameters()[source]

Reset the module’s parameters.

Return type:

None