RandomWalkPositionalEncodingInitializer

class RandomWalkPositionalEncodingInitializer(*, triples_factory: CoreTriplesFactory | None = None, mapped_triples: Tensor | None = None, edge_index: Tensor | None = None, dim: int, num_entities: int | None = None, space_dim: int = 0, skip_first_power: bool = True)[source]

Bases: PretrainedInitializer

Initialize nodes via random-walk positional encoding.

The random walk positional encoding is given as

\[\mathbf{x}_i = [\mathbf{R}_{i, i}, \mathbf{R}^{2}_{i, i}, \ldots, \mathbf{R}^{d}_{i, i}] \in \mathbb{R}^{d}\]

where \(\mathbf{R} := \mathbf{A}\mathbf{D}^{-1}\) is the random walk matrix, with \(\mathbf{D} := \sum_i \mathbf{A}_{i, i}\).

See also

https://arxiv.org/abs/2110.07875

Initialize the positional encoding.

One of triples_factory, mapped_triples or edge_index will be used. The preference order is:

  1. triples_factory

  2. mapped_triples

  3. edge_index

Parameters:

  • triples_factory (CoreTriplesFactory | None) – the triples factory

  • mapped_triples (Tensor | None) – shape: (m, 3) the mapped triples

  • edge_index (Tensor | None) – shape: (2, m) the edge index

  • dim (int) – the dimensionality

  • num_entities (int | None) – the number of entities. If None, it will be inferred from edge_index

  • space_dim (int) – estimated dimensionality of the space. Used to correct the random-walk diagonal by a factor k^(space_dim/2). In euclidean space, this correction means that the height of the gaussian distribution stays almost constant across the number of steps, if space_dim is the dimension of the euclidean space.

  • skip_first_power (bool) – in most cases the adjacencies diagonal values will be zeros (since reflexive edges are not that common). This flag enables skipping the first matrix power.