DistMult

class DistMult(*, embedding_dim=50, entity_initializer=<function xavier_uniform_>, entity_constrainer=<function normalize>, relation_initializer=<pykeen.utils.compose object>, regularizer=<class 'pykeen.regularizers.LpRegularizer'>, regularizer_kwargs=None, **kwargs)

Bases: ERModel

An implementation of DistMult from [yang2014].

This model simplifies RESCAL by restricting matrices representing relations as diagonal matrices.

DistMult is a simplification of pykeen.models.RESCAL where the relation matrices \(\textbf{W}_{r} \in \mathbb{R}^{d \times d}\) are restricted to diagonal matrices:

\[f(h,r,t) = \textbf{e}_h^{T} \textbf{W}_r \textbf{e}_t = \sum_{i=1}^{d}(\textbf{e}_h)_i \cdot diag(\textbf{W}_r)_i \cdot (\textbf{e}_t)_i\]

Because of its restriction to diagonal matrices, DistMult is more computationally than RESCAL, but at the same time it is less expressive. For instance, it is not able to model anti-symmetric relations, since \(f(h,r, t) = f(t,r,h)\). This can alternatively be formulated with relation vectors \(\textbf{r}_r \in \mathbb{R}^d\) and the Hadamard operator and the \(l_1\) norm.

Note

DistMult uses a hard constraint on the embedding norm, but applies a (soft) regularization term on the relation vector norms

\[f(h,r,t) = \|\textbf{e}_h \odot \textbf{r}_r \odot \textbf{e}_t\|_1\]

Note:

• For FB15k, Yang et al. report 2 negatives per each positive.

See also

• OpenKE implementation of DistMult

Initialize DistMult.

Parameters:

• embedding_dim (int) – The entity embedding dimension \(d\). Is usually \(d \in [50, 300]\).

• entity_initializer (Union[str, Callable[[FloatTensor], FloatTensor], None]) – Default: xavier uniform, c.f. https://github.com/thunlp/OpenKE/blob/adeed2c0d2bef939807ed4f69c1ea4db35fd149b/models/DistMult.py#L16-L17

• entity_constrainer (Union[str, Callable[[FloatTensor], FloatTensor], None]) – Default: constrain entity embeddings to unit length

• relation_initializer (Union[str, Callable[[FloatTensor], FloatTensor], None]) – Default: relations are initialized to unit length (but not constrained)

• regularizer (Union[str, Regularizer, Type[Regularizer], None]) – the relation representation regularizer

• regularizer_kwargs (Optional[Mapping[str, Any]]) – additional keyword-based parameters. defaults to DistMult.regularizer_default_kwargs for the default regularizer

• kwargs – Remaining keyword arguments to forward to pykeen.models.ERModel

Attributes Summary

hpo_default	The default strategy for optimizing the model's hyper-parameters
regularizer_default_kwargs	The LP settings used by [yang2014] for DistMult

Attributes Documentation

hpo_default: ClassVar[Mapping[str, Any]] = {'embedding_dim': {'high': 256, 'low': 16, 'q': 16, 'type': <class 'int'>}}

The default strategy for optimizing the model’s hyper-parameters

regularizer_default_kwargs: ClassVar[Mapping[str, Any]] = {'normalize': True, 'p': 2.0, 'weight': 0.1}

The LP settings used by [yang2014] for DistMult