DistMult
- class DistMult(*, embedding_dim=50, entity_initializer=<function xavier_uniform_>, entity_constrainer=<function normalize>, relation_initializer=<pykeen.utils.compose object>, regularizer=<class 'pykeen.regularizers.LpRegularizer'>, regularizer_kwargs=None, **kwargs)[source]
Bases:
ERModel
An implementation of DistMult from [yang2014].
This model simplifies RESCAL by restricting matrices representing relations as diagonal matrices.
DistMult is a simplification of
pykeen.models.RESCAL
where the relation matrices \(\textbf{W}_{r} \in \mathbb{R}^{d \times d}\) are restricted to diagonal matrices:\[f(h,r,t) = \textbf{e}_h^{T} \textbf{W}_r \textbf{e}_t = \sum_{i=1}^{d}(\textbf{e}_h)_i \cdot diag(\textbf{W}_r)_i \cdot (\textbf{e}_t)_i\]Because of its restriction to diagonal matrices, DistMult is more computationally than RESCAL, but at the same time it is less expressive. For instance, it is not able to model anti-symmetric relations, since \(f(h,r, t) = f(t,r,h)\). This can alternatively be formulated with relation vectors \(\textbf{r}_r \in \mathbb{R}^d\) and the Hadamard operator and the \(l_1\) norm.
Note
DistMult uses a hard constraint on the embedding norm, but applies a (soft) regularization term on the relation vector norms
\[f(h,r,t) = \|\textbf{e}_h \odot \textbf{r}_r \odot \textbf{e}_t\|_1\]- Note:
For FB15k, Yang et al. report 2 negatives per each positive.
See also
OpenKE implementation of DistMult
Initialize DistMult.
- Parameters:
embedding_dim (
int
) – The entity embedding dimension \(d\). Is usually \(d \in [50, 300]\).entity_initializer (
Union
[str
,Callable
[[FloatTensor
],FloatTensor
],None
]) – Default: xavier uniform, c.f. https://github.com/thunlp/OpenKE/blob/adeed2c0d2bef939807ed4f69c1ea4db35fd149b/models/DistMult.py#L16-L17entity_constrainer (
Union
[str
,Callable
[[FloatTensor
],FloatTensor
],None
]) – Default: constrain entity embeddings to unit lengthrelation_initializer (
Union
[str
,Callable
[[FloatTensor
],FloatTensor
],None
]) – Default: relations are initialized to unit length (but not constrained)regularizer (
Union
[str
,Regularizer
,Type
[Regularizer
],None
]) – the relation representation regularizerregularizer_kwargs (
Optional
[Mapping
[str
,Any
]]) – additional keyword-based parameters. defaults toDistMult.regularizer_default_kwargs
for the default regularizerkwargs – Remaining keyword arguments to forward to
pykeen.models.ERModel
Attributes Summary
The default strategy for optimizing the model's hyper-parameters
The LP settings used by [yang2014] for DistMult
Attributes Documentation