NSSALoss
- class NSSALoss(margin=9.0, adversarial_temperature=1.0, reduction='mean')[source]
Bases:
pykeen.losses.SetwiseLoss
An implementation of the self-adversarial negative sampling loss function proposed by [sun2019].
Initialize the NSSA loss.
- Parameters
margin (
float
) – The loss’s margin (also written as gamma in the reference paper)adversarial_temperature (
float
) –The negative sampling temperature (also written as alpha in the reference paper)
Note
The adversarial temperature is the inverse of the softmax temperature used when computing the weights! Its name is only kept for consistency with the nomenclature of [sun2019].
reduction (
str
) – The name of the reduction operation to aggregate the individual loss values from a batch to a scalar loss value. From {‘mean’, ‘sum’}.
Note
The default hyperparameters are based on the experiments for FB15k-237 in [sun2019].
Attributes Summary
The default strategy for optimizing the loss's hyper-parameters
Methods Summary
forward
(pos_scores, neg_scores, neg_weights)Calculate the loss for the given scores.
process_lcwa_scores
(predictions, labels[, ...])Process scores from LCWA training loop.
process_slcwa_scores
(positive_scores, ...[, ...])Process scores from sLCWA training loop.
Attributes Documentation
- hpo_default: ClassVar[Mapping[str, Any]] = {'adversarial_temperature': {'high': 1.0, 'low': 0.5, 'type': <class 'float'>}, 'margin': {'high': 30, 'low': 3, 'q': 3, 'type': <class 'int'>}}
The default strategy for optimizing the loss’s hyper-parameters
- synonyms: ClassVar[Optional[Set[str]]] = {'Negative Sampling Self-Adversarial Loss', 'Self-Adversarial Negative Sampling Loss'}
Methods Documentation
- forward(pos_scores, neg_scores, neg_weights)[source]
Calculate the loss for the given scores.
- Parameters
pos_scores (
FloatTensor
) – shape: s_p Positive score tensorneg_scores (
FloatTensor
) – shape: s_n Negative score tensorneg_weights (
FloatTensor
) – shape: s_n
- Return type
FloatTensor
- Returns
A loss value
- process_lcwa_scores(predictions, labels, label_smoothing=None, num_entities=None)[source]
Process scores from LCWA training loop.
- Parameters
- Return type
FloatTensor
- Returns
A scalar loss value.
- process_slcwa_scores(positive_scores, negative_scores, label_smoothing=None, batch_filter=None, num_entities=None)[source]
Process scores from sLCWA training loop.
- Parameters
positive_scores (
FloatTensor
) – shape: (batch_size, 1) The scores for positive triples.negative_scores (
FloatTensor
) – shape: (batch_size, num_neg_per_pos) or (num_unfiltered_negatives,) The scores for the negative triples, either in dense 2D shape, or in case they are already filtered, in sparse shape. If they are given in sparse shape, batch_filter needs to be provided, too.label_smoothing (
Optional
[float
]) – An optional label smoothing parameter.batch_filter (
Optional
[BoolTensor
]) – shape: (batch_size, num_neg_per_pos) An optional filter of negative scores which were kept. Given if and only if negative_scores have been pre-filtered.num_entities (
Optional
[int
]) – The number of entities. Only required if label smoothing is enabled.
- Return type
FloatTensor
- Returns
A scalar loss term.