# Training

## Stochastic Local Closed World Assumption

class SLCWATrainingLoop(model, triples_factory, optimizer=None, lr_scheduler=None, negative_sampler=None, negative_sampler_kwargs=None, automatic_memory_optimization=True)[source]

A training loop that uses the stochastic local closed world assumption training approach.

Initialize the training loop.

Parameters

Find the maximum batch size for training with the current setting.

This method checks how big the batch size can be for the current model with the given training data and the hardware at hand. If possible, the method will output the determined batch size and a boolean value indicating that this batch size was successfully evaluated. Otherwise, the output will be batch size 1 and the boolean value will be False.

Parameters
Return type
Returns

Tuple containing the maximum possible batch size as well as an indicator if the evaluation with that size was successful.

property checksum: str

The checksum of the model and optimizer the training loop was configured with.

Return type

str

property device

The device used by the model.

classmethod get_normalized_name()

Get the normalized name of the training loop.

Return type

str

property loss

The loss used by the model.

sub_batch_and_slice(*, batch_size, sampler, triples_factory, training_instances)

Check if sub-batching and/or slicing is necessary to train the model on the hardware at hand.

Return type
train(triples_factory, num_epochs=1, batch_size=None, slice_size=None, label_smoothing=0.0, sampler=None, continue_training=False, only_size_probing=False, use_tqdm=True, use_tqdm_batch=True, tqdm_kwargs=None, stopper=None, result_tracker=None, sub_batch_size=None, num_workers=None, clear_optimizer=False, checkpoint_directory=None, checkpoint_name=None, checkpoint_frequency=None, checkpoint_on_failure=False, drop_last=None, callbacks=None)

Train the KGE model.

Parameters
Return type
Returns

The losses per epoch.

## Local Closed World Assumption

class LCWATrainingLoop(*, target=None, **kwargs)[source]

A training loop that is based upon the local closed world assumption (LCWA).

Under the LCWA, for a given true training triple $$(h, r, t) \in \mathcal{T}_{train}$$, all triples $$(h, r, t') \notin \mathcal{T}_{train}$$ are assumed to be false. The training approach thus uses a 1-n scoring, where it efficiently computes scores for all triples $$(h, r, t')$$ for $$t' \in \mathcal{E}$$, i.e., sharing the same (head, relation)-pair.

This implementation slightly generalizes the original LCWA, and allows to make the same assumption for relation, or head entity. In particular the second, i.e., predicting the relation, is commonly encountered in visual relation prediction.

Initialize the training loop.

Parameters

Find the maximum batch size for training with the current setting.

This method checks how big the batch size can be for the current model with the given training data and the hardware at hand. If possible, the method will output the determined batch size and a boolean value indicating that this batch size was successfully evaluated. Otherwise, the output will be batch size 1 and the boolean value will be False.

Parameters
Return type
Returns

Tuple containing the maximum possible batch size as well as an indicator if the evaluation with that size was successful.

property checksum: str

The checksum of the model and optimizer the training loop was configured with.

Return type

str

property device

The device used by the model.

classmethod get_normalized_name()

Get the normalized name of the training loop.

Return type

str

property loss

The loss used by the model.

sub_batch_and_slice(*, batch_size, sampler, triples_factory, training_instances)

Check if sub-batching and/or slicing is necessary to train the model on the hardware at hand.

Return type
train(triples_factory, num_epochs=1, batch_size=None, slice_size=None, label_smoothing=0.0, sampler=None, continue_training=False, only_size_probing=False, use_tqdm=True, use_tqdm_batch=True, tqdm_kwargs=None, stopper=None, result_tracker=None, sub_batch_size=None, num_workers=None, clear_optimizer=False, checkpoint_directory=None, checkpoint_name=None, checkpoint_frequency=None, checkpoint_on_failure=False, drop_last=None, callbacks=None)

Train the KGE model.

Parameters
Return type
Returns

The losses per epoch.

## Base Classes

class TrainingLoop(model, triples_factory, optimizer=None, lr_scheduler=None, automatic_memory_optimization=True)[source]

A training loop.

Initialize the training loop.

Parameters
• model (Model) – The model to train

• triples_factory (CoreTriplesFactory) – The training triples factory

• optimizer (Optional[Optimizer]) – The optimizer to use while training the model

• lr_scheduler (Optional[_LRScheduler]) – The learning rate scheduler you want to use while training the model

• automatic_memory_optimization (bool) – bool Whether to automatically optimize the sub-batch size during training and batch size during evaluation with regards to the hardware at hand.

Find the maximum batch size for training with the current setting.

This method checks how big the batch size can be for the current model with the given training data and the hardware at hand. If possible, the method will output the determined batch size and a boolean value indicating that this batch size was successfully evaluated. Otherwise, the output will be batch size 1 and the boolean value will be False.

Parameters
Return type
Returns

Tuple containing the maximum possible batch size as well as an indicator if the evaluation with that size was successful.

property checksum: str

The checksum of the model and optimizer the training loop was configured with.

Return type

str

property device

The device used by the model.

classmethod get_normalized_name()[source]

Get the normalized name of the training loop.

Return type

str

property loss

The loss used by the model.

sub_batch_and_slice(*, batch_size, sampler, triples_factory, training_instances)[source]

Check if sub-batching and/or slicing is necessary to train the model on the hardware at hand.

Return type
train(triples_factory, num_epochs=1, batch_size=None, slice_size=None, label_smoothing=0.0, sampler=None, continue_training=False, only_size_probing=False, use_tqdm=True, use_tqdm_batch=True, tqdm_kwargs=None, stopper=None, result_tracker=None, sub_batch_size=None, num_workers=None, clear_optimizer=False, checkpoint_directory=None, checkpoint_name=None, checkpoint_frequency=None, checkpoint_on_failure=False, drop_last=None, callbacks=None)[source]

Train the KGE model.

Parameters
Return type
Returns

The losses per epoch.

## Callbacks

Training callbacks.

Training callbacks allow for arbitrary extension of the functionality of the pykeen.training.TrainingLoop without subclassing it. Each callback instance has a loop attribute that allows access to the parent training loop and all of its attributes, including the model. The interaction points are similar to those of Keras.

### Examples

It was suggested in Issue #333 that it might be useful to log all batch losses. This could be accomplished with the following:

from pykeen.training import TrainingCallback

class BatchLossReportCallback(TrainingCallback):
def on_batch(self, epoch: int, batch, batch_loss: float):
print(epoch, batch_loss)


### Classes

 An interface for training callbacks. TrackerCallback(result_tracker) An adapter for the pykeen.trackers.ResultTracker. MultiTrainingCallback([callbacks]) A wrapper for calling multiple training callbacks together.

## Learning Rate Schedulers

Learning Rate Schedulers available in PyKEEN.