# Initialization¶

Embedding weight initialization routines.

init_phases(x)[source]

Generate random phases between 0 and $$2\pi$$.

Return type

Tensor

xavier_normal_(tensor, gain=1.0)[source]

Initialize weights of the tensor similarly to Glorot/Xavier initialization.

Proceed as if it was a linear layer with fan_in of zero and Xavier normal initialization is used. Fill the weight of input embedding with values values sampled from $$\mathcal{N}(0, a^2)$$ where

$a = \text{gain} \times \sqrt{\frac{2}{\text{embedding_dim}}}$
Parameters
Return type

Tensor

Returns

Embedding with weights by the Xavier normal initializer.

xavier_uniform_(tensor, gain=1.0)[source]

Initialize weights of the tensor similarly to Glorot/Xavier initialization.

Proceed as if it was a linear layer with fan_in of zero and Xavier uniform initialization is used, i.e. fill the weight of input embedding with values values sampled from $$\mathcal{U}(-a, a)$$ where

$a = \text{gain} \times \sqrt{\frac{6}{\text{embedding_dim}}}$
Parameters
• tensor – A tensor

• gain (float) – An optional scaling factor, defaults to 1.0.

Returns

Embedding with weights by the Xavier uniform initializer.