Module ops

Tensor ops.

Structs

• Dropout
• Identity

Functions

• attn_softmax_last_dim
  Softmax with fused broadcast addition of a mask and scale. Equivalent to:
• dropout
• hard_sigmoid
• inplace_attn_softmax_last_dim
  Inplace equivalent of attn_softmax_last_dim
• inplace_softmax_last_dim
• kvconcat
• layer_norm
• layer_norm_slow
• leaky_relu
• log_softmax
• pixel_shuffle
• pixel_unshuffle
• replication_pad2d
• rms_norm
• rms_norm_slow
• sdpa
  Scaled dot product attention with a fused kernel.
• sigmoid
• silu
• softmax
  Applies the softmax function to the input tensor, rescaling the element so that elements on a slice of fixed index on dimension dim are between 0 and 1 and sum to 1.
• softmax_last_dim
• swiglu