Mixed Precision

Track the dtypes used for parameters, activations, accumulation, gradients, and optimizer state, including when loss scaling is needed.