Clip the Gradient
The loss is the sum of (w · x)² over the elements, where w holds the parameters and x the inputs, element by element. Compute the gradient of the loss with respect to w. If its length (the square root of the sum of its squares) is more than max_norm, scale the whole gradient down so that its length is max_norm; otherwise leave it. Return the gradient.
Example 1
w [1.0,1.0]
x [1.0,2.0]
max_norm 1.0
returns [0.2425356,0.9701425]
The gradient is [2, 8], of length 8.25: scaled down to length 1 it keeps its direction.
Example 2
w [1.0,1.0]
x [1.0,2.0]
max_norm 100.0
returns [2.0,8.0]
A gradient shorter than the limit is left alone.
Constraints
- 1 to 5 elements
Write it in your browser, against the clock. The model solution, stepped through line by line, comes after.
More PyTorch problems
- Keep RollingHard
- Learning Rate StepsHard
- Logistic RegressionHard
- Skip the PaddingHard
- Slide a KernelHard
- The Second DerivativeHard
- Windows and TargetsHard
- A Table of ZerosBeginner