Problem
Adam keeps two moving averages per weight, of the gradient (m) and of the squared gradient (v), and uses them to give each weight its own step size.
Start with m = 0 and v = 0 for every weight. For each step t = 1, 2, 3, … with gradient g:
- m ← beta1 × m + (1 - beta1) × g
- v ← beta2 × v + (1 - beta2) × g²
- m̂ = m / (1 - beta1ᵗ) and v̂ = v / (1 - beta2ᵗ) (bias correction)
- w ← w - learning_rate × m̂ / (√v̂ + epsilon)
Implement adam_steps(gradients, weights, learning_rate, beta1, beta2, epsilon), applying one update per row of gradients, in order.
Input
gradients: One row per step; each row has one gradient per weight.
weights: The starting weights.
learning_rate, beta1, beta2, epsilon: Adam's hyperparameters (commonly 0.001, 0.9, 0.999 and 0.00000001).
Output
Return the weights after all steps.
Constraints
0 ≤ steps ≤ 1,000, 1 ≤ weights ≤ 100
0 < learning_rate ≤ 1, 0 ≤ beta1, beta2 < 1, 0 < epsilon ≤ 0.001
-100 ≤ each value ≤ 100
Examples
Input:
beta1 = 0.9
beta2 = 0.999
epsilon = 1.0e-08
weights = [1.0]
gradients = [
[2.0]
]
learning_rate = 0.1
Expected output:
expected = [0.9]Input:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
std::vector<double> weights = {1.0};
std::vector<std::vector<double>> gradients = {
{2.0}
};
double learning_rate = 0.1;
Expected output:
std::vector<double> expected = {0.9};Input:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
double[] weights = new double[] {1.0};
double[][] gradients = new double[][] {
new double[] {2.0}
};
double learning_rate = 0.1;
Expected output:
double[] expected = new double[] {0.9};Explanation
m = 0.1 × 2 = 0.2 and v = 0.001 × 4 = 0.004. Bias correction gives m̂ = 0.2 / 0.1 = 2 and v̂ = 0.004 / 0.001 = 4, so the step is 0.1 × 2 / √4 = 0.1: w = 0.9. On its first step Adam moves each weight by about the learning rate, whatever the gradient's size.
Input:
beta1 = 0.9
beta2 = 0.999
epsilon = 1.0e-08
weights = [0.0, 0.0]
gradients = [
[100.0, 0.01]
]
learning_rate = 0.01
Expected output:
expected = [-0.01, -0.01]Input:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
std::vector<double> weights = {0.0, 0.0};
std::vector<std::vector<double>> gradients = {
{100.0, 0.01}
};
double learning_rate = 0.01;
Expected output:
std::vector<double> expected = {-0.01, -0.01};Input:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
double[] weights = new double[] {0.0, 0.0};
double[][] gradients = new double[][] {
new double[] {100.0, 0.01}
};
double learning_rate = 0.01;
Expected output:
double[] expected = new double[] {-0.01, -0.01};Explanation
The gradients differ by a factor of 10,000, yet both weights move by about 0.01, to [-0.01, -0.01]: Adam scales each step by the gradient's own size.
How to approach it
Try on your own first, then open one hint at a time.
Hint 1
Track two moving averages per weight: one for the gradient direction and one for its squared size.
Hint 2
Apply bias correction using the current step number before dividing by the square root term.
Hint 3
Pay attention to epsilon in the denominator, because it prevents division by zero and changes the final update slightly.
Expected complexity
O(s × w) time, O(w) extra space
References
This challenge is adapted for CodeUtility. Follow the statement and constraints above.