Problem
Adam keeps two moving averages per weight, of the gradient (m) and of the squared gradient (v), and uses them to give each weight its own step size.
Start with m = 0 and v = 0 for every weight. For each step t = 1, 2, 3, … with gradient g:
- m ← beta1 × m + (1 - beta1) × g
- v ← beta2 × v + (1 - beta2) × g²
- m̂ = m / (1 - beta1ᵗ) and v̂ = v / (1 - beta2ᵗ) (bias correction)
- w ← w - learning_rate × m̂ / (√v̂ + epsilon)
Implement adam_steps(gradients, weights, learning_rate, beta1, beta2, epsilon), applying one update per row of gradients, in order.
Input
gradients: One row per step; each row has one gradient per weight.
weights: The starting weights.
learning_rate, beta1, beta2, epsilon: Adam's hyperparameters (commonly 0.001, 0.9, 0.999 and 0.00000001).
Output
Return the weights after all steps.
Einschränkungen
0 ≤ steps ≤ 1,000, 1 ≤ weights ≤ 100
0 < learning_rate ≤ 1, 0 ≤ beta1, beta2 < 1, 0 < epsilon ≤ 0.001
-100 ≤ each value ≤ 100
Beispiele
Eingabe:
beta1 = 0.9
beta2 = 0.999
epsilon = 1.0e-08
weights = [1.0]
gradients = [
[2.0]
]
learning_rate = 0.1
Erwartete Ausgabe:
expected = [0.9]Eingabe:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
std::vector<double> weights = {1.0};
std::vector<std::vector<double>> gradients = {
{2.0}
};
double learning_rate = 0.1;
Erwartete Ausgabe:
std::vector<double> expected = {0.9};Eingabe:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
double[] weights = new double[] {1.0};
double[][] gradients = new double[][] {
new double[] {2.0}
};
double learning_rate = 0.1;
Erwartete Ausgabe:
double[] expected = new double[] {0.9};Erläuterung
m = 0.1 × 2 = 0.2 and v = 0.001 × 4 = 0.004. Bias correction gives m̂ = 0.2 / 0.1 = 2 and v̂ = 0.004 / 0.001 = 4, so the step is 0.1 × 2 / √4 = 0.1: w = 0.9. On its first step Adam moves each weight by about the learning rate, whatever the gradient's size.
Eingabe:
beta1 = 0.9
beta2 = 0.999
epsilon = 1.0e-08
weights = [0.0, 0.0]
gradients = [
[100.0, 0.01]
]
learning_rate = 0.01
Erwartete Ausgabe:
expected = [-0.01, -0.01]Eingabe:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
std::vector<double> weights = {0.0, 0.0};
std::vector<std::vector<double>> gradients = {
{100.0, 0.01}
};
double learning_rate = 0.01;
Erwartete Ausgabe:
std::vector<double> expected = {-0.01, -0.01};Eingabe:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
double[] weights = new double[] {0.0, 0.0};
double[][] gradients = new double[][] {
new double[] {100.0, 0.01}
};
double learning_rate = 0.01;
Erwartete Ausgabe:
double[] expected = new double[] {-0.01, -0.01};Erläuterung
The gradients differ by a factor of 10,000, yet both weights move by about 0.01, to [-0.01, -0.01]: Adam scales each step by the gradient's own size.
So gehst du vor
Versuch es zuerst allein und öffne dann einen Hinweis nach dem anderen.
Hinweis 1
Track two moving averages per weight: one for the gradient direction and one for its squared size.
Hinweis 2
Apply bias correction using the current step number before dividing by the square root term.
Hinweis 3
Pay attention to epsilon in the denominator, because it prevents division by zero and changes the final update slightly.
Erwartete Komplexität
O(s × w) time, O(w) extra space
Referenzen
Diese Aufgabe wurde für CodeUtility angepasst. Beachte die oben genannten Vorgaben und Einschränkungen.