Problème
Adam keeps two moving averages per weight, of the gradient (m) and of the squared gradient (v), and uses them to give each weight its own step size.
Start with m = 0 and v = 0 for every weight. For each step t = 1, 2, 3, … with gradient g:
- m ← beta1 × m + (1 - beta1) × g
- v ← beta2 × v + (1 - beta2) × g²
- m̂ = m / (1 - beta1ᵗ) and v̂ = v / (1 - beta2ᵗ) (bias correction)
- w ← w - learning_rate × m̂ / (√v̂ + epsilon)
Implement adam_steps(gradients, weights, learning_rate, beta1, beta2, epsilon), applying one update per row of gradients, in order.
Input
gradients: One row per step; each row has one gradient per weight.
weights: The starting weights.
learning_rate, beta1, beta2, epsilon: Adam's hyperparameters (commonly 0.001, 0.9, 0.999 and 0.00000001).
Output
Return the weights after all steps.
Contraintes
0 ≤ steps ≤ 1,000, 1 ≤ weights ≤ 100
0 < learning_rate ≤ 1, 0 ≤ beta1, beta2 < 1, 0 < epsilon ≤ 0.001
-100 ≤ each value ≤ 100
Exemples
Entrée :
beta1 = 0.9
beta2 = 0.999
epsilon = 1.0e-08
weights = [1.0]
gradients = [
[2.0]
]
learning_rate = 0.1
Sortie attendue :
expected = [0.9]Entrée :
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
std::vector<double> weights = {1.0};
std::vector<std::vector<double>> gradients = {
{2.0}
};
double learning_rate = 0.1;
Sortie attendue :
std::vector<double> expected = {0.9};Entrée :
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
double[] weights = new double[] {1.0};
double[][] gradients = new double[][] {
new double[] {2.0}
};
double learning_rate = 0.1;
Sortie attendue :
double[] expected = new double[] {0.9};Explication
m = 0.1 × 2 = 0.2 and v = 0.001 × 4 = 0.004. Bias correction gives m̂ = 0.2 / 0.1 = 2 and v̂ = 0.004 / 0.001 = 4, so the step is 0.1 × 2 / √4 = 0.1: w = 0.9. On its first step Adam moves each weight by about the learning rate, whatever the gradient's size.
Entrée :
beta1 = 0.9
beta2 = 0.999
epsilon = 1.0e-08
weights = [0.0, 0.0]
gradients = [
[100.0, 0.01]
]
learning_rate = 0.01
Sortie attendue :
expected = [-0.01, -0.01]Entrée :
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
std::vector<double> weights = {0.0, 0.0};
std::vector<std::vector<double>> gradients = {
{100.0, 0.01}
};
double learning_rate = 0.01;
Sortie attendue :
std::vector<double> expected = {-0.01, -0.01};Entrée :
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
double[] weights = new double[] {0.0, 0.0};
double[][] gradients = new double[][] {
new double[] {100.0, 0.01}
};
double learning_rate = 0.01;
Sortie attendue :
double[] expected = new double[] {-0.01, -0.01};Explication
The gradients differ by a factor of 10,000, yet both weights move by about 0.01, to [-0.01, -0.01]: Adam scales each step by the gradient's own size.
Comment l'aborder
Essayez d'abord seul, puis ouvrez les indices un par un.
Indice 1
Track two moving averages per weight: one for the gradient direction and one for its squared size.
Indice 2
Apply bias correction using the current step number before dividing by the square root term.
Indice 3
Pay attention to epsilon in the denominator, because it prevents division by zero and changes the final update slightly.
Complexité attendue
O(s × w) time, O(w) extra space
Références
Ce défi a été adapté pour CodeUtility. Suivez l'énoncé et les contraintes ci-dessus.