Problema
Adam keeps two moving averages per weight, of the gradient (m) and of the squared gradient (v), and uses them to give each weight its own step size.
Start with m = 0 and v = 0 for every weight. For each step t = 1, 2, 3, … with gradient g:
- m ← beta1 × m + (1 - beta1) × g
- v ← beta2 × v + (1 - beta2) × g²
- m̂ = m / (1 - beta1ᵗ) and v̂ = v / (1 - beta2ᵗ) (bias correction)
- w ← w - learning_rate × m̂ / (√v̂ + epsilon)
Implement adam_steps(gradients, weights, learning_rate, beta1, beta2, epsilon), applying one update per row of gradients, in order.
Input
gradients: One row per step; each row has one gradient per weight.
weights: The starting weights.
learning_rate, beta1, beta2, epsilon: Adam's hyperparameters (commonly 0.001, 0.9, 0.999 and 0.00000001).
Output
Return the weights after all steps.
Restricciones
0 ≤ steps ≤ 1,000, 1 ≤ weights ≤ 100
0 < learning_rate ≤ 1, 0 ≤ beta1, beta2 < 1, 0 < epsilon ≤ 0.001
-100 ≤ each value ≤ 100
Ejemplos
Entrada:
beta1 = 0.9
beta2 = 0.999
epsilon = 1.0e-08
weights = [1.0]
gradients = [
[2.0]
]
learning_rate = 0.1
Salida esperada:
expected = [0.9]Entrada:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
std::vector<double> weights = {1.0};
std::vector<std::vector<double>> gradients = {
{2.0}
};
double learning_rate = 0.1;
Salida esperada:
std::vector<double> expected = {0.9};Entrada:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
double[] weights = new double[] {1.0};
double[][] gradients = new double[][] {
new double[] {2.0}
};
double learning_rate = 0.1;
Salida esperada:
double[] expected = new double[] {0.9};Explicación
m = 0.1 × 2 = 0.2 and v = 0.001 × 4 = 0.004. Bias correction gives m̂ = 0.2 / 0.1 = 2 and v̂ = 0.004 / 0.001 = 4, so the step is 0.1 × 2 / √4 = 0.1: w = 0.9. On its first step Adam moves each weight by about the learning rate, whatever the gradient's size.
Entrada:
beta1 = 0.9
beta2 = 0.999
epsilon = 1.0e-08
weights = [0.0, 0.0]
gradients = [
[100.0, 0.01]
]
learning_rate = 0.01
Salida esperada:
expected = [-0.01, -0.01]Entrada:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
std::vector<double> weights = {0.0, 0.0};
std::vector<std::vector<double>> gradients = {
{100.0, 0.01}
};
double learning_rate = 0.01;
Salida esperada:
std::vector<double> expected = {-0.01, -0.01};Entrada:
double beta1 = 0.9;
double beta2 = 0.999;
double epsilon = 1.0e-08;
double[] weights = new double[] {0.0, 0.0};
double[][] gradients = new double[][] {
new double[] {100.0, 0.01}
};
double learning_rate = 0.01;
Salida esperada:
double[] expected = new double[] {-0.01, -0.01};Explicación
The gradients differ by a factor of 10,000, yet both weights move by about 0.01, to [-0.01, -0.01]: Adam scales each step by the gradient's own size.
Referencias
Este reto está adaptado para CodeUtility. Sigue el enunciado y las restricciones indicadas arriba.