Problem
Layer normalization keeps the activations of each token on a stable scale. For each row x of the matrix:
- mean = Σ x / d and var = Σ (x - mean)² / d (population variance over the row's d values)
- x̂ = (x - mean) / √(var + epsilon)
- output = gamma × x̂ + beta (element-wise: column j uses gamma[j] and beta[j])
Implement layer_norm(x, gamma, beta, epsilon).
Input
x: A matrix with one row per token and d columns.
gamma, beta: The learned scale and shift, d values each.
epsilon: A small number that avoids division by zero.
Output
Return the normalized matrix, same shape as x.
Einschränkungen
1 ≤ rows ≤ 100, 1 ≤ d ≤ 128
-100 ≤ each value ≤ 100, 0 < epsilon ≤ 0.001
Beispiele
Eingabe:
x = [
[1.0, 2.0, 3.0]
]
beta = [0.0, 0.0, 0.0]
gamma = [1.0, 1.0, 1.0]
epsilon = 1.0e-05
Erwartete Ausgabe:
expected = [
[-1.224736, 0.0, 1.224736]
]Eingabe:
std::vector<std::vector<double>> x = {
{1.0, 2.0, 3.0}
};
std::vector<double> beta = {0.0, 0.0, 0.0};
std::vector<double> gamma = {1.0, 1.0, 1.0};
double epsilon = 1.0e-05;
Erwartete Ausgabe:
std::vector<std::vector<double>> expected = {
{-1.224736, 0.0, 1.224736}
};Eingabe:
double[][] x = new double[][] {
new double[] {1.0, 2.0, 3.0}
};
double[] beta = new double[] {0.0, 0.0, 0.0};
double[] gamma = new double[] {1.0, 1.0, 1.0};
double epsilon = 1.0e-05;
Erwartete Ausgabe:
double[][] expected = new double[][] {
new double[] {-1.224736, 0.0, 1.224736}
};Erläuterung
mean = 2 and var = (1 + 0 + 1) / 3 ≈ 0.666667, so √(var + epsilon) ≈ 0.816503. The row becomes about [-1.224736, 0, 1.224736]: centered at 0 with variance 1.
Eingabe:
x = [
[5.0, 5.0]
]
beta = [1.0, -1.0]
gamma = [2.0, 3.0]
epsilon = 1.0e-05
Erwartete Ausgabe:
expected = [
[1.0, -1.0]
]Eingabe:
std::vector<std::vector<double>> x = {
{5.0, 5.0}
};
std::vector<double> beta = {1.0, -1.0};
std::vector<double> gamma = {2.0, 3.0};
double epsilon = 1.0e-05;
Erwartete Ausgabe:
std::vector<std::vector<double>> expected = {
{1.0, -1.0}
};Eingabe:
double[][] x = new double[][] {
new double[] {5.0, 5.0}
};
double[] beta = new double[] {1.0, -1.0};
double[] gamma = new double[] {2.0, 3.0};
double epsilon = 1.0e-05;
Erwartete Ausgabe:
double[][] expected = new double[][] {
new double[] {1.0, -1.0}
};Erläuterung
Both values equal the mean, so x̂ = [0, 0] and the output is just beta: [1, -1]. Epsilon prevents dividing by zero.
So gehst du vor
Versuch es zuerst allein und öffne dann einen Hinweis nach dem anderen.
Hinweis 1
Normalize each row independently using that row’s own mean and variance, not statistics across the whole matrix.
Hinweis 2
After standardizing a row, apply the learned scale and shift element-wise with gamma and beta.
Hinweis 3
Use the population variance and add epsilon before the square root to avoid division by zero on constant rows.
Erwartete Komplexität
O(rows * d) time, O(1) extra space
Referenzen
Diese Aufgabe wurde für CodeUtility angepasst. Beachte die oben genannten Vorgaben und Einschränkungen.