問題
A language model must not peek at future words. Causal attention masks the future: query row i may only attend to key rows 0 to i.
For each row i (queries, keys and values all have n rows):
- Score each allowed key j ≤ i: score = (queries[i] · keys[j]) / √d, where d is the number of columns of keys.
- Apply softmax over those allowed scores only (subtract the largest before taking e^x).
- Output row i = Σ weight_j × values[j] over j ≤ i.
Implement causal_attention(queries, keys, values).
Input
queries, keys: n rows of d numbers each.
values: n rows of v numbers each.
Output
Return the n × v output matrix.
制約
1 ≤ n ≤ 50, 1 ≤ d, v ≤ 32
-10 ≤ each value ≤ 10
例
例 1
入力：
keys = [
[1.0],
[2.0],
[3.0]
]
values = [
[3.0],
[6.0],
[9.0]
]
queries = [
[0.0],
[0.0],
[0.0]
]
期待される出力：
expected = [
[3.0],
[4.5],
[6.0]
]入力：
std::vector<std::vector<double>> keys = {
{1.0},
{2.0},
{3.0}
};
std::vector<std::vector<double>> values = {
{3.0},
{6.0},
{9.0}
};
std::vector<std::vector<double>> queries = {
{0.0},
{0.0},
{0.0}
};
期待される出力：
std::vector<std::vector<double>> expected = {
{3.0},
{4.5},
{6.0}
};入力：
double[][] keys = new double[][] {
new double[] {1.0},
new double[] {2.0},
new double[] {3.0}
};
double[][] values = new double[][] {
new double[] {3.0},
new double[] {6.0},
new double[] {9.0}
};
double[][] queries = new double[][] {
new double[] {0.0},
new double[] {0.0},
new double[] {0.0}
};
期待される出力：
double[][] expected = new double[][] {
new double[] {3.0},
new double[] {4.5},
new double[] {6.0}
};解説
All scores are 0, so each row averages the values it may see: row 0 sees only 3, row 1 averages 3 and 6 (4.5), row 2 averages all three (6).
例 2
入力：
keys = [
[1.0, 0.0],
[1.0, 0.0]
]
values = [
[1.0, 2.0],
[3.0, 4.0]
]
queries = [
[1.0, 0.0],
[1.0, 0.0]
]
期待される出力：
expected = [
[1.0, 2.0],
[2.0, 3.0]
]入力：
std::vector<std::vector<double>> keys = {
{1.0, 0.0},
{1.0, 0.0}
};
std::vector<std::vector<double>> values = {
{1.0, 2.0},
{3.0, 4.0}
};
std::vector<std::vector<double>> queries = {
{1.0, 0.0},
{1.0, 0.0}
};
期待される出力：
std::vector<std::vector<double>> expected = {
{1.0, 2.0},
{2.0, 3.0}
};入力：
double[][] keys = new double[][] {
new double[] {1.0, 0.0},
new double[] {1.0, 0.0}
};
double[][] values = new double[][] {
new double[] {1.0, 2.0},
new double[] {3.0, 4.0}
};
double[][] queries = new double[][] {
new double[] {1.0, 0.0},
new double[] {1.0, 0.0}
};
期待される出力：
double[][] expected = new double[][] {
new double[] {1.0, 2.0},
new double[] {2.0, 3.0}
};解説
Row 0 can only see itself, so it copies values[0] = [1, 2]. Row 1 gives both keys the same score, so it averages them: [2, 3].
参考資料
この課題はCodeUtility用に調整されています。上記の説明と制約に従ってください。