CodeUtility
कोडिंग को डेवलपर्स के लिए आसान बनाना।!
सभी चुनौतियाँ
मध्यम 0 हल किए गए 0% स्वीकृति

Causal (Masked) Self-Attention

Implement the masked attention GPT models use, where each token can only look at itself and earlier tokens.

आप क्या सीखेंगे

  • Hide future tokens with a causal mask
  • Apply softmax over allowed positions only
  • Understand how language models generate text

समस्या

अनुवाद दिखाएँ

A language model must not peek at future words. Causal attention masks the future: query row i may only attend to key rows 0 to i.

For each row i (queries, keys and values all have n rows):

  1. Score each allowed key j ≤ i: score = (queries[i] · keys[j]) / √d, where d is the number of columns of keys.
  2. Apply softmax over those allowed scores only (subtract the largest before taking e^x).
  3. Output row i = Σ weight_j × values[j] over j ≤ i.

Implement causal_attention(queries, keys, values).

Input

queries, keys: n rows of d numbers each.

values: n rows of v numbers each.

Output

Return the n × v output matrix.

सीमाएँ

1 ≤ n ≤ 50, 1 ≤ d, v ≤ 32

-10 ≤ each value ≤ 10

उदाहरण

उदाहरण 1 
इनपुट:
keys = [
    [1.0],
    [2.0],
    [3.0]
]
values = [
    [3.0],
    [6.0],
    [9.0]
]
queries = [
    [0.0],
    [0.0],
    [0.0]
]

अपेक्षित आउटपुट:
expected = [
    [3.0],
    [4.5],
    [6.0]
]
इनपुट:
std::vector<std::vector<double>> keys = {
    {1.0},
    {2.0},
    {3.0}
};
std::vector<std::vector<double>> values = {
    {3.0},
    {6.0},
    {9.0}
};
std::vector<std::vector<double>> queries = {
    {0.0},
    {0.0},
    {0.0}
};

अपेक्षित आउटपुट:
std::vector<std::vector<double>> expected = {
    {3.0},
    {4.5},
    {6.0}
};
इनपुट:
double[][] keys = new double[][] {
    new double[] {1.0},
    new double[] {2.0},
    new double[] {3.0}
};
double[][] values = new double[][] {
    new double[] {3.0},
    new double[] {6.0},
    new double[] {9.0}
};
double[][] queries = new double[][] {
    new double[] {0.0},
    new double[] {0.0},
    new double[] {0.0}
};

अपेक्षित आउटपुट:
double[][] expected = new double[][] {
    new double[] {3.0},
    new double[] {4.5},
    new double[] {6.0}
};

व्याख्या

All scores are 0, so each row averages the values it may see: row 0 sees only 3, row 1 averages 3 and 6 (4.5), row 2 averages all three (6).

उदाहरण 2 
इनपुट:
keys = [
    [1.0, 0.0],
    [1.0, 0.0]
]
values = [
    [1.0, 2.0],
    [3.0, 4.0]
]
queries = [
    [1.0, 0.0],
    [1.0, 0.0]
]

अपेक्षित आउटपुट:
expected = [
    [1.0, 2.0],
    [2.0, 3.0]
]
इनपुट:
std::vector<std::vector<double>> keys = {
    {1.0, 0.0},
    {1.0, 0.0}
};
std::vector<std::vector<double>> values = {
    {1.0, 2.0},
    {3.0, 4.0}
};
std::vector<std::vector<double>> queries = {
    {1.0, 0.0},
    {1.0, 0.0}
};

अपेक्षित आउटपुट:
std::vector<std::vector<double>> expected = {
    {1.0, 2.0},
    {2.0, 3.0}
};
इनपुट:
double[][] keys = new double[][] {
    new double[] {1.0, 0.0},
    new double[] {1.0, 0.0}
};
double[][] values = new double[][] {
    new double[] {1.0, 2.0},
    new double[] {3.0, 4.0}
};
double[][] queries = new double[][] {
    new double[] {1.0, 0.0},
    new double[] {1.0, 0.0}
};

अपेक्षित आउटपुट:
double[][] expected = new double[][] {
    new double[] {1.0, 2.0},
    new double[] {2.0, 3.0}
};

व्याख्या

Row 0 can only see itself, so it copies values[0] = [1, 2]. Row 1 gives both keys the same score, so it averages them: [2, 3].

संदर्भ

यह चुनौती CodeUtility के लिए अनुकूलित की गई है। ऊपर दिए गए स्टेटमेंट और सीमाओं का पालन करें।

दूसरों से सीखें

समुदाय समाधान

नवीनतमसर्वाधिक रेटेड

अभी तक कोई प्रकाशित समाधान नहीं। पहला साझा करने के लिए इस चुनौती को हल करें।