समस्या
Weights that start too big or too small make deep networks hard to train. Two standard schemes pick the scale from the layer size, where fan_in is the number of inputs and fan_out the number of outputs:
| method | returns |
|---|---|
"xavier_uniform" | limit √(6 / (fan_in + fan_out)), weights drawn from [-limit, limit] |
"xavier_normal" | standard deviation √(2 / (fan_in + fan_out)) |
"he_uniform" | limit √(6 / fan_in) |
"he_normal" | standard deviation √(2 / fan_in) |
Xavier (Glorot) suits sigmoid and tanh layers; He suits ReLU layers.
Implement init_bound(method, fan_in, fan_out).
Input
method: One of "xavier_uniform", "xavier_normal", "he_uniform" or "he_normal".
fan_in, fan_out: The layer's number of inputs and outputs.
Output
Return the limit or standard deviation as a decimal number.
सीमाएँ
1 ≤ fan_in, fan_out ≤ 100,000
उदाहरण
इनपुट:
fan_in = 3
method = "xavier_uniform"
fan_out = 3
अपेक्षित आउटपुट:
expected = 1.0इनपुट:
long long fan_in = 3;
std::string method = "xavier_uniform";
long long fan_out = 3;
अपेक्षित आउटपुट:
double expected = 1.0;इनपुट:
long fan_in = 3;
string method = "xavier_uniform";
long fan_out = 3;
अपेक्षित आउटपुट:
double expected = 1.0;व्याख्या
√(6 / (3 + 3)) = √1 = 1, so weights are drawn from [-1, 1].
इनपुट:
fan_in = 50
method = "he_normal"
fan_out = 10
अपेक्षित आउटपुट:
expected = 0.2इनपुट:
long long fan_in = 50;
std::string method = "he_normal";
long long fan_out = 10;
अपेक्षित आउटपुट:
double expected = 0.2;इनपुट:
long fan_in = 50;
string method = "he_normal";
long fan_out = 10;
अपेक्षित आउटपुट:
double expected = 0.2;व्याख्या
He only uses fan_in: √(2 / 50) = √0.04 = 0.2.
कैसे हल करें
पहले खुद कोशिश करें, फिर एक-एक करके संकेत खोलें।
संकेत 1
Each method only depends on the layer sizes and whether you need a bound or a spread.
संकेत 2
Use fan_in alone for He methods, and fan_in plus fan_out for Xavier methods.
संकेत 3
Do not mix up uniform and normal formulas, since one returns a limit and the other returns a standard deviation.
अपेक्षित जटिलता
O(1) time, O(1) extra space
संदर्भ
यह चुनौती CodeUtility के लिए अनुकूलित की गई है। ऊपर दिए गए स्टेटमेंट और सीमाओं का पालन करें।