Mini-Batch Gradient Descent

Mini-batch gradient descent is how almost every neural network is trained: split the data into small batches and update the weights after each one.

Train y = w × x + b on the mean squared error, starting from w = 0 and b = 0. In each epoch, split the samples in order into batches of batch_size (the last batch may be smaller). For each batch of size n:

1. errorᵢ = w × xᵢ + b - yᵢ for each sample in the batch
2. dw = (2 / n) × Σ errorᵢ × xᵢ and db = (2 / n) × Σ errorᵢ
3. w ← w - learning_rate × dw and b ← b - learning_rate × db

Implement train_mini_batch(xs, ys, batch_size, learning_rate, epochs).

Input

xs, ys: The data points (same length).

batch_size: Samples per batch.

learning_rate, epochs: The step size and the number of passes over the data.

Output

Return [w, b] after training.

सीमाएँ

1 ≤ number of points ≤ 1,000, 1 ≤ batch_size ≤ number of points, 0 ≤ epochs ≤ 1,000

0 < learning_rate ≤ 1

-100 ≤ each value ≤ 100

उदाहरण

इनपुट:
xs = [1.0, 2.0, 3.0]
ys = [2.0, 4.0, 6.0]
epochs = 1
batch_size = 2
learning_rate = 0.1

अपेक्षित आउटपुट:
expected = [2.44, 1.08]

इनपुट:
std::vector<double> xs = {1.0, 2.0, 3.0};
std::vector<double> ys = {2.0, 4.0, 6.0};
long long epochs = 1;
long long batch_size = 2;
double learning_rate = 0.1;

अपेक्षित आउटपुट:
std::vector<double> expected = {2.44, 1.08};

इनपुट:
double[] xs = new double[] {1.0, 2.0, 3.0};
double[] ys = new double[] {2.0, 4.0, 6.0};
long epochs = 1;
long batch_size = 2;
double learning_rate = 0.1;

अपेक्षित आउटपुट:
double[] expected = new double[] {2.44, 1.08};

इनपुट:
xs = [1.0, 2.0, 3.0, 4.0]
ys = [3.0, 5.0, 7.0, 9.0]
epochs = 1
batch_size = 4
learning_rate = 0.05

अपेक्षित आउटपुट:
expected = [1.75, 0.6]

इनपुट:
std::vector<double> xs = {1.0, 2.0, 3.0, 4.0};
std::vector<double> ys = {3.0, 5.0, 7.0, 9.0};
long long epochs = 1;
long long batch_size = 4;
double learning_rate = 0.05;

अपेक्षित आउटपुट:
std::vector<double> expected = {1.75, 0.6};

इनपुट:
double[] xs = new double[] {1.0, 2.0, 3.0, 4.0};
double[] ys = new double[] {3.0, 5.0, 7.0, 9.0};
long epochs = 1;
long batch_size = 4;
double learning_rate = 0.05;

अपेक्षित आउटपुट:
double[] expected = new double[] {1.75, 0.6};

संदर्भ

यह चुनौती CodeUtility के लिए अनुकूलित की गई है। ऊपर दिए गए स्टेटमेंट और सीमाओं का पालन करें।

• Deep Learning book, Chapter 8: Optimization for Training Deep Models
• Stanford CS231n, Neural Networks Part 3: parameter updates