University of California, Berkeley

COMPSCI 188

Q1. [38 pts] Neural Networks and MLE
(a) Neural Network 1 (b) Neural Network 2
(c) Neural Network 3 (d) Neural Network 4
(e) Neural Network 5 (f) Neural Network 6
(a) We first investigate what functions different neural network architectures can represent. For each of the
six following graphs, select the neural networks that can represent the function exactly on the range x ∈
(−∞, ∞). In the networks above, ReLU represents the rectified linear unit and σ represents the sigmoid
function: ReLU (x) = max(0, x), σ(x) = 1
1+e−x .
(i) [3 pts]
y = x
■ Neural Network 1
2 Neural Network 2
2 Neural Network 3
2 Neural Network 4
■ Neural Network 5
■ Neural Network 6
# None of the above
(ii) [3 pts]
y =
(
x x ≤ 2
2 x > 2
2 Neural Network 1
■ Neural Network 2
2 Neural Network 3
2 Neural Network 4
■ Neural Network 5
2 Neural Network 6
# None of the above