neural network-Lecture3

Backpropagation Network
- In 1969 a method for learning in multi-layer network,
Backpropagation, was invented by Bryson and Ho.
- The Backpropagation algorithm is a sensible approach
for dividing the contribution of each weight.
- Works basically the same as perceptron
Backpropagation Learning Principles: Hidden Layers and Gradients
There are two differences for the updating rule :
1) The activation of the hidden unit is used instead of activation of the input value.
2) The rule contains a term for the gradient of the activation
function.
Backpropagation Network training
•1. Initialize network with random weights
•2. For all training cases (called examples):
–a. Present training inputs to network and calculate output
–b. For all layers (starting with output layer, back to input layer):
•i. Compare network output with correct output
(error function)
•ii. Adapt weights in current layer
Backpropagation Learning Details
•Method for learning weights in feed-forward (FF) nets
•Can’t use Perceptron Learning Rule
–no teacher values are possible for hidden units
•Use gradient descent to minimize the error
–propagate deltas to adjust for errors
backward from outputs
to hidden layers
to inputs
forward
backward