ANN

A neural network learns through adjusting the weights.

1. Decide how many neurons & how to connect them
2. Decide learning algorithm
3. Train - initialize the weights, update the weights

Activation Function

How the neuron determine an output.

• Hard limit functions (used in decision-making neuron for classification & pattern recognition)
  • Step function
  • Sign function

Perceptron

Simplest form of NN with adjustable synaptic weights & hard limiter.Learn the linearly-separable function $$\sum^n_{i=1} x_i w_i - \theta = 0$$.

1. Initialize weights & threshold

2. Activation
   $$Y(p) = step(\sum^n_{i=1} x_i(p) w_i(p) - \theta)$$

3. Weight training
   $$\begin{aligned} w_i(p+1) &= w_i(p) + \Delta w_i(p)\ \Delta w_i(p) &= \alpha \cdot x_i(p) \cdot e_i(p)\ e_i(p) &= Y_d(p) - Y(p) \end{aligned}$$
   $$\begin{aligned} \theta_i(p+1) &= \theta_i(p) + \Delta \theta_i(p)\ \Delta \theta_i(p) &= \alpha \cdot (-1) \cdot e_i(p) \end{aligned}$$

4. Iteration

Application

• Prediction
• Classification
• Clustering

Input Data

• Continuous
  • $$scaledValue = \frac{value - min}{max - min}$$
• Discrete
  • Map inputs of discrete values to values in [0, 1]
• Categorical
  • 1-of-N encoding