Deep Learning 101

Neurons

 

Cortical columns

 

Neural Networks 종류:

• Feedforward NN
• Convolutional NN
• Recurrent NN
  • LSTM, GRU

 

Activation Functions

• Linear
  • backpropagation 불가
• Binary step function
  • on/off 둘 중 하나
  • multiple classification 불가
  • 미분 불가
• sigmoid
• logistic
• TanH(선호하는 경우 많음)
• ReLU(Rectified Linear Unit)
• leaky ReLU
• PReLU(parametric ReLU)
• ELU(Exponential ReLU)
• Swish
• Maxout
• softmax

softmax

 

 

multiple  classification은 softmax를 output layer에 사용한다

RNN, TanH 조합 좋음

swish 는 deep deep deep networks에 좋음