Deep Learning (DL) & Neural Networks (NNs)

Review: High-level Understanding of ML from AIF-C01

💡 This section is a deeper dive into ML and DL, builds on top of this section from AIF-C01 notes:

Inspired by biology
- Neurons (nodes) in cerebral cortex connected via synapses (axons)
- When input signals activated enough, neuron triggers/fires to connected neurons
- Billions of neurons, each with thousands of synapses → many layers → learning mind
- Cortical columns → neurons in one column process info in parallel
  - GPUs work in a similar way
- Biology Diagram
- Deep NN diagram

TensorFlow
- Open-source, developed by Google
- Keras: high-level NN API integrated into TensorFlow
  - easy-to-use interface, while TF handles all the computations under the hood
PyTorch
- Developed by Facebook
- Pythonic interface with NumPy-like behavior
MXNet (”MixNet”)
- Backed by AWS & Apache MXNet

Feed-forward NN (FFNN or FNN) → uni-directional flow (only forward)
- 💡 "Vanilla” NN
- Some subtypes
  - Multilayer perceptron (MLP) → a modern type of FFNN with multiple layers
    - Fully connected → every neuron in one layer connects to every neuron in the next
    - Notable for being able to distinguish data that is not linearly separable
  - Convolutional NN (CNN)
    - Uses convolutional layers instead of fully connected layers to detect patterns
    - Excel at spatial data (usually images) → e.g. “Is there a STOP sign in this image?”
- Diagram
Recurrent NN (RNN) → flow can go backwards (loops)
- Useful for sequences or time-series
  - e.g. Predict stock prices, understand words in a sentence, translation…
- Diagram

Ref: https://www.udemy.com/course/aws-certified-machine-learning-engineer-associate-mla-c01/learn/lecture/45285617

🔧 Math function inside the node of a NN
- Given input (signals), defines the output of the node
- Diagram