Auto-regression & diffusion: concepts, differences

Resources: Why Does Diffusion Work Better than Auto-Regression?


Tensors

arrays of scalars


Chain rule (calculus)

Intuitively, the chain rule states that knowing the instantaneous rate of change of z relative to y and that of y relative to x allows one to calculate the instantaneous rate of change of z relative to x as the product of the two rates of change.

As put by George F. Simmons: “If a car travels twice as fast as a bicycle and the bicycle is four times as fast as a walking man, then the car travels 2 × 4 = 8 times as fast as the man.”


Neuron


Multi-Layer Perceptron (MLP)


Underscores in Python (_ and __): Instance

In Python, __self__ is used to refer to the current instance of the class within instance methods. It is explicitly passed as the first parameter to instance methods.

Resources:


Weights of a Neural Network

Weights are not inputted data. Leaf nodes that will affect the loss function. Get iterated using the gradient information.

Need more.


Biases in the context of a Neural Network


So, weights and biases are...

Here will be the content.


Cost Function


Computation Graph

(mentioned in autograd)

Resources:

Sample computation graph