Auto-regression & diffusion: concepts, differences

Resources: Why Does Diffusion Work Better than Auto-Regression?


arrays of scalars

Chain rule (calculus)

Intuitively, the chain rule states that knowing the instantaneous rate of change of z relative to y and that of y relative to x allows one to calculate the instantaneous rate of change of z relative to x as the product of the two rates of change.

As put by George F. Simmons: “If a car travels twice as fast as a bicycle and the bicycle is four times as fast as a walking man, then the car travels 2 × 4 = 8 times as fast as the man.”


Multi-Layer Perceptron (MLP)

Underscores in Python (_ and __): Instance

In Python, __self__ is used to refer to the current instance of the class within instance methods. It is explicitly passed as the first parameter to instance methods.


Weights of a Neural Network

Weights are not inputted data. Leaf nodes that will affect the loss function. Get iterated using the gradient information.

Need more.

Biases in the context of a Neural Network

So, weights and biases are...

Here will be the content.

Cost Function

Computation Graph

(mentioned in autograd)


Sample computation graph