Tag: Math

Partial Derivatives, Reverse-Mode Autodiff

A partial derivative measures how much a function changes when you nudge one input, holding every other input fixed. For f(x, y) = x²y + y + 2, the partial derivative with respect to x asks: if y stays put, how fast does f move as x moves?

Manual differentiation

Mathematically, we know that ∂f/∂x = 2xy and ∂f/∂y = x² + 1, using a handful of rules:

  • the derivative of a constant is 0
  • the derivative of ax is a
  • the derivative of x^a is a·x^(a-1)
  • the derivative of a sum is the sum of the derivatives: (u + v)' = u' + v'
  • the derivative of a product follows the product rule: (u·v)' = u'v + uv'

But how can a program know that?

[…889 words]