Not-o-matic Differentiation

https://ajknapp.github.io/2018/08/14/notomatic-differentiation.html

by jxub • 8 years ago

50 17 8 years ago

ajknapp.github.io

mlevental3• 8 years ago

this might be the right place to ask: can you differentiate a recursively defined function?

im3w1l• 8 years ago

If I understand you correctly: yes. You differentiate both sides and then get an equation for the derivative.

throwawayjava• 8 years ago

Yes. You can also integrate a recursively defined function(s). E.g., recall your undergraduate differential equations course.

mlevental3• 8 years ago

not sure what this means?

danmg• 8 years ago

yes. it generally won't have a closed form however.

edflsafoiewq• 8 years ago

Let's try

    pow x 0 = 1
    pow x n = x * pow x (n-1)
      ergo
    pow' x 0 = 0
    pow' x n = pow x (n-1) + x * pow' x (n-1)
      ergo
    pow' x 1 = pow x 0 + x * pow' x 0
             = 1 + x * 0
             = 1
    pow' x 2 = pow x 1 + x * pow' x 1
             = x + x
             = 2 * x
    pow' x 3 = pow x 2 + x * pow' x 2
             = x * x + 2 * x * x
             = 3 * x * x

perl4ever• 8 years ago

Isn't differentiation itself a recursively defined function?

tome• 8 years ago

Yes, but good luck differentiating differentiation!

joe_the_user• 8 years ago

I'm trying to figure why the person making the post failed. There are existing Haskell libraries for automatic differentiation [1].

As opposed to ordinary symbolic differentiation or numerical differentiation, automatic differentiation is a method that takes the code of a function and a value and returns the derivative of the function at run time.

Of course, the author is working with a vector library and so his version will take N-dimension vectors/tensors and a functions of these and return N+something dimension vectors/tensors. That part is kind of head spinning and I assume that's where things fell apart.

[1] http://hackage.haskell.org/package/ad

eastWestMath3• 8 years ago

Well if you look at the categorical semantics of differentiation, it takes in a function A -> B and spits out AxA -> B. It’s a fairly well developed area of logic/type theory (even at the level of term rewriting systems), it seems the author of the blog post missed it. I probably would have emailed him some references if I’d seen the initial announcement...

nmca• 8 years ago

Because this was intended to support `accelerate` and uses the variable-mode ad approach presented in 'the simple essence of automatic differentiation' which afaik has not been implemented in conjunction with `accelerate` .

(Your comment reads quite uncharitable/mean btw, hard to tell if it was intentional)

tree_of_item• 8 years ago

(Your comment reads quite uncharitable/mean btw, hard to tell if it was intentional)

Really? I didn't get that at all. Was it edited?

eastWestMath3• 8 years ago

There’s a fairly large body of work on the differential lambda calculus that you seem to have missed...

heyjudy• 8 years ago

My HP 48G(X) from the 1990's did automatic symbolic differentiation and integration. And, IIRC, there's probably a textbook in either Fortran or C for symbolic mathematical evaluation (partial evaluation, simplification and Monte Carlo fallback) and transformations. We had to deal a lot with this in the code used by a nuclear reactor simulator (in Fortran) that effectively did glorified finite element analysis with some actual direct calculations here and there.

eastWestMath3• 8 years ago

Huh?

Not-o-matic Differentiation

Related Stories