Viewing a single comment thread. View all comments

bohreffect t1_iz6emfb wrote

Reply to comment by [deleted] in [R] The Forward-Forward Algorithm: Some Preliminary Investigations [Geoffrey Hinton] by shitboots

I mean, can you not compute the Jacobian of a constrained optimization program and stack that into any differentiable composition of functions?

People snoozin'.

1

[deleted] t1_iz6hlao wrote

no you can't because it's not actually a Jacobian

1

bohreffect t1_iz6j2xr wrote

The Jacobian of the solution of a constrained optimization program with respect to its parameters, but I thought that was understood amongst the towering intellect of neural network afficiandos, e.g. the original commenter finding backprop to be stale.

Here's the stochastic programming version: Section 3.3. https://proceedings.neurips.cc/paper/2017/file/3fc2c60b5782f641f76bcefc39fb2392-Paper.pdf

1

Ulfgardleo t1_iz9fjio wrote

Funny that stuff always comes back. We used to differentiate SVM solutions wrt kernel parameters like that back in the day.

1