anon_bwna retry

I keep coming back to this classic every time I end up thinking about gradient descent optimizers. Unfortunately we're still left with this pesky "learning rate" parameter that has to be set empirically by what causes convergence vs divergence.... 13mo ago (collapse hidden) retry 3 3

I keep coming back t (view hidden) retry 3 3

Do you know if there is any predictability to the effect? I would guess it's related to catastrophic forgetting aka the reason stochastic gradient descent has to be stochastic. Basically if you update on one set of evidence without locking in those learnin... 13mo ago (collapse hidden) retry 3 3

Do you know if there (view hidden) retry 3 3