![Decoupled Descent: Enforcing Exact Train-Test Error Tracking Via AMP Onsager Corrections [R]](https://preview.redd.it/kvlzc5378tih1.png?width=140&height=56&auto=webp&s=a5e05d304bcb94e3d955ccf6181541b0ce477939)
MachineLearning
Decoupled Descent: Enforcing Exact Train-Test Error Tracking Via AMP Onsager Corrections [R]
Link: https://arxiv.org/pdf/2604.27883 Hi, Most of use are familiar with the headache of training a neural network using gradient descent where the training error may go to zero but the test error may stay the same as initialization or even increases. My paper treats this phenomena as a consequence of data reuse bias and can be isolated by studying full batch gradient descent on a set of stylize Gaussian mixture models. I turns out that this fundamental issue can be avoided using some...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit