I'm struggling to understand what improvements at labs look like. Some part of it is more training data, more weights, more post training. That all seems.straightforward to me. But what % of improvements come from that as opposed to techniques and model design changes? MoE stuff, swapping out the activation function, etc etc? Do we have any info on this?
Comments URL: https://news.ycombinator.com/item?id=49684921
Points: 2
# Comments: 0