I am trying to figure out if it's worth upgrading my RAM, but I've noticed that some MoE models don't seem to do well with many layers shared from VRAM --> CPU/RAM. This may be something on my end; a software config or perhaps my specific hardware config. This made me curious as to how many are doing this. I'm thinking many are, especially with non-dense models, but even better if you do this with dense models; I'd like to know the results you get! Example: You have 16gb of VRAM but you h...