LocalLLaMA
nvidias nemotron omni only loads its text half on a mac, so i wrote the vision and audio towers in mlx
nvidias nemotron omni is open weights and it sees, hears and reasons. theres already a 4bit mlx quant on hugging face but only the text backbone loads with standard mlx tooling. the model card says it plainly, the vision and audio towers need a runtime that implements the c-radio vit-h and parakeet conformer forward passes. so i wrote that runtime in pure mlx. vision tower, audio tower, the processor and the token splicing, all ported from nvidias reference implementation. it runs the mlx-comm...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit