
LocalLLaMA
dvlt.cu: inference engine written from scratch in CUDA/C++ for NVIDIA's DVLT 3D transformer model
Im into both HPC and 3D reconstruction, so I built this as a side project. dvlt.cu is a single 5MB binary: - No python, torch, TF, ONNX, llama.cpp, vLLM, or huggingface runtime - Nearly no dependencies: only cuBLASLt (shipped with libcuda ) + cuTLASS ( header only lib ) - mmap'd bf16 weights, one bulk GPU upload, static dims, one-shot arena, deterministic - Weights (117M Params) are NVIDIA's (non-commercial), fetched separately at setup. - Just download the weights, build, and ...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit