Artificial
Benchmarks compare open models against closed products, not closed models. We might be missing what were actually paying for
So this has been on my mind for a while and it kinda bugs me. Every time someone benchmarks glm-5.2 or deepseek against claude or gpt, the closed one wins on some tasks and people just assume the underlying model is smarter. but thats not really what were measuring. We dont know what these closed providers actually do behind the api. they might be running rag over their own docs, injecting hidden system prompts based on your query, routing to specialized expert models depending on task type, ...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit