Tech
I was burning context feeding HTML to LLMs, so I built a Markdown converter for agents
If you build AI agents, scrapers, or RAG pipelines, you have run into this: feeding raw HTML to a model is mostly waste. The <div> , the class="..." , the nav, the cookie banner. None of it is the content you wanted, and all of it costs tokens.
I knew this was inefficient. I did not appreciate how inefficient until I measured it. Here is a single page of GitHub's documentation, run through an audit:
HTML Markdown Savings
βββββββββββββββββββββββββββ...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to