Ran into this problem: a flat markdown file (or even a GraphRAG "brain") gives agents better context, but when a chain of actions fails, you cannot point to which stpe broke or what to fix. No explicability, a slow convergence, and no ceiling. So I modeled it as a partially observable MDP instead. States the agent can observe become nodes, actions become edges, and each edge is weighted by success probability, token cost, and outcome. The agent explores to complete the graph, then ex...