llama cpp json formatting
One cool feature in llama.cpp is the ability to provide a grammer from the LLM output, which has some powerful implications. This post is meant to show some examples of my work messing with python and llama.cpp grammars
Format LLM response with a given JSON schema Lets say I want to generate API calls for a given prompt. I can define a JSON schema like this:
schema = ''' { "$schema": "http://json-schema.
[Read More]