Submitted by Singularian2501 t3_11zsdwv in MachineLearning
light24bulbs t1_jdijmr3 wrote
Reply to comment by TFenrir in [N] ChatGPT plugins by Singularian2501
My strategy was to have the outer LLM make a JSON object where one of the args is an instruction or question, and then pass that to the inner LLM wrapped in a template like "given the following document, <instruction>"
Works for a fair few general cases and it can get the context that ends up in the outer LLM down to a few sentences aka few tokens, meaning there's plenty of room for more reasoning and cost savings
TFenrir t1_jdim3vv wrote
That is a really good tip.
I'm using langchainjs (I can do python, but my js background is 10x python) - one of the things I want to play with more is getting consistent json output from a response - there is a helper tool I tried with a bud a while back when we were pairing... Typescript validator or something or other, that seemed to help.
Any tips with that?
light24bulbs t1_jditg4b wrote
Nope, I'm struggling along with you on that I'm afraid. That's why these new plugins will be nice.
Maybe we can make some money selling premium feature access to ours once we get it
Viewing a single comment thread. View all comments