Viewing a single comment thread. View all comments

LanchestersLaw t1_jdszbjk wrote

Reply to comment by addition in [D] GPT4 and coding problems by enryu42

What I think is the most amazing thing is that GPT got this far while only trying to predict the very next word one word at a time. The fact it can generate essays by only considering one token at a time is mind boggling.

With all the feedback from ChatGPT it should be easy to program a supervisor who can look at the entire final output of GPT and make a prediction what the user would say in response; then it asks that to GPT to revise the output recursively until it converges. That should be relatively easy to do but would be very powerful.

28

Flag_Red t1_jdtskoy wrote

It's not really accurate to say it's "only considering one token at a time". Foresight and (implicit) planning are taking place. You can see this clearly during programming tasks, where imports come hundreds of tokens before they are eventually used.

22

lacraque t1_jdunvp4 wrote

Well for me often it also imports a bunch of crap that’s never used…

6

modeless t1_jdtx2eu wrote

I like the idea of predicting the user's response. How's this as an architecture for a helpful agent:

Given a user question, before you generate an answer you predict the user's ideal response to the model's answer (e.g. "thanks, that was helpful", or more likely a distribution over such responses), then generate an answer and iteratively optimize it to make the ideal user response more likely.

This way you're explicitly modeling the user's intent, and you can adapt the amount of computation appropriately for the complexity of the question by controlling the number of iterations on the answer.

3