There's a pretty nice demo implementation of this concept at https://toolformerzero.com/

> I have this amazing idea thousands of programmers have been building variations of for the past two months using GPT-Index.

> Nobody cares about GPT-Index, this is a completely novel idea!

Redditor finds out about GPT-Index.

Yep, i get what you mean, but i think the critical point is your last sentence, and that is where i disagree. I think LLMs largely already have this capability, and can show it when faced with well designed prompts.

For your example, here's a prompt design i came up with pretty quickly, and that might generalize to the kind of "assistant" usecase decently well:

​

Prompt: You are an AI assistant aiming to provide accurate answers to the questions of your users. For each answer, follow a logical chain of reasoning to arrive at the correct answer. Then write "OUTPUT: " and summarize that answer in as few words as possible. If the answer can be summarized using only a number or range of numbers, only summarize using those numbers.The first user question is: Please, tell me the average lifespan of a cat

Response: To determine the average lifespan of a cat, we must first consider the various factors that can affect a cat's lifespan, such as breed, health, and environmental conditions. We can then gather data from a variety of sources, such as veterinary studies and population statistics, to determine the average lifespan of cats in general.After analyzing this data, it is clear that the average lifespan of a cat is approximately 12-16 years. However, some cats may live longer or shorter lives depending on the factors mentioned earlier.OUTPUT: 12-16 years.

​

Now, all we need to do is hide the complex prompt from the user, search the response for OUTPUT, subtract all tokens appearing prior to OUTPUT from that response (maybe keep them around in the background if the user asks for more detail), and we got an assistant that will look like this:

​

Prompt: Please, tell me the average lifespan of a cat

Response: 12-16 years.

Imo this doesn't matter.
This sort of criticism only sticks when treating LLMs as commercial software, not raw token predictors.

I.e. the thing you desire is not a less verbose model, it is an implementation of an arbitrarily verbose model that responds well enough to 0-shot prompts to enable 3rd party implementations to achieve minimally verbose chat bots.

I'd recommend the article above as well as Language Models Perform Reasoning via Chain of Thought for some examples of how incredibly effective "prompt programming" can be. In fact, i think it is completely unreasonable to expect LLM output to be popular (in the sense that the top-upvoted comment in this thread would be "Wow! This is exactly the kind of output i want from my AI chatbots!"), since there simply is no such common ground for un-optimized prompts.
Meaning the default verbosity is not important for any applications past the sandbox interface provided by whoever trained the model. That can still be important mind you, in the sense that user penetration might be highest for that interface (as is certainly the case with ChatGPT), but at that point we are talking about a failure of people developing useful products with the model (or a failure to provide access to the model via API) rather than a failure of the model.