cas13f t1_jdcbtkv wrote on March 23, 2023 at 11:43 AM

Reply to comment by agm1984 in Google and Microsoft’s chatbots are already citing one another in a misinformation shitshow by altmorty

The real issue is people expecting a language model, which is all it is, to be an "ai" to "knows everything".

It can write. That is what it is for. It does not have any intelligence, regardless of being a "language AI". The purpose of the model is to generate text in a generally grammatically correct manner when prompted, which is why it has been known to just make up citations when prompted to include them--because as far as the model is concerned, a citation looks a certain way and includes specific grammatical configuration, so it just needs to do that in relation to the prompted words.

It's why it can't do math. It wasn't designed to do so. The model was not trained to do so. It can use word association to write something that looks grammatically relevant.

agm1984 t1_jdcoyl5 wrote on March 23, 2023 at 1:35 PM

I agree with you, but it also represents the interface between human and machine, so it must be accurate.

The issue I am highlighting is minor but might be revealing some unfortunate aspects. For example, if you can adopt a mathematical approach to deciding what words to use, there is a kind of latent space in which the answer to your question draws an octopus tentacle of sorts. The shape of the tentacle is analogous to the chosen words.

My argument is that the tentacle can be deformed at parts of a sentence related to the word 'is' (which is comically an equals sign) because it misrepresents the level of precision it is aware of. For me this is a huge problem because it means either (or both) the "AI" is not extrapolating the correct meaning from the lowest common denominator of cumulative source materials, or the source materials themselves are causing the "AI" to derive a bum value in the specific context with the problem.

My example of gravity 'at all scales' is interesting because there is no world where a scientist can assert such a concrete clause. In actual english terms, it's more like a restrictive clause because the statement hinges on the context around it. Maybe there is a sentence that says "currently" or "to the best of our knowledge", or maybe there is an advanced word vector such as "has been" that helps indicate that gravity is solved here at the moment but might not be in the future.

It's very minor, but my warning extends to a time when a human is reading that and taking fragments at face value because they feel like the "AI" is compiling the real derived truth from the knowledge base of humankind. My warning also extends to a time when a different "AI" is receiving a paragraph from ChatGPT and for the exact same reasons misinterprets it due to these subtle errors of confidence. There's something uncanny about it, and this is where I see an issue currently if you want to use it as an interface. Maybe my side point is that it doesn't make sense to use it as an AI-to-AI interface because you lose so much mathematical accuracy and precision when you render the final output into fuzzy english. Other AIs need to know the exact angle and rotation between words and paragraphs.