I’m pretty critical / sceptical at current attempts at AI / AGI and the jargon around LLM’s (Large Language Models). So without any particular expertise in the current developments (GPT-4 et al) I needed to check out some details:
They’re “Large” in so far as they (arbitrarily) have “Billions” of elements.
They’re “Language” in so far as any representation of a body of knowledge is expressed in some linguistic symbolism – a language.
They’re “Models” in the sense they represent a mass of “data” as an information model. It’s a model of that source data, but not a model of the knowledge or the real world represented by that data.
The form of that model is a “Neural Net” (Artificial Neural Network) whereby the elements are nodes and connections between them are edges – a totally generic network model. They are “Neural” only by analogy with neurones. There is no sense in which the nodes and edges actually represent the elements or processes of biological brains. Similarly there is no sense in which either the learning processes or the Q&A / searching and usage processes represent actual mental processes. They have nothing to do with intelligence or even knowledge in the sense of understanding.
The clever bit – which interests me the least – is the strategy by which the text, symbols & images are indexed and the connections between them are weighted as answers to requests are scored and output weightings adjusted accordingly. A fancy version of Google search algorithms and everybody’s got one of those now.
Now, Networks have been pretty standard for formal information models too, since our ubiquitous platform became the web. Before that models were tables, but tabular or networked formal models represent reality to a greater or lesser extent, as the nodes and edges (tables, columns and rows) are classified and specialised according to some ontology of all the types of element and relationship deemed to best represent the problem domain. A semantic web where properties of the nodes and edges represent meaning in the world, not just a piece of data about it. Modelling done by modellers, with or without the help of algorithmic semi-automation. (For me, long term, this kind of computer-aided modelling is most likely to deliver a model of the world itself and knowledge about it, rather than simply a model of all the data already available to it. This is an area where I do have expertise and experience.)
Anyway the point, again, knowledge of or about a thing (Savoir/Wissen) is quite different to knowing (Connaitre/Kennen) the thing.
And this post was prompted by the coincidence of two Twitter threads crossing my timeline simultaneously this afternoon:
One from Jessica Flack at Santa Fe – a person and institution with deep expertise in the world of information.
One from Sam Norton – a lay person from the computer-based information modelling perspective, but with a deep interest in humans knowing the world.
8/8 Tweet in Jessica’s thread:
8. It does seem likely, however, that humans using LLMs will dramatically change economic organization + the nature of work.
— Jessica Flack (@C4COMPUTATION) March 29, 2023
1/15 Tweet in Sam’s thread:
A few thoughts about AI, on which I have been reading around a little, and which is clearly going to be disruptive – but within limits that I’d like to describe 1/n
— Sam Charles Norton (@Elizaphanian) March 29, 2023
PS – Some people closer to what used to be my day-job on Linked-In also talking about using the artificial stupidity of GPT to create better underlying models than simply LLM. Basically LLM with a better language model (ontology) as noted above. Will be great improvement, except for the same fatal flaw. A better model of the left-brain, with even more ignorance of the right.