I am considering “ontology generation”. I have not yet read the specifics of these techniques.
Still, the point must surely be to identify some kind of cooccurrences / associations between words. Because, I think of an “ontology” in two ways:
One, it assumes that concepts of the world are the words. The ontology is not trying to discover new concepts. It is only trying to detect the certain kinds of relationships between the concepts that its provided.
Two, an ontology generation is a deeper structure - the lexemes are mere generators of a cathedral of conceptual potentialities. Because, we can always combine concepts to form new ones. In a way, the words we have do not determine or limit what concepts exist, but are only small deposits from a much larger, cavernous realm. Two basic supportive notions of this are that new words regularly come into being; and that anything at the end of the day can be considered a concept, even a disparate unity of concepts. Concepts are just like sets, they both contain and are contained.
Assume you are trying to link or associate words in some way that implies that somehow meaningful, and non-arbitrary, linkages have occurred.
It seems more or less impossible that this could happen with words alone, since their virtue is not explicitly imprinted on them. So we have to make the basic elements of the technique some sort of word in context. And I believe this is already a famous idea, an embedding.
I guess there must be a wide range of ways to embed words, I would like to think more about that. I think you can take them from a certain context they are already in, or I think a really standard technique is to create a vector of every context a word had in every unit (say, sentence, or text) in your corpus.
I was basically thinking that once you generate that ontology, there seem to be a number of particular ways you can operate within that space, and one realizes how distinctively mathematical it is.
For example, imagine some stochastic / random walk from a point (concept) in the ontology, perhaps being optimized by some criterion - sort of like gradient descent, you could arguably move from a random “concept” and iteratively tend towards a concept with some particular feature you were looking for. In other words, you can search your ontology for a concept which you do not know the identity of, but which you believe exists, and which you are able to describe a little bit. This is just like in mathematics where someone can consider some hypothetical object which would have a particular property, and then ask: but can anyone prove the existence of such an object?
You could also do this perfusively, meaning the same thing but many at the same time, at a saturation high enough, that you believe yourself to be finding all concepts in the space meeting a certain criterion. This is like finding the complete set of elements of some type.
You can judge how “similar” two concepts are to one another - since concepts are composite, you can consider how much of their internalities they share, like set intersection. Or, I assume there are other ways to explore “similarity”, intrinsic and extrinsic. (And I believe that “similarity” is often taken as a distance metric on that space, making this a metric space.)
Or, you can take two concepts, and take the union of them. If we tried to using a generating method that also had some degree of non-arbitrariness, maybe - a limit on how random or disparate combinations could be - then the union of those two concepts may not be in the ontology space. But, there could be an algorithm which recognizes the nearest point in concept space that they have in common; and bring it into the set. It could keep doing this, finding the nearest point the current set tends to, until it finally attains resolution by growing or shrinking to a concept considered legitimate, and stable. This is like the Least Common Multiple in multiplication. Isn’t this also exactly like topology? Like, you have to take the union of subsets in a valid way; if you don’t you create something that’s not part of the topology.
Also, it is possible the ontology generation might recognize how some concepts seem to have a kind of primality or terminality - they cannot be reduced. They are the prime numbers of a concept space. The rest of the concepts are composite.
I feel like we should abstract away the notion that the points are meant to represent words/concepts and just look at this purely mathematically. What kind of object/space do we have here? It would be fascinating to identify, because then, importing all the known theorems of mathematics, we could wonder how they elucidate interesting properties, or even useful, appliable techniques, in some way.
