Semantic Core Clarification
Many concepts that we use are inherently ambiguous. Terms like God, truth, and goodness are complex and variable concepts that lack a tangible referent – compared to concrete terms like tables and chairs.
Semantic Core Clarification (SCC) reduces the ambiguity of concepts by finding the tangible concepts that cumulatively approximate the their meaning.
Approach
The last hidden layer of Large Language Models (LLMs) represents the relative meaning of words (to that model) in a semantic space. Words that are close to each other in that space (e.g. truck and car) have a similar meaning, and words that are far apart (e.g. truck and sandwich) have a dissimilar meaning.
When the coordinates of specific words in this space are added together, the resulting vector represents the composite meaning of those words. The composite meaning can be inferred from the vectors that are nearest to the composite. For example, when the coordinates of boy, king and heir are combined, the resulting vector is very close to the coordinates for prince. This aligns with our intuitive understanding of the concept of a prince – a prince is a boy who is heir to be king.
SCC searches through the semantic space of LLMs to find which terms, when combined, most closely approximate the position of target words in that space.
Application
To clarify the concept of God, the hidden layer of two LLMs was probed: DeciLM-7B and Llama-3-8B. Both of these models are top performers in their respective classes as of April 2024.
All possible 3-word combinations of 1000 common abstract nouns were computed, and their respective positions were calculated relative to the position of the "God" vector in each semantic space (using cosine similarity, where θ = 1 represents a perfect concordance).
In the Llama-3 semantic space, the combination of fortune, awe, and law produced a vector that was closest to the God vector (θ = 0.975). For comparison, in this space the combination of Lord, Yahweh and Elohim was farther from the God vector (θ = 0.844)
In the DeciLM semantic space, the combination of spirit, law, and greed produced a vector that was closest to the God vector (θ = 0.905). Again, in this space the combination of Lord, Yahweh and Elohim was farther from the God vector (θ = 0.890)
Why this matters for AI research
SCC reveals how AI models conceptualize the world. The biases imparted into these models through corpus selection and fine-tuning are reflected in the semantic space of these models.
The DeciLM and the Llama-3 models converge in their attribution of law to the concept of God. Though the Deci model tilts the concept in a negative direction with the attribution of greed.
By elucidating how AI models see the world, SCC can provide critical insight into the inherent biases and blindspots of these models, and thereby contribute significantly to the ongoing conversation about AI ethics.
Understanding ourselves
Insofar as there is agreement between LLMs – or unbiased models are used in SCC – the resulting semantic map can provide insight into our collective consciousness. This is because LLMs encode the semantic maps that are contained in our minds.
The DeciLM and the Llama-3 models converge in their attribution of law to the concept of God. This suggests that a normative element is central to the concept of God, independent of the corpus and fine-tuning.
By rank ordering the concept combinations in both semantic spaces, we find three terms to closely approximate the meaning of God in both spaces: law, will, and wealth (Llama θ = 0.962; Deci θ = 0.898; ). This suggests, that at its core, our collective conception of God is derived from our intuitions of normativity, volition, and abundance.
Techniques
Determine how AI models conceptualize the world.
Uncover the motivational first principles of AI models.
Map the hierarchy of value contained within AI models.