Description: In this edition of QuEST, Michael Robinson will discuss topological features in large language models
Key Moments and Questions in the video include:
Acknowledgement of colleagues from DARPA and Galois
Manifolds in machine learning
LLM token space is higher dimensional
Manifold spaces tend to be negatively curved
LLM turn text into vectors
Transformers turn vectors into new text
How do we turn the text into vectors?
We think of LLM as being trained on all human language, but they have not
GPT2 Open source LLM as the source for model
ChatGPT2 used as the example
Tokens have topology and geometry
Words are a categorical variable
Vectors are a numerical variable
Mixing data types can lead to some problems
Why...
As National Domestic Violence Awareness Month comes to an end, Airmen from Altus Air Force Base, Oklahoma, highlighted resources both on base and throughout the community to shed light on domestic violence and empower victims.