DOCS · ARCHITECTURE · MEMORY · TENSOR AS SUBSTRATE

Tensor as the universal substrate

Memory and model share the same primitive. EAV → KV → tensor is one decomposition chain.

Every other agent memory system stacks property-graph over relational-DB over disk, and pays a serialisation tax at each layer crossing. The engine inside ioboxx-core does the opposite: it picks one primitive — the tensor — and stores memory in it natively. That primitive is also the only thing every AI model speaks. There is no impedance mismatch between the memory layer and the model layer, because they share the data structure.

The decomposition chain

Each rung of the chain below is the same data viewed from a tighter angle. The base case is tensor — every other primitive reduces to it.

A relational row has fixed columns. An EAV row is (entity, attribute, value) — one fact at a time. A key-value pair is (key → value) with the entity implied. A tensor cell is the same fact again, this time as a single non-zero entry in a sparse n-dimensional array. The EAV row (entity_id, attribute_key, value) is exactly one non-zero cell in a sparse 3-tensor with axes Entity × Attribute × Value.

In the code

The Node struct in crates/core/src/types.rs:216 holds four coordinated tensor slices. Three live inline on the node; the embedding tensor is attached separately in crates/vectors/src/index.rs (TurboQuant 3-bit compressed, indexed for HNSW search) and joined by NodeId.

// crates/core/src/types.rs
pub struct Node {
    pub id: NodeId,
    pub node_type: TypeId,
    pub attributes: BTreeMap<FieldKey, TypedValue>,  // sparse row of attribute tensor
    pub edges: Vec<Edge>,                            // sparse entries in relation tensor
    pub surprise: f32,                               // scalar (0-tensor) per node
    pub momentum: f32,                               // scalar (0-tensor) per node
    pub created_at: u64,
    pub updated_at: u64,
}

// crates/vectors/src/index.rs — attached by NodeId
compressed: HashMap<NodeId, CompressedVector>,       // dense row of embedding tensor

Step back to the graph level and four coordinated tensors appear. The BTreeMap is how a sparse row is stored when most cells are empty — the same idea as scipy CSR or PyTorch sparse COO.

Tensor	Shape	Density	What it is
Attribute tensor	N × M field-keys × TypedValue	sparse	What each node knows about itself — every BTreeMap entry is one non-zero cell.
Relation tensor	N × K edge-types × N	sparse	How nodes link — exactly RESCAL’s 3-way KG tensor; each Edge is one non-zero cell.
Embedding tensor	N × 384	dense (TurboQuant 3-bit)	Semantic fingerprint per node; compressed for the on-device budget.
Relevance tensors	N × 1 (surprise, momentum)	scalar	Inline Titans-style memory signals; smallest tensor slice on the node.

The graph is a view of these four tensors. Nothing more is added; the relation tensor is exactly RESCAL's 3-way knowledge-graph tensor, and the embedding tensor is exactly what every retrieval system already stores. They just happen to live on the same struct.

Every AI model speaks tensor

Tensors aren't used by AI. Tensors are AI's data substrate. Everything else — CSV, JSON, SQL rows, EAV rows, property-graph nodes — has to be converted into tensors before a model can touch it.

Thing in AI	Tensor?
Every weight in every neural net (Claude, GPT, Gemini, Llama, …)	Tensor
Every input embedding fed to a model	Tensor
Every hidden state between layers	Tensor
Q, K, V matrices inside attention	Tensor
The KV cache — a stack of tensors per token	Tensor
Knowledge-graph embeddings (RESCAL, DistMult, ComplEx, TuckER, TransE — all tensor factorisations of EAV)	Tensor
GNN message passing (GCN, GAT, GraphSAGE — operate on the adjacency tensor directly)	Tensor
PyTorch / TensorFlow / JAX / MLX — one primary type: Tensor	Tensor

Knowledge-graph embeddings make this explicit. RESCAL (Nickel et al. 2011) defines a KG as a 3-way tensor X ∈ ℝ^(n×n×m) where X[i,j,k] = 1 if relation k holds between entities i and j. DistMult, ComplEx, TuckER, and TransE are all different factorisations of that same tensor. The knowledge graph and the tensor are the same object viewed from two angles.

The crucial claim

EAV decomposed all the way down is a sparse tensor. ioboxx-core stores its memory natively in the same primitive that lives inside the AI model. There is no impedance mismatch between the memory layer and the model layer — they share the data structure.

This is what makes the engine architecturally different. Mem0, Zep, Cognee, GraphRAG, and HippoRAG all stack property-graph over relational-DB over disk and serialise at every layer crossing. The node struct here is a stack of tensor slices, sitting in RAM, ready to be read by an AI model with zero translation. The 4.4× memory reduction documented under EAV → inline-Node migration is not the headline result — it is the consequence of having picked the right primitive.

Smartphone reframe

A phone with an on-device LLM plus the engine is not “AI plus a database.” It is one tensor substrate — model and memory living on the same primitive in the same RAM. The NPU reads attention KV tensors from one region of memory and node attribute tensors from another; both are tensors, both are read with the same primitives. No serialisation at the boundary, because there is no boundary.

This is why 3-bit embedding compression matters. It is not an optimisation — it brings the tensor substrate into the budget the on-device model already operates in. Same primitive, same RAM, same compression class. The memory reduction is a consequence of having picked the right primitive, not a headline number.

Why this wasn't documented before

The EAV page explains why EAV is the right logical model. The KV-cache page explains what the KV-cache mechanism is. The Titans page explains surprise-gated memory. None of them name tensor as the universal floor that EAV → KV → property-graph all reduce to, and none connect the engine's data schema to the AI model's native primitive. The decomposition was implicit in the code from the day the Node struct was written — it just never got a page. This is that page.