Organizations are surprisingly good at forgetting.
Selections are made on calls, insights get buried in Slack threads, and a month later nobody remembers why issues are the way in which they’re.
Arkency isn’t any exception.
Weekly calls, ad-hoc conferences, our e book golf equipment, Slack discussions, GitHub mentions, e-mail inbox – we may use some help in organizing all these alerts.
Then Ruby Neighborhood Convention 2026 occurred in March.
In Kraków, Obie Fernandez confirmed some components of his NEXUS system.
He had already described it on his weblog again in January, however the convention was the place I first got here throughout it.
That was the push I wanted to begin constructing our personal software program.
When it was already taking form, Andrej Karpathy printed his LLM Wiki observe.
As a substitute of a RAG system rediscovering your paperwork on each question, an LLM incrementally maintains a persistent wiki: interlinked markdown pages, immutable sources beneath, and a human curating the loop.
It was fairly thrilling to comprehend I used to be engaged on one thing that had simply turn out to be one of many hottest matters within the business.
We ended up with Planet Arkency – a multi-tenant information graph with a closed ontology, constructed on Rails Occasion Retailer.
On this put up, I need to stroll you thru the design selections I made.
Unstructured enter is the place LLMs really shine
For structured information, you would have constructed such a system like twenty years in the past.
Webhooks, types, integrations – parsing structured enter right into a graph is a solved drawback.
However probably the most attention-grabbing information lives within the enter no parser may ever deal with: assembly transcripts, Slack discussions, emails, or something coming from an integration no one has constructed but.
That is the place LLMs modified the sport for us.
Every little thing flows into the system by means of a single ingestion endpoint.
Transcripts, Slack threads somebody flagged with a devoted emoji response, emails arriving at a bridge inbox, RSS feeds, calendar invitations, private notes.
We don’t even write code for the combination factors.
Instruments like Zapier or n8n watch the sources and push the content material to that single endpoint.
Each ingested piece of content material then goes by means of an extraction – the guts of the system.
An LLM reads the content material and works out what it means for our information: which entities seem in it, what we discovered about them, and the way they relate to one another.
Most of this put up is about what occurs round that single step.
Why a graph?
The identical names maintain coming again in our conversations: individuals, tasks, shoppers, instruments, selections.
What modifications from week to week is what we learn about them and the way they relate to one another.
That maps naturally to a graph: entities with attributes, linked by typed relations.
Who works on what.
Who made which determination, and when.
Which challenge will depend on which software.
That is the place we differ most from the LLM Wiki strategy.
In a wiki, the truth that somebody works on some challenge is written down in a sentence on a web page, at finest with a hyperlink between the 2 pages.
The information is there, however solely a reader could make use of it.
In a typed graph, particular person --works_on--> challenge is a chunk of information: you may question it, traverse it, depend it.
The graph itself sits on PostgreSQL: a nodes desk, an edges desk with a novel (supply, goal, relation) triple, jsonb attributes on each – no rocket science right here.
Devoted graph databases (Neo4j, triple shops just like the one NEXUS makes use of) could possibly be a greater match for some particular workloads, like deep multi-hop traversal.
However nothing above is Postgres-specific.
A schema this plain is what makes the storage an precise element – there may be little to port while you write one other adapter for the information layer.
The ontology
Which sorts of nodes and relations might exist is outlined in an ontology, saved in a plain YAML file:
# from config/ontology.yml
node_kinds:
- form: particular person
description: "crew member, candidate, shopper contact, exterior particular person"
- form: determination
description: "formal determination requiring group verdict — for informal strategies use thought"
edge_relations:
- relation: works_on
signature: "particular person --works_on--> challenge"
The ontology is closed – if a form or relation shouldn’t be on the checklist, the mannequin can not use it.
Initially I used to be excited about an open ontology, the place the LLM may introduce its personal varieties.
It introduced full chaos into the graph surprisingly quick.
For my part, it’s higher to inform the mannequin upfront what to search for.
Not one graph, however many
“The organizational information graph” suggests one common graph for all completely different functions.
We don’t consider in that, and DDD practitioners will acknowledge why.
We use a multi-tenant structure to keep up separate graphs with their very own ontologies, which actually means their very own ubiquitous languages.
Our inner Arkency graph speaks in individuals, tasks and selections – a site fairly near a CRM.
The graph we run as Rails Occasion Retailer maintainers speaks in releases, recognized issues and group content material:
Completely different domains, completely different vocabularies, the identical equipment beneath.
The boundaries of a bounded context let you know the place one graph ends and one other begins.
The ontology is rendered into the extraction immediate as markdown tables and into the schema as enums.
(from app/lib/prompts/extraction.md.erb)
You're an organizational information analyst for <%= Tenancy.current_tenant.title %>. We're constructing an inner information graph.
Extract a information graph from the supplied content material: nodes and edges. The graph ought to enable full reconstruction of the supplied content material.
## Nodes
Every node has: title, form, short_description, description, attrs (elective key-value pairs).
Allowed varieties:
| form | what it represents | typical attrs |
|---|---|---|
<% ontology.node_kinds.every do |ok| -%>
| <%= ok.fetch("form") %> | <%= ok.fetch("description") %> | <%= ok.fetch("attrs", []).then { |attrs| attrs.empty? ? "—" : attrs.map { |a| a.is_a?(Hash) ? (a["values"] ? "#{a["name"]} (#{a["values"].be part of(", ")})" : a["name"]) : a }.be part of(", ") } %> |
<% finish -%>
## Edges
Every edge has: supply, goal, relation, context, attrs (elective key-value pairs).
Allowed relations:
| relation | supply form | goal form | trace | attrs |
|---|---|---|---|---|
<% ontology.edge_relations.every do |r| -%>
<%
sig = Ontology.parse_signature(r.fetch("signature"))
source_kind = sig[:source].be part of("https://weblog.arkency.com/")
target_kind = sig[:target].be part of("https://weblog.arkency.com/")
trace = r["hint"] || "—"
attrs = r.fetch("attrs", []).then { |a| a.empty? ? "—" : a.map { |at| at.is_a?(Hash) ? (at["values"] ? "#{at["name"]} (#{at["values"].be part of(", ")})" : at["name"]) : at }.be part of(", ") }
-%>
| <%= r.fetch("relation") %> | <%= source_kind %> | <%= target_kind %> | <%= trace %> | <%= attrs %> |
<% finish -%>
...
Every extraction ends with the mannequin returning one structured consequence: the entities it discovered within the content material, the relations between them, and the way the present graph ought to change to mirror them.
We use RubyLLM’s schema help for that.
# from app/lib/extraction_result_schema.rb
array :nodes, description: "Entities to create or replace. Every title have to be distinctive — no duplicate nodes." do
object do
string :standing, enum: ["new", "existing"], description: "'present' iff the node was returned by search_nodes/list_nodes_by_kind/get_node_edges and you're reusing it. 'new' if you're introducing it. The system verifies the canonical title and aborts on mismatch."
string :title, description: "Entity title. For 'present' nodes use the EXACT canonical title from the software name consequence. For 'new' nodes the canonical title you're introducing."
string :new_name, required: false, description: "Non-obligatory. Set ONLY for 'present' nodes when the content material reveals a extra specific canonical type (e.g. acronym → full time period, diminutive → full title). The node is regarded up by `title` and renamed to `new_name`."
string :form, description: "Should be considered one of: #{kind_names}"
string :short_description, description: "Secure synthesis of what this entity is (for search). Basic and identity-focused, not episode-specific. Max 15 phrases."
string :description, description: "For brand spanking new nodes: temporary description based mostly on the content material. For present nodes: synthesize prior description with new info. Rewriting for readability is ok, however protect prior details."
array :attrs, description: "Key-value attributes. Solely embody what is understood from the content material." do ... finish
array :aliases, required: false, description: "Non-obligatory. Various floor types (diminutives, acronyms, full vs brief types) underneath which this entity was referred to within the content material, or — when renaming through `new_name` — the outdated canonical if it stays a legitimate floor type. Solely embody NEW aliases not already current on the present node. An alias is the SAME entity underneath one other title — by no means a separate entity." do ... finish
finish
finish
array :edges, description: "ALL relationships. Be thorough and exact." do
object do
string :supply, description: "Supply node title (actual match — present or newly created)"
string :goal, description: "Goal node title (actual match — present or newly created)"
string :relation, description: "Should be considered one of: #{relation_names}"
string :context, description: "Briefly clarify why this relationship exists, grounded within the content material"
array :attrs, description: "Key-value attributes for this edge (e.g. since, weight)" do ... finish
finish
finish
Precise information operations (create or replace, with the precise field-level diff) are derived server-side.
We load or initialize an ActiveRecord mannequin, assign what the LLM returned, and let soiled monitoring do the remaining:
# from app/handlers/propose_graph_change.rb
node = Node.find_or_initialize_by(title: information[:name])
enforce_status!(information[:name], information[:status], node) # raises when the mannequin's new/present declare disagrees with the DB
was_new = node.new_record?
node.assign_attributes(short_description: ..., description: ..., attrs: node.attrs.merge(attrs))
modifications = node.modifications.besides("updated_at", "created_at", "form", "slug")
{ op: was_new ? "create" : "replace", node_id: node.endured? ? node.id : nil, modifications: modifications }
node.modifications provides us {area => [before, after]} pairs totally free, and this earlier than/after snapshot turns into the wire format of the graph change proposal.
Edges get precisely the identical remedy – regarded up by their (supply, goal, relation) triple and diffed with soiled monitoring.
We additionally don’t blindly belief what the LLM claims.
It has to declare every node as new or present, and a validator cross-checks it towards the database.
On mismatch, the LLM will get natural-language suggestions and one other try on the identical dialog.
Id decision is the arduous half
I simply wrote that the mannequin has to declare every node as new or present.
However how wouldn’t it know?
Will we load the entire graph into LLM context?
No – that is the place software calls are available.
And it’s tougher than a easy lookup.
“Piotrek”, “Piotr Jurewicz” and no matter Zoom’s transcription makes out of my title are the identical particular person.
In case you create a node per floor type, your graph turns into rubbish inside per week.
We deal with it on three ranges.
First, the mannequin should look earlier than it writes.
Throughout extraction it has entry to read-only instruments like search_nodes or get_node_edges.
The extraction immediate is specific about it:
(from app/lib/prompts/extraction.md.erb)
- Earlier than creating any node, use search_nodes to test if it already exists. (...)
- If search_nodes returns no outcomes, the node doesn't exist but — proceed to create it. (...)
- If search_nodes returns ambiguous outcomes, otherwise you want broader context to make extraction selections, use get_node_edges to examine the node's connections.
- After discovering nodes with search_nodes, use get_node_edges to see their present relationships earlier than deciding learn how to join them.
Second, aliases are the id mechanism.
Every node has one canonical title and any variety of aliases.
The schema instructs the mannequin that an alias is identical entity underneath one other title – by no means a separate entity.
When the content material reveals a greater canonical type, the mannequin units new_name and the outdated title stays as an alias, so future fuzzy searches nonetheless resolve it.
Third, the search is hybrid.
Trigram similarity (pg_trgm with GIN indexes) over node names and aliases catches misspellings.
Embedding search catches semantic matches which share no characters:
# from app/fashions/node.rb
def self.hybrid_search(question, restrict: 10)
# fuzzy match on canonical names and aliases, powered by pg_trgm
by_name = the place("similarity(nodes.title, ?) > 0.3", question)
by_alias = joins(:aliases).the place("similarity(node_aliases.title, ?) > 0.3", question)
trigram_results = union_by_best_similarity(by_name, by_alias)
response = RubyLLM.embed(question, mannequin: "bge-m3", supplier: :ollama)
semantic_results =
nearest_neighbors(:embedding, response.vectors, distance: "cosine")
.choose n
merge_and_rank(trigram_results, semantic_results, restrict)
finish
The embeddings come from a self-hosted bge-m3 mannequin on Ollama, saved in pgvector.
Each truth has a supply
A graph edited by an AI is barely reliable in the event you can audit each change.
For each node and edge we are able to reply: which extraction created you, which extractions up to date you, and what precisely modified every time.
Provenance lives in be part of tables (node_extractions and edge_extractions): one row per extraction and entity pair, holding the operation, the standing, and the field-level diff produced by the soiled monitoring described earlier than.
Ranging from any node, you may stroll again by means of these rows to the extraction that touched it, and from the extraction to the ingested content material it was based mostly on.
Each truth within the graph traces again to its supply.
We additionally document one thing we name the learn set.
Each software name the mannequin makes throughout extraction is printed as an ExtractionToolCalled occasion and projected into tool_invocations, linked to the nodes and edges the decision returned.
So we all know not solely what an extraction wrote, but in addition what it learn earlier than deciding.
If you surprise “why did the mannequin merge these two individuals?”, the reply is on the extraction web page: right here is the search it ran, and here’s what got here again.
Every node’s web page exhibits its full historical past: created in, final up to date in, learn by N extractions.
Maintaining a tally of the prices
In addition to auditing modifications within the graph, we additionally monitor how a lot every extraction prices: token utilization and the ensuing value.
If you work with an LLM API, it’s price holding a finger on the heart beat right here.
A transcript of some hours of dialog, processed in a number of rounds interleaved with software calls, can generate vital prices.
Immediate caching helps loads – the system immediate and the content material keep similar between rounds, so many of the enter is billed on the cache-read price.
The precise numbers rely on the mannequin you run the extraction on, however most of ours price properly underneath a greenback.
Human within the loop
We don’t let the LLM write to the graph instantly.
Extraction produces a proposal with the earlier than/after diffs, and making use of it to the graph is a separate step.
Proposals can sit in a evaluation window earlier than they get utilized.
As quickly as an extraction completes, we get a brief abstract of it on Slack.
A human can examine the diff, apply it early, or simply let it stream after the configured delay.
Time passes between suggest and apply, so the graph might have moved within the meantime.
When the present state not matches what the proposal was based mostly on, the apply stops and the affected rows get marked as conflicted, with a human-readable clarification.
Occasion sourcing ties all of it collectively
You’ll have seen that each mechanism above was described by way of occasions.
Nicely, that is an Arkency weblog in spite of everything.
The entire pipeline is an occasion stream: TranscriptIngested → ExtractionRequested → KnowledgeExtracted → GraphChangeProposed → GraphChangeApplied (or GraphChangeConflicted).
Two small aggregates guard the invariants: one per ingestion (no two concurrent extractions of the identical content material), one per extraction (the suggest → apply state machine).
Every little thing you see within the UI (ingestions, extractions, diffs, software invocations) is a learn mannequin constructed from these occasions.
On this structure, the evaluation window is only one extra state within the mixture’s state machine, and provenance is only one extra learn mannequin constructed from an occasion we already had.
I can not perceive individuals claiming that occasion sourcing makes issues extra complicated 😉
The graph can feed itself
One characteristic exhibits the worth of a uniform pipeline properly.
From any node you may request analysis.
A job asks a mannequin outfitted with Anthropic’s server-side web_search and web_fetch instruments to compile a short in regards to the entity:
# from app/jobs/research_topic.rb
chat = RubyLLM
.chat(mannequin: MODEL)
.with_params(instruments: [
{ type: "web_search_20250305", name: "web_search", max_uses: 10 },
{ type: "web_fetch_20250910", name: "web_fetch", max_uses: 10 }
])
.with_schema(ResearchBriefSchema.construct)
The immediate grounds the analysis in what the graph already is aware of in regards to the entity, and tells the mannequin when to surrender:
# from app/jobs/research_topic.rb
Analysis "#{subject}". Use web_search and web_fetch as wanted to collect details.
In our information base this entity is at present described as:
- Variety: ...
- Brief description: ...
- Attributes: ...
When you may produce a helpful temporary, return standing="accomplished" and put the
temporary in `temporary` as Markdown. (...) Cowl id, key details a educated
reader ought to know, latest exercise price recording, and relationships to
different named entities. Embrace supply URLs inline subsequent to claims that come
from a particular web page. Maintain it factual; don't speculate.
Return standing="aborted" as an alternative — with `abort_reason` naming the particular
drawback — when any of those holds:
- The subject is ambiguous and you can't confidently decide the meant
interpretation from the disambiguation context above.
- You can't discover substantive, verifiable details about this actual
entity (...)
Don't pad an aborted consequence with related-but-different info.
The ensuing temporary shouldn’t be utilized to the graph instantly.
It will get printed as a daily TranscriptIngested occasion with its personal form, and flows by means of the identical extraction, proposal and evaluation pipeline as another enter.
The graph speaks MCP
The graph shouldn’t be locked inside its personal UI.
We expose it over MCP, so any AI assistant with entry to our server can search it by asking questions in pure language – and reply from the graph, with sources.
Last ideas
Engaged on Planet Arkency taught me loads.
About graphs, about LLMs, and about ideas I had by no means even heard of earlier than: ontologies, id decision, provenance.
I hope a few of that information stays with you after studying this put up.
It additionally reassured me in regards to the instruments we now have been utilizing at Arkency for years.
Occasion-driven structure and Rails Occasion Retailer carried this challenge naturally.
On the time of writing, the principle manufacturing Arkency graph holds virtually 2000 nodes and over 5200 edges, constructed by round 300 extractions.
And behind all of that, over 3600 occasions recording how each single truth received there.
I nonetheless have a head filled with concepts on the place to take this challenge subsequent.
Working with RubyLLM was a pure pleasure – credit to Carmine Paolino for this gem.
In case you are excited about organizational reminiscence to your firm, or need us that can assist you construct one, get in contact.
