AI NPC Framework

Description

AI NPC Brain is a Unity-focused system that turns NPCs into LLM-driven characters: they chat in character, remember facts about the player, track relationship stats (affinity, trust, fear, respect, flags), and can suggest game actions as structured events. Those suggestions are validated against a whitelist (your event catalog) before anything runs, so the model can’t arbitrarily trigger unknown gameplay.

It ships with pluggable backends (OpenAI, local Ollama, and a mock fallback for offline/UI testing), file-based persistence for conversation and memory, sample UI, demo scene, and editor tools (e.g. animation/catalog setup).

Technical Features

  • LLM-backed NPC dialogue — NPCs reply in character from a persona you define (ScriptableObject + JSON), with OpenAI, local Ollama, or an offline mock for testing.
  • Persistent memory — Stores facts about the player across sessions (file-based), with limits so prompts stay bounded.
  • Relationship model — Tracks affinity, trust, fear, respect, narrative flags, and stage; each turn can apply clamped deltas from the model.
  • Structured, safe game events — The model returns JSON; only catalog-whitelisted events can fire, with typed args, optional allowed values, and a confidence threshold.
  • Conversation history — Rolling turn history in the prompt (configurable depth) plus relevant memories retrieval.
  • World context — worldStateCompact on the brain injects free-form game state into prompts (time, location, inventory snippets, etc.).
  • Unity integration — Central AiNpcBrain component with UnityEvents (OnNpcReply, OnEmotionChanged, OnAnyNpcEventPayload, etc.) for UI, animation, and gameplay hooks.
  • Animation bridge — NpcAnimationMap maps semantic animation IDs to Animator triggers; NpcAnimationReceiver plays them from validated events.
  • UI workflow — Sample TMP chat UIs (simple and scroll-view), optional window IDs and WindowRegistryRuntime for opening shop/quest/relationship panels.
  • Editor tooling — Tools > AI NPC Brain (e.g. NPC Animation Setup) to sync Animator triggers with the animation map and event catalog.
  • Demo content — Demo scene, sample assets, and prefabs so buyers can press Play after wiring an API key or Ollama.
  • Unity 6 + URP oriented — Documented for Unity 6 and URP, with standard dependencies (Input System, Newtonsoft JSON, TextMesh Pro).

Documentation

Event System

Event System

The event system is how NPC dialogue translates into gameplay actions. The LLM suggests events as part of its structured response, and the system validates and dispatches them through a strict pipeline before anything happens in-game.


Event Lifecycle

LLM Response
  └─ event_suggestions: [
       { "event_id": "shop.open", "confidence": 0.95, "args": { "shop_id": "..." } },
       { "event_id": "anim.trigger", "confidence": 0.8, "args": { "anim_id": "nod" } }
     ]
        │
        ▼
  ┌─────────────────────────────┐
  │ 1. Confidence Gate          │  event.confidence >= config.fireConfidenceThreshold?
  │    (NpcBrainConfig)         │  No → event silently skipped
  └──────────────┬──────────────┘
                 │ Yes
                 ▼
  ┌─────────────────────────────┐
  │ 2. Normalization            │  AiNpcBrain fills in NPC-specific args:
  │    (AiNpcBrain)             │  - shop_id from NpcUiBindings
  │                             │  - quest_board_id from NpcUiBindings
  │                             │  - relationship_id from NpcUiBindings
  │                             │  - anim_id fallback from context
  └──────────────┬──────────────┘
                 │
                 ▼
  ┌─────────────────────────────┐
  │ 3. EventGate Validation     │  Checks against NpcEventCatalog:
  │    (EventGate.cs)           │  - Event exists in catalog?
  │                             │  - All required args present?
  │                             │  - Arg types correct?
  │                             │  - Values in allowed list?
  │                             │  - No unexpected extra args?
  └──────────────┬──────────────┘
                 │ Pass
                 ▼
  ┌─────────────────────────────┐
  │ 4. Payload Dispatch         │  JSON string fired via:
  │    (UnityEvent<string>)     │  OnAnyNpcEventPayload.Invoke(payload)
  └──────────────┬──────────────┘
                 │
        ┌────────┴────────┐
        ▼                 ▼
  NpcAnimationReceiver  AureliaDemoReceiver
  (or your receiver)    (or your receiver)

Event Catalog Setup

The NpcEventCatalog ScriptableObject is the whitelist. Every event the NPC can suggest must be defined here.

Defining an Event

Each EventDefinition has:

Field Example Purpose
eventId shop.open The unique identifier used in payloads.
description "Opens the NPC's shop for the player to browse and buy items." Included in the LLM prompt so it understands when to suggest this event.
args [{ name: "shop_id", type: String, allowedValues: ["npc_aurelia_alchemist.shop"] }] Schema for required arguments.

Standard Events

These are the events used in the demo. You can add, remove, or modify them.

Shop Events

Event ID Args Description
shop.open shop_id (String) Opens the NPC's shop inventory.
shop.sell_mode shop_id (String) Opens the shop in sell/buyback mode.

Quest Events

Event ID Args Description
quest.offer quest_id (String) The NPC offers a specific quest.
quest.open_board quest_board_id (String) Opens the quest board panel.
quest.turn_in quest_id (String) The player completes a quest.

Relationship Events

Event ID Args Description
relationship.open_status relationship_id (String) Opens the relationship status display.
relationship.request_stage_up relationship_id (String) NPC requests advancing the relationship stage.
relationship.require_item relationship_id (String) NPC requires an item to progress the relationship.
relationship.give_gift relationship_id (String) Player gives a gift to the NPC.

Animation Events

Event ID Args Description
anim.trigger anim_id (String, constrained to allowed values) Triggers an NPC animation.
anim.move_to move_id (String, constrained to allowed values) Moves the NPC to a predefined position.

UI Events

Event ID Args Description
ui.show_hint hint_id (String) Shows a UI hint or tooltip.

EventGate Validation Rules

EventGate.TryBuildPayload() performs these checks in order:

  1. Catalog exists — returns false if no catalog is assigned.
  2. Event ID exists — the event_id must match an entry in NpcEventCatalog.events.
  3. Args object exists — the suggestion must have a non-null args dictionary.
  4. Required args present — every arg defined in the catalog schema must be provided.
  5. Type validation — each arg value must parse as the expected type (String always passes, Int/Float/Bool must parse).
  6. Allowed values — if the schema arg has a non-empty allowedValues list, the provided value must be in that list.
  7. No extra args — any arg key not in the schema is rejected.

If all checks pass, a minimal JSON payload is built and returned:

{"event_id":"shop.open","args":{"shop_id":"npc_aurelia_alchemist.shop"}}

Event Normalization

Before validation, AiNpcBrain.NormalizeEventSuggestion() patches the event args to ensure consistency:

Event ID Normalization
shop.open, shop.sell_mode Forces shop_id to the value from NpcUiBindings.shopWindowId.
quest.open, quest.board Forces quest_board_id to NpcUiBindings.questBoardWindowId.
relationship.* Forces relationship_id to NpcUiBindings.relationshipWindowId.
anim.trigger If anim_id is missing, auto-selects one based on reply text, emotion, and user input.
anim.move_to If move_id is missing, defaults to step_forward.

This means even if the LLM omits or gets an arg wrong, the system fills in the correct NPC-specific value.


Auto-Animation Fallback

If the LLM doesn't suggest any anim.trigger event in a turn, AiNpcBrain automatically picks an animation based on context:

Context Animation Chosen
Reply contains "hello", "hi", "greetings" wave or smile
Reply contains "yes", "of course", "certainly" nod or smile
Reply contains "maybe", "i'm not sure" shrug or think
Emotion = happy smile or nod
Emotion = curious nod or point
Emotion = sad or afraid shrug
Generic fallback nodsmileblinkidle_shift

This ensures the NPC always has some body language, even without explicit LLM guidance.


Receiving Events

NpcAnimationReceiver

Handles anim.trigger and anim.move_to events.

Animation triggers: Looks up the anim_id in NpcAnimationMap to find the actual Animator trigger name, then calls Animator.SetTrigger().

Movement: Supports these built-in move_id values:

move_id Behavior
step_forward Moves the NPC forward by stepDistance units.
step_back Moves the NPC backward by stepDistance units.
approach_player Moves to the playerAnchor transform.
leave_player Moves to the idleAnchor transform.

AureliaDemoReceiver

Routes non-animation events to UI panels via WindowRegistryRuntime. It:

  1. Parses the JSON payload to extract event_id and args.
  2. Normalizes common model aliases (e.g. quest.openquest.open_board).
  3. Looks up the target window ID (from args or NpcUiBindings fallback).
  4. Closes all panels, then opens the matching one.
  5. Supports Escape to close all panels.

Writing a Custom Event Receiver

To handle events your own way:

public class MyEventReceiver : MonoBehaviour
{
    public void OnEventPayload(string payload)
    {
        // Parse the JSON payload
        // payload format: {"event_id":"...","args":{"key":"value"}}

        // Route to your game systems
    }
}

Then wire AiNpcBrain.OnAnyNpcEventPayloadMyEventReceiver.OnEventPayload in the Inspector.


Adding a Custom Event

  1. Open your NpcEventCatalog asset in the Inspector.
  2. Add a new entry to the events list:
  3. eventId: your unique ID, e.g. trade.barter
  4. description: explain to the LLM when to suggest it
  5. args: define required arguments with types and optional allowed values
  6. Write a receiver that handles this event ID.
  7. Wire it to OnAnyNpcEventPayload.

The LLM will automatically see the new event in its prompt and can start suggesting it in contextually appropriate moments.