System Full View

System Overview: From UE5 to Core

The value of LiteRT-LM lies not in a single inference, but in this incremental architecture deeply optimized for Unreal Engine long-conversation scenarios.

Three-Layer Progressive Architecture

graph TD subgraph UE5[Unreal Business Layer - UmgMcp Provider] Agent[Agent Object] Tools[Tool Definitions] end subgraph UnrealApi[Integration Abstraction - LiteRtLmUnrealApi] Subsystem[Global Subsystem] Sync[Incremental Sync Engine] Norm[Role Normalizer] end subgraph DLL[Isolated Driver - litert_lm_wrapper.dll] ABI[ABI Firewall] Worker[Background Inference Thread] end subgraph Core[Inference Core - LiteRT Core] GPU[GPU Inference] KV[Persistent KV Cache] end Agent --> Subsystem Subsystem --> Sync Sync --> Norm Norm --> ABI ABI --> Worker Worker --> GPU GPU --- KV style DLL fill:#1E293B,stroke:#3B82F6,stroke-width:2px style Core fill:#000,stroke:#8B5CF6,stroke-width:2px

Business Integration

Handles FabServer message models, mapping in-game NPC logic to AI requests.

Integration Abstraction

Implements incremental message synchronization. Maintains one-to-one persistent mapping between Session and Agent.

Physical Isolation

Completely resolves symbol conflicts via ABI firewall. Background thread drives WaitUntilDone heartbeat.

Incremental Sync Model

sequenceDiagram participant UE as UE5 Subsystem participant DLL as Wrapper Session Note over UE,DLL: Round 1: Initial Conversation UE->>DLL: Append(Msg 1-3) UE->>DLL: RunInference() Note right of DLL: Save to VRAM KV Cache Note over UE,DLL: Round 2: Follow-up UE->>UE: Detect Synced = 3 UE->>DLL: Sync New Messages Only (Msg 4) UE->>DLL: RunInference() Note right of DLL: Cache Hit, $O(\Delta N)$ Computation

Physical KV Cache Persistence

In LiteRtLmUnrealApi.cpp, the system precisely tracks the synced message index for each Agent via SessionMsgCountMap. Switching Agents is like switching pointers in GPU memory, enabling zero-overhead multi-Agent interaction.

"This is the core 'black magic' that reduces Time To First Token (TTFT) from seconds to milliseconds."

Role Normalization Logic

Since underlying models (like Gemma-2) do not natively support system or tool roles, the UnrealApi layer performs complex protocol downgrading:

System Message Merged into User

Tool Result Structured User Msg

Code Snippet (Normalization logic)

if (Role == "tool") {
  NewMsg->SetStringField("role", "user");
  NewMsg->SetStringField("content", 
    "[Tool Result] (id: " + id + ")\n" + Content);
}

Inference Driving Sequence

sequenceDiagram participant GT as Game Thread (UE5) participant BT as Background Worker participant DLL as Wrapper DLL (C) participant GPU as GPU Queue GT->>BT: AsyncTask: RunInference activate BT BT->>DLL: LiteRtLm_RunInference() DLL->>GPU: Submit Task loop Driving Loop BT->>DLL: WaitUntilDone(blocking) GPU-->>DLL: Output Token DLL-->>GT: Callback (AsyncTask/GameThread) GT->>GT: Update STextBlock UI end BT->>GT: Signal Completion deactivate BT

GameThread: UI & Logic Worker: WaitUntilDone (Engine Heartbeat) GPU: Heavy Lifting