EN 中文

01 INITIALIZATION & CONFIG

Stage: Before the Loop

Fig 1.1: Physical Resource Allocation on CreateEngine

LiteRtLm_CreateEngine CRITICAL ENTRY

DLL_EXPORT void* LiteRtLm_CreateEngine(LiteRtLm_Config config);

Physical Lock: This function locks VRAM immediately. If VRAM is insufficient, it returns NULL.
ABI Firewall: Returns a void* handle; type casting on the UE5 side is prohibited.
Invocation Timing: Recommended during level loading or Subsystem initialization; do not call repeatedly in the main loop.

struct LiteRtLm_Config

max_num_tokens

Determines the physical pre-allocation size for KV Cache. Once allocated, VRAM usage is fixed.

Formula: VRAM ≈ Weights + (Tokens * 128 KB)

bOptimizeShader

Mandatory for UE5. Set to 1 to compile PSOs at load time, eliminating frame drops on the first token.

ATTENTION: model_path uses mmap. Corrupted model files or invalid paths will cause a direct process crash; verify physical existence before calling.

LiteRtLm_GetAvailableBackends

Environment probing interface. Returns a string of system acceleration capabilities (e.g., "gpu,cpu").

Strategy: If (!str.contains("gpu")) { config.num_threads = 16; }