In-Game UI

Build HUDs and menus from UINode elements - screen-space text, panels, buttons, and images that render as a DOM overlay above the canvas, look identical in the editor and the shipped game, and are driven from scripts by name.

The model

A UINode is a screen-space widget attached to an object, exactly like any other component. Four kinds cover the range:

• text - a HUD label (score, timer, prompt). Non-interactive.
• panel - a filled rounded box, for menu backgrounds and dialogs.
• button - a clickable panel that fires a script handler.
• image - a sprite or texture (a HUD frame, icon, portrait) scaled to fit its box. Non-interactive.

The key idea is that UI is real DOM, not drawn geometry. Every frame, a UIOverlay reconciles your UINode objects into <div> and <button> elements layered over the WebGPU canvas - creating, updating, and removing elements to match the world. That overlay runs identically while you edit, in the editor's Game view, and in the exported player, so what you lay out is exactly what your players see - it is WYSIWYG, and it ships with the scene because a UINode serialises like any other component. (Buttons only fire their handlers in the Game view and the shipped game; see How clicks are dispatched.)

Anchoring and layout

Every UINode is pinned to one of nine anchor points on the screen, then offset from there by x / y pixels. This is what keeps a HUD in the right corner across any window size - a top-right element stays glued to the top-right whether the canvas is 800px or 4K wide.

The offset direction follows the anchor:

• On a left edge, x pushes rightward from the left; on a right edge, x pushes leftward from the right.
• On a top edge, y pushes down; on a bottom edge, y pushes up.
• A centred axis (center, or the centred half of top/bottom/left/right) positions at 50% and uses the offset as a nudge from centre.

So a score label at top-left with x: 20, y: 20 sits 20px in from the top-left corner, and a bottom element with y: 40 floats 40px above the bottom edge, horizontally centred.

Layout is computed by a single pure function, uiNodeStyle(node), that maps a UINode to a CSS style object - one source of truth shared by the editor and the player, and unit-tested without a DOM.

The UINode fields

Field	Type	Default	Meaning
kind	enum	text	text | panel | button | image.
anchor	enum	top-left	One of the nine anchor points.
x	number	20	Horizontal pixel offset from the anchor.
y	number	20	Vertical pixel offset from the anchor.
w	number	0	Width in px; 0 = auto (fit content).
h	number	0	Height in px; 0 = auto.
text	string	"Text"	The label / button caption.
fontSize	number	24	Font size in px.
color	color	1,1,1	Text / foreground colour (RGB, 0–1).
bg	color	0,0,0	Background colour for panels / buttons.
opacity	number	0.5	Background alpha, 0–1 (also the image kind's alpha).
visible	bool	true	Scripts toggle this to show / hide.
image	string	""	image kind: the imported texture/sprite id to draw. api.ui.setImage swaps it.
fit	enum	contain	image kind: how the sprite fills the w×h box - contain (letterbox), cover (fill + crop), stretch.
rotation	number	0	Clockwise degrees about the element's centre. Mostly driven by UI animation.
scaleX / scaleY	number	1	Scale about the centre; 1 = normal. UI animation pulses elements by scaling.
font	string	""	Font family for text/buttons - an imported font name, or "" for the system font.

Kind changes the chrome: text is transparent with a drop shadow and ignores pointer events; panel and button get a filled rgba(bg, opacity) background, padding, and an 8px rounded corner; only button captures clicks (cursor: pointer); image draws its sprite as a scaled background-image and is click-through. Setting visible to false (or disabling the component) hides the element with display: none.

The pixel fields (x, y, w, h, fontSize) are raw pixels by default, so a HUD tuned on a large canvas looks oversized on a small one. Turn on HUD scaling to make every pixel dimension scale with the window while each element stays pinned to its anchor.

Creating UI in the viewport

Right-click in the Hierarchy (or use its ＋ button) → Create → UI, and pick:

• Text - a top-left label reading "Text".
• Panel - a centred 260×140 box at 60% black.
• Button - a centred blue button captioned "Button".
• Image - a centred 128×128 image box; assign a sprite in the Inspector (empty until you do).

The new element appears immediately in the overlay (WYSIWYG). Select it and tune the anchor, offsets, colours, sprite, and text in the Inspector. Give the object a clear name in the Hierarchy - scripts address UI elements by that name.

(The sibling 2D submenu - Sprite and 2D Camera - is a different thing: those are world-space quads drawn by the 3D renderer, not screen-space overlay elements. See Camera.)

📸 Screenshot - save as img/media-ui-hud.png

A Game view HUD: a top-left "Score: 120" text element, a centred semi-transparent panel, and a blue "Start" button, with one UINode selected in the Hierarchy and its fields shown in the Inspector.

Driving UI from scripts

Scripts reach the UI two ways, both addressing elements by their entity name.

api.ui - the UIApi for individual UINode elements:

Method	Effect
setText(name, text)	Change an element's text.
setImage(name, imageId)	Set an image element's sprite (an imported texture id) - swap a HUD icon or portrait.
show(name)	Set visible = true.
hide(name)	Set visible = false.
onClick(name, cb)	Register a click handler for a button element.
play(name, clip)	Play a UI animation clip on a widget (adds/restarts its UIAnimator).
stop(name)	Freeze a widget's UI animation.

api.setHud(text) - a shortcut for a single line of HUD text, routed to the game overlay's HUD sink. Use it for a quick throwaway readout without wiring up a named element.

export default class Game implements ScriptBehavior {
  private score = 0;

  onStart(api: ScriptApi) {
    // Wire the "StartButton" element's click to a handler.
    api.ui.onClick("StartButton", () => {
      api.ui.hide("StartButton");
      api.ui.show("ScoreLabel");
    });
  }

  onUpdate(api: ScriptApi) {
    api.ui.setText("ScoreLabel", `Score: ${this.score}`);
    // Or, without a named element:
    api.setHud(`Time ${api.time.toFixed(1)}`);
  }
}

Register onClick handlers in onStart; they reset when the scene changes or Play stops.

How clicks are dispatched

The overlay wires a real DOM click listener on each button. When clicked, it resolves the object's current name and calls UIState.fire(name), which invokes whatever handler your script registered with api.ui.onClick under that name. Keeping the click registry (UIState) separate from the DOM is what lets gameplay logic be tested headlessly - a test can fire("StartButton") without a browser.

Note that buttons are interactive only in the Game view and the shipped player - in the editor's edit mode the overlay renders for layout (so you see it), but clicks do not fire handlers.

Images

An image UINode draws a sprite or texture (a HUD frame, an icon, a portrait) as a scaled background-image. Set its image field to an imported texture id in the Inspector, or swap it at runtime with api.ui.setImage(name, id). fit controls how the sprite fills the w×h box: contain fits the whole sprite (letterboxed), cover fills and crops, stretch distorts to the exact box. With w = h = 0 the box is the sprite's own pixel size. The overlay resolves the texture id to a blob URL (the same embedded bytes the GPU textures come from), so an image element ships with the scene like any other node. An unresolved id draws nothing (an empty box) rather than failing.

UI animation

A UINode's numeric fields can be keyframed and played back, so a HUD widget can fade, slide, pulse, or spin. Two pieces drive it:

• A UIClip - the keyframe data (packages/core/src/uiClip.ts): a name, duration, loop flag, and a list of UITracks. Each track has a target (a child element's name, "" = the widget root), a prop, and a sorted list of { t, v } keys sampled by linear interpolation. Animatable props are x, y, w, h, opacity, rotation, scaleX, scaleY, and the colour channels colorR / colorG / colorB.
• A UIAnimator component on the widget's root: fields clip (the UIClip id), playing, speed, and the runtime time playhead. Its clip's tracks address this entity and its descendant UINodes by name.

Each frame, before the overlay syncs, driveUIAnimators advances every enabled, playing UIAnimator by dt · speed, samples its clip at the new time (looping if the clip loops, else clamping at the end), and writes the values onto the resolved UINodes. The overlay then renders those values, so the widget animates.

Drive it from a script by widget name:

api.ui.play("Reticle", "spin");   // attach/restart a UIAnimator playing the "spin" clip
api.ui.stop("Reticle");           // freeze it

UIClips come in through imports (a Unity .anim becomes a UIClip; a .controller's default state auto-attaches a UIAnimator) - there is no in-editor clip-authoring tool yet. See the import notes. UIClips ship in the exported game (SceneFile.uiClips) and the player runs the same driveUIAnimators, so an imported HUD widget animates in the shipped game exactly as in the editor Game view.

Fonts

A UINode's font field names a font family; "" uses the system UI font. Custom fonts arrive as part of an imported HUD / interface pack: each .ttf / .otf is registered as a browser @font-face (family = the font name) and stored in the asset store, so a UINode set to that family renders in it. There is no standalone "import a font" action - fonts come in with the pack. In the editor Game view the face is injected immediately; in the exported game the font bytes ride SceneFile.fonts (base64) and the player registers each @font-face at boot, so the shipped HUD uses the pack's typeface.

HUD scaling

By default UINode pixel fields are raw pixels, so a HUD authored on a 1440p canvas is undersized on a 4K screen. HUD scaling makes the whole HUD resolution-independent, like Unity's CanvasScaler "Scale With Screen Size" (height match).

Set it in the editor's Render Settings → UI / HUD → HUD scaling: pick a reference height (Off, 720p, 1080p, 1440p, 2160p). The value is RenderSettings.uiReferenceHeight (0 = off). When it is greater than 0, every UINode scales by overlay height ÷ uiReferenceHeight - offsets, size, font, padding, and radius all scale together, while each element stays pinned to its anchor. uiNodeStyle(node, scale) applies the factor, recomputed each sync so it tracks a live window resize. The same setting drives the editor Game view and the shipped player. Importing a HUD/menu pack turns it on at 1080 automatically.

Editing UI in the scene view

Screen-space elements ignore the 3D camera, so you edit them with a dedicated overlay gizmo rather than the translate/rotate handles. The ◱ toggle button (Scene view, top-left, just below the stats ⓘ button) turns UI editing on: click a HUD element to select it, drag its body to move, or drag the resize handles to change its size. Each drag writes back to the UINode's x / y / w / h as one undoable step, and it honours HUD scaling (a box measured in scaled screen pixels converts back to design pixels). The gizmo lives only in the editor Scene view; the runtime overlay stays pointer-inert there, and the shipped player never shows it. Turn the toggle off to orbit the camera freely over the UI again.

📸 Screenshot - save as img/media-ui-edit-gizmo.png

The Scene view with UI editing on (the ◱ button highlighted): a HUD panel selected, its eight resize handles visible, mid-drag on a corner handle.

See also

• Scripting API - api.ui, api.setHud, and the full surface.
• Scripting: UI - patterns for menus, HUDs, and buttons.
• Render Settings - the HUD scaling (uiReferenceHeight) control.
• Import Overview - where imported fonts and UI animation clips come from.
• Hierarchy - naming objects so scripts can find them.
• Audio - the companion overlay system for sound.
• Shipping: Player - how the overlay ships in the exported game.