fool-cardgame/extensions/cocos-code-mode/README.md

Code Mode for Cocos Creator

Code Mode turns the Cocos Creator Editor into an AI-controllable tool. It runs an HTTP server inside the editor that exposes scene manipulation, asset management, and property inspection as structured tool calls via UTCP Protocol — letting AI agents build, inspect, and modify Cocos Creator projects the same way a developer would through the UI. These tools are combined in UTCP Code Mode environment to achieve maximum performance and token efficiency for AI agents, letting them call the tools in isolated JS sandbox.

Tools

Category	Tools	Purpose
Scene	nodeGetTree, nodeGetAtPath, nodeCreate, nodeCreatePrimitive, nodeOperate	Navigate and build scene hierarchy
Components	nodeComponentsGet, nodeComponentAdd, nodeComponentRemove, nodeGetAvailableComponentTypes	Attach, remove, and discover components
Inspector	inspectorGetInstanceDefinition, inspectorGetSettingsDefinition, inspectorGetInstanceProperties, inspectorGetSettingsProperties, inspectorSetInstanceProperties, inspectorSetSettingsProperties	Introspect types and read/write properties
Assets	assetGetTree, assetGetAtPath, assetCreate, assetImport, assetOperate, assetGetPreview	Browse, create, and manage project assets
Editor	editorOperate, editorGetLogs, editorGetScenePreview	Control editor state and capture previews

How It Works

Code Mode follows a discover, then act pattern. AI agents never guess at property names or component structures — they query for the real definitions first.

1. Get the scene tree         →  find the node you need
2. Get its type definition    →  learn its actual properties
3. Set properties by name     →  make precise changes

Example

// Find the camera node
const tree = CocosEditor.nodeGetTree({});
const cameraRef = tree.children[0].components[0]; // component reference

// Discover what properties Camera has
const def = CocosEditor.inspectorGetInstanceDefinition({ reference: cameraRef });
// → "export class Camera { fov: number; near: number; far: number; ... }"

// Set multiple properties in one call
CocosEditor.inspectorSetInstanceProperties({
  reference: cameraRef,
  propertyPaths: ["fov", "near", "far"],
  values: [60, 0.1, 1000]
});

Architecture

Tool Execution

The extension runs an Express.js HTTP server on a configurable port (default: auto-assigned). Tool handlers execute asynchronously using Cocos Creator's Editor Message API — all editor interactions go through Editor.Message.request, which marshals calls to the appropriate editor subsystem.

• Read tools (GET) — query scene state and return structured data immediately.
• Write tools (POST) — mutate scene state and call Editor.Message.request('scene', 'snapshot') to register the change as an undoable step.

This means AI agents can safely chain read calls and batch writes without blocking the editor.

Tool Discovery

Tools are TypeScript class methods decorated with @utcpTool. The ToolRegistry collects them at startup, builds JSON schemas from the inline definitions, and serves a UTCP manual at the /utcp endpoint.

export class SceneTools {

    @utcpTool(
        'nodeGetTree',
        'Get the hierarchy tree of specific node or scene root if no reference is provided.',
        {
            type: 'object',
            properties: {
                reference: InstanceReferenceSchema
            }
        },
        SceneTreeItemSchema, "GET", ['scene', 'graph', 'node', 'hierarchy', 'tree']
    )
    async nodeGetTree(args: { reference?: IInstanceReference }): Promise<ISceneTreeItem> {
        // ...
    }
}

Instance References

Nodes, components, and assets are passed around as lightweight UUID-based handles:

{ id: "a1b2c3d4-...", type: "cc.Camera" }

Returned by tree queries, component lookups, and creation tools. Passed back to any tool that needs to target a specific object.

TypeScript Definitions

Code Mode dynamically generates TypeScript class definitions from the live editor property dump. When an AI agent calls inspectorGetInstanceDefinition, it receives a complete TypeScript class with the correct field names, types, enums, and decorator hints — including @property attributes like min, max, unit, and tooltip.

// Example output for a Transform-like node
export class Node {
    readonly uuid: string;
    /** World position */
    worldPosition: Vec3;
    /** World rotation (euler angles) */
    worldRotation: Vec3;
    worldScale: Vec3;
    active: boolean;
}

Components without special handling are reflected automatically from the serialized property dump. Common Cocos math types (Vec2, Vec3, Vec4, Color, Rect, Quat, Mat4, Gradient, etc.) are always available via inspectorGetSettingsDefinition({ settingsType: 'CommonTypes' }).

Settings Inspection

Two special settings types can be inspected and modified directly:

• CurrentSceneGlobals — ambient light, skybox, shadows, and other scene-level rendering settings.
• ProjectSettings — engine and project configuration.

Installation

From release

1. Download last release from this repository.
2. Open Cocos Creator, go to Extension → Extension Manager, and click Import Extension File(.zip) button (icon with arrow).
3. Select the downloaded zip file.
4. The UTCP server starts automatically and registers itself in ~/.utcp_config.json by default.

Build from source

1. Clone this repository.
2. Install node and npm.
3. run

git clone https://github.com/romarogov/cocos-code-mode.git
cd cocos-code-mode
npm i
npm run package

4. If everything builds fine, cocos-code-mode.zip file should appear in repository root.
5. Install it in Cocos Creator with Extension Manager.

Adding Custom Tools

You should add custom tools right in extension package and build it from source as described above.

Implementation example:

import { utcpTool } from './utcp/decorators';

export class MyTools {

    @utcpTool(
        'myCustomTool',
        'Describe what this tool does',
        {
            type: 'object',
            properties: {
                input: { type: 'string' },
                count: { type: 'number', default: 10 }
            },
            required: ['input']
        },
        { type: 'object', properties: { result: { type: 'string' } } },
        "POST",
        ['custom', 'tags']
    )
    async myCustomTool(args: { input: string, count?: number }): Promise<{ result: string }> {
        // Implementation
    }
}

Register the class by importing it in utcp-server.ts. Tools are served automatically at startup. No additional registration needed.

UTCP Call Templates Configuration

The extension provides a Configuration panel accessible from the Code Mode top menu. It shows the current server port, the path to the UTCP config file, and lets you manage additional UTCP call templates to connect other UTCP-compatible tool providers (including MCP servers) into the same Code Mode execution context.

You can find Call Template structures in UTCP documentation:

• MCP Call Template
• HTTP Call Template (Streamable, SSE)
• CLI Call Template
• Text Call Template

The extension automatically maintains a CocosEditor entry in UTCP Config pointing to the running server port.

Integration

Code Mode works with any UTCP-compatible client, including the Code Mode MCP server for AI assistants.

MCP Server Config

{
  "mcpServers": {
    "code-mode": {
      "command": "npx",
      "args": ["@utcp/code-mode-mcp"],
      "env": {
        "UTCP_CONFIG_FILE": "~/.utcp_config.json"
      }
    }
  }
}

Claude Code Configuration

To set up a Claude Code agent to use Code Mode, open your project and run:

Linux/MacOS:

claude mcp add --transport stdio --env UTCP_CONFIG_FILE="~/.utcp_config.json" -- code-mode npx @utcp/code-mode-mcp

Windows:

claude mcp add --transport stdio --env UTCP_CONFIG_FILE="%userprofile%/.utcp_config.json" -- code-mode cmd /c npx @utcp/code-mode-mcp