Добавил Cocos Code Mode
This commit is contained in:
@@ -0,0 +1,219 @@
|
||||
# Code Mode for Cocos Creator
|
||||
|
||||
**Code Mode** turns the Cocos Creator Editor into an AI-controllable tool. It runs an HTTP server inside the editor that exposes scene manipulation, asset management, and property inspection as structured tool calls via [UTCP Protocol](https://www.utcp.io/) — letting AI agents build, inspect, and modify Cocos Creator projects the same way a developer would through the UI.
|
||||
These tools are combined in [UTCP Code Mode](https://github.com/universal-tool-calling-protocol/code-mode/) environment to achieve maximum performance and token efficiency for AI agents, letting them call the tools in isolated JS sandbox.
|
||||
|
||||
## Tools
|
||||
|
||||
|
||||
|
||||
| Category | Tools | Purpose |
|
||||
|----------|-------|---------|
|
||||
| **Scene** | `nodeGetTree`, `nodeGetAtPath`, `nodeCreate`, `nodeCreatePrimitive`, `nodeOperate` | Navigate and build scene hierarchy |
|
||||
| **Components** | `nodeComponentsGet`, `nodeComponentAdd`, `nodeComponentRemove`, `nodeGetAvailableComponentTypes` | Attach, remove, and discover components |
|
||||
| **Inspector** | `inspectorGetInstanceDefinition`, `inspectorGetSettingsDefinition`, `inspectorGetInstanceProperties`, `inspectorGetSettingsProperties`, `inspectorSetInstanceProperties`, `inspectorSetSettingsProperties` | Introspect types and read/write properties |
|
||||
| **Assets** | `assetGetTree`, `assetGetAtPath`, `assetCreate`, `assetImport`, `assetOperate`, `assetGetPreview` | Browse, create, and manage project assets |
|
||||
| **Editor** | `editorOperate`, `editorGetLogs`, `editorGetScenePreview` | Control editor state and capture previews |
|
||||
|
||||
|
||||
## How It Works
|
||||
|
||||
**Code Mode** follows a **discover, then act** pattern. AI agents never guess at property names or component structures — they query for the real definitions first.
|
||||
|
||||
```
|
||||
1. Get the scene tree → find the node you need
|
||||
2. Get its type definition → learn its actual properties
|
||||
3. Set properties by name → make precise changes
|
||||
```
|
||||
|
||||
### Example
|
||||
|
||||
```typescript
|
||||
// Find the camera node
|
||||
const tree = CocosEditor.nodeGetTree({});
|
||||
const cameraRef = tree.children[0].components[0]; // component reference
|
||||
|
||||
// Discover what properties Camera has
|
||||
const def = CocosEditor.inspectorGetInstanceDefinition({ reference: cameraRef });
|
||||
// → "export class Camera { fov: number; near: number; far: number; ... }"
|
||||
|
||||
// Set multiple properties in one call
|
||||
CocosEditor.inspectorSetInstanceProperties({
|
||||
reference: cameraRef,
|
||||
propertyPaths: ["fov", "near", "far"],
|
||||
values: [60, 0.1, 1000]
|
||||
});
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Tool Execution
|
||||
|
||||
The extension runs an Express.js HTTP server on a configurable port (default: auto-assigned). Tool handlers execute asynchronously using Cocos Creator's Editor Message API — all editor interactions go through `Editor.Message.request`, which marshals calls to the appropriate editor subsystem.
|
||||
|
||||
- **Read tools** (GET) — query scene state and return structured data immediately.
|
||||
- **Write tools** (POST) — mutate scene state and call `Editor.Message.request('scene', 'snapshot')` to register the change as an undoable step.
|
||||
|
||||
This means AI agents can safely chain read calls and batch writes without blocking the editor.
|
||||
|
||||
### Tool Discovery
|
||||
|
||||
Tools are TypeScript class methods decorated with `@utcpTool`. The `ToolRegistry` collects them at startup, builds JSON schemas from the inline definitions, and serves a UTCP manual at the `/utcp` endpoint.
|
||||
|
||||
```typescript
|
||||
export class SceneTools {
|
||||
|
||||
@utcpTool(
|
||||
'nodeGetTree',
|
||||
'Get the hierarchy tree of specific node or scene root if no reference is provided.',
|
||||
{
|
||||
type: 'object',
|
||||
properties: {
|
||||
reference: InstanceReferenceSchema
|
||||
}
|
||||
},
|
||||
SceneTreeItemSchema, "GET", ['scene', 'graph', 'node', 'hierarchy', 'tree']
|
||||
)
|
||||
async nodeGetTree(args: { reference?: IInstanceReference }): Promise<ISceneTreeItem> {
|
||||
// ...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Instance References
|
||||
|
||||
Nodes, components, and assets are passed around as lightweight UUID-based handles:
|
||||
|
||||
```typescript
|
||||
{ id: "a1b2c3d4-...", type: "cc.Camera" }
|
||||
```
|
||||
|
||||
Returned by tree queries, component lookups, and creation tools. Passed back to any tool that needs to target a specific object.
|
||||
|
||||
### TypeScript Definitions
|
||||
|
||||
Code Mode dynamically generates TypeScript class definitions from the live editor property dump. When an AI agent calls `inspectorGetInstanceDefinition`, it receives a complete TypeScript class with the correct field names, types, enums, and decorator hints — including `@property` attributes like `min`, `max`, `unit`, and `tooltip`.
|
||||
|
||||
```typescript
|
||||
// Example output for a Transform-like node
|
||||
export class Node {
|
||||
readonly uuid: string;
|
||||
/** World position */
|
||||
worldPosition: Vec3;
|
||||
/** World rotation (euler angles) */
|
||||
worldRotation: Vec3;
|
||||
worldScale: Vec3;
|
||||
active: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
Components without special handling are reflected automatically from the serialized property dump. Common Cocos math types (`Vec2`, `Vec3`, `Vec4`, `Color`, `Rect`, `Quat`, `Mat4`, `Gradient`, etc.) are always available via `inspectorGetSettingsDefinition({ settingsType: 'CommonTypes' })`.
|
||||
|
||||
### Settings Inspection
|
||||
|
||||
Two special settings types can be inspected and modified directly:
|
||||
- **`CurrentSceneGlobals`** — ambient light, skybox, shadows, and other scene-level rendering settings.
|
||||
- **`ProjectSettings`** — engine and project configuration.
|
||||
|
||||
## Installation
|
||||
|
||||
### From release
|
||||
|
||||
1. Download last release from this repository.
|
||||
2. Open Cocos Creator, go to **Extension → Extension Manager**, and click `Import Extension File(.zip)` button (icon with arrow).
|
||||
3. Select the downloaded zip file.
|
||||
4. The UTCP server starts automatically and registers itself in `~/.utcp_config.json` by default.
|
||||
|
||||
### Build from source
|
||||
|
||||
1. Clone this repository.
|
||||
2. Install `node` and `npm`.
|
||||
3. run
|
||||
```bash
|
||||
git clone https://github.com/romarogov/cocos-code-mode.git
|
||||
cd cocos-code-mode
|
||||
npm i
|
||||
npm run package
|
||||
```
|
||||
4. If everything builds fine, `cocos-code-mode.zip` file should appear in repository root.
|
||||
5. Install it in Cocos Creator with **Extension Manager**.
|
||||
|
||||
## Adding Custom Tools
|
||||
|
||||
You should add custom tools right in extension package and build it from source as described above.
|
||||
|
||||
Implementation example:
|
||||
|
||||
```typescript
|
||||
import { utcpTool } from './utcp/decorators';
|
||||
|
||||
export class MyTools {
|
||||
|
||||
@utcpTool(
|
||||
'myCustomTool',
|
||||
'Describe what this tool does',
|
||||
{
|
||||
type: 'object',
|
||||
properties: {
|
||||
input: { type: 'string' },
|
||||
count: { type: 'number', default: 10 }
|
||||
},
|
||||
required: ['input']
|
||||
},
|
||||
{ type: 'object', properties: { result: { type: 'string' } } },
|
||||
"POST",
|
||||
['custom', 'tags']
|
||||
)
|
||||
async myCustomTool(args: { input: string, count?: number }): Promise<{ result: string }> {
|
||||
// Implementation
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Register the class by importing it in `utcp-server.ts`. Tools are served automatically at startup. No additional registration needed.
|
||||
|
||||
## UTCP Call Templates Configuration
|
||||
|
||||
The extension provides a **Configuration** panel accessible from the **Code Mode** top menu. It shows the current server port, the path to the UTCP config file, and lets you manage additional UTCP call templates to connect other UTCP-compatible tool providers (including MCP servers) into the same Code Mode execution context.
|
||||
|
||||
You can find Call Template structures in [UTCP documentation](https://www.utcp.io/protocols):
|
||||
- [MCP Call Template](https://utcp.io/protocols/mcp#call-template-structure)
|
||||
- [HTTP Call Template](https://utcp.io/protocols/http#call-template-structure) ([Streamable](https://utcp.io/protocols/http#call-template-structure), [SSE](https://utcp.io/protocols/http#call-template-structure))
|
||||
- [CLI Call Template](https://utcp.io/protocols/cli#call-template-structure)
|
||||
- [Text Call Template](http://utcp.io/protocols/text#call-template-structure)
|
||||
|
||||
The extension automatically maintains a `CocosEditor` entry in UTCP Config pointing to the running server port.
|
||||
|
||||
## Integration
|
||||
|
||||
Code Mode works with any UTCP-compatible client, including the [Code Mode MCP server](https://github.com/universal-tool-calling-protocol/code-mode/?tab=readme-ov-file#even-easier-ready-to-use-mcp-server) for AI assistants.
|
||||
|
||||
### MCP Server Config
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"code-mode": {
|
||||
"command": "npx",
|
||||
"args": ["@utcp/code-mode-mcp"],
|
||||
"env": {
|
||||
"UTCP_CONFIG_FILE": "~/.utcp_config.json"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Claude Code Configuration
|
||||
|
||||
To set up a Claude Code agent to use Code Mode, open your project and run:
|
||||
|
||||
Linux/MacOS:
|
||||
``` bash
|
||||
claude mcp add --transport stdio --env UTCP_CONFIG_FILE="~/.utcp_config.json" -- code-mode npx @utcp/code-mode-mcp
|
||||
```
|
||||
|
||||
Windows:
|
||||
``` powershell
|
||||
claude mcp add --transport stdio --env UTCP_CONFIG_FILE="%userprofile%/.utcp_config.json" -- code-mode cmd /c npx @utcp/code-mode-mcp
|
||||
```
|
||||
Reference in New Issue
Block a user