Building videos with AI coding agents

How It Fits Together

Four pieces connect the AI agent to the player runtime:

Skill Installation

Each agent has its own skill package that teaches it the video-making workflow. The instructions are identical; only the file layout differs.

Claude Code

Install three files into your project:

your-project/
├── CLAUDE.md                          # Skill instructions (append to existing)
└── .claude/
    ├── commands/
    │   └── make-video.md              # Enables /make-video slash command
    └── references/
        └── player-api.md             # Player API reference for context

Invoke with /make-video or describe your video and Claude Code picks up the workflow from the CLAUDE.md instructions.

Codex

Place the skill folder in the Codex skills directory:

~/.codex/skills/app-video-studio-video-maker/
├── SKILL.md                           # Skill instructions
├── agents/
│   └── openai.yaml                    # Agent metadata and invocation policy
└── references/
    └── player-api.md                 # Player API reference

Invoke with $app-video-studio-video-maker. The skill supports implicit invocation, so Codex can also activate it automatically when video tasks are detected.

Component Catalog

The catalog is a set of JSON endpoints auto-generated from the template registry during deploy. The index at /api/catalog.json is lightweight and links to individual resources.

Endpoints

How agents use it

The skill instructs the agent to fetch the catalog index, pick templates by summary, then fetch only the individual template endpoints it needs for control schemas and asset slot requirements. This avoids loading a single large file. The agent can also query templates at runtime via player.listTemplates() and player.getTemplate(id) if a live player instance is available.

Regenerating

Regenerate the catalog with the same internal build task used during deploys.

That task reads the template registry and writes the index to website/api/catalog.json, the document model to website/api/catalog/document-model.json, and individual templates to website/api/catalog/templates/*.json.

SDK Package

@appvideostudio/sdk provides document helpers, the player web component, and the test harness. It ships as ES modules.

npm install @appvideostudio/sdk

Entry points

Document helpers

The main entry point exports functions for building and validating video documents:

import {
  createEmptyVideoDocument,  // Create a new document with defaults
  validateVideoDocument,     // Returns array of {code, path, message} errors
  normalizeVideoDocument,    // Sanitize and fill missing fields
  exportVideoDocumentFromState,
  getDocumentDuration,
  getDocumentTimelineEnd,
  getTimelineWindow,
  sortTimeline,
  createDefaultLayers,
  DEFAULT_PROJECT_DIMENSIONS,  // { width: 1080, height: 1920 }
  DEFAULT_TIMELINE_DURATION,   // 24
  PLAYER_SCHEMA_VERSION,       // 1
} from '@appvideostudio/sdk';

Player element

The <avs-player> web component renders video documents in an iframe-isolated player. Key methods:

const player = document.querySelector('avs-player');
await player.ready;

// Load and play
await player.load(videoDocument);
await player.preloadAssets();
await player.play();

// Query
const status = await player.getStatus();
const errors = await player.getErrors();
const assets = await player.getAssets();

// Mutate
await player.addComponent(definition);
await player.updateComponentParams(id, patch);
await player.updateComponentTiming(id, { startTime, duration });
await player.removeComponent(id);
await player.replaceDocument(newDoc);

Supports autoplay, loop, no-controls, defer, and auto-pause attributes. See the API Overview for the full attribute reference.

Test Harness

The test harness wraps Playwright to load a document in the real player runtime, run commands, and capture screenshots. Playwright is an optional peer dependency.

npm install --save-dev @playwright/test

Basic usage

import { test } from '@playwright/test';
import { createPlayerHarness } from '@appvideostudio/sdk/test-harness';
import { createEmptyVideoDocument } from '@appvideostudio/sdk';

test('video renders correctly', async ({ page }) => {
  const doc = createEmptyVideoDocument({ name: 'test' });
  // ... add components to doc ...

  const harness = await createPlayerHarness(page, {
    document: doc,
    // playerUrl: 'https://create.appvideostudio.com', (default)
    // validate: true,    (validates before loading)
    // width: 1280,
    // height: 720,
  });

  // Playback
  await harness.play();
  await harness.seek(3);
  await harness.pause();

  // Screenshots
  const frame = await harness.screenshotAt(2.5);
  const sequence = await harness.screenshotSequence([0, 3, 6, 9]);

  // Query state
  const status = await harness.getStatus();
  const errors = await harness.getErrors();

  // Mutate and re-check
  await harness.updateComponentParams('comp-1', { title: 'Updated' });
  await harness.removeComponent('comp-2');

  await harness.destroy();
});

Harness options

End-to-End Workflow

This is the sequence the skill teaches an AI agent to follow. Whether you're using Claude Code or Codex, the workflow is the same:

# 1. Agent reads skill instructions (CLAUDE.md or SKILL.md)
#    Learns the brief → audit → plan → build → iterate loop

# 2. Agent fetches /api/catalog.json (index)
#    Gets template list, then fetches individual template endpoints as needed

# 3. Agent builds a video document using SDK helpers
import { createEmptyVideoDocument, validateVideoDocument } from '@appvideostudio/sdk';

const doc = createEmptyVideoDocument({
  name: 'Launch Video',
  dimensions: { width: 1080, height: 1920 },
  duration: 20,
});

# 4. Agent adds components, assets, and timeline entries to the document
#    Uses catalog data to set correct params and asset bindings

# 5. Agent validates the document
const errors = validateVideoDocument(doc);

# 6. Agent loads into player and previews
#    Uses <avs-player> directly or the Playwright test harness

# 7. Agent iterates on feedback, then exports

The skill files, catalog, SDK, and test harness are independent pieces. You can use any of them on their own—but together they give an AI agent everything it needs to go from a text prompt to a validated, previewable video composition.