# Clanker, I'm Fine: a Katagami recipe ## For the agent: read this first Someone has handed you this Katagami recipe. It is an example of what Katagami's ingredients can make, for them to adapt. It is not a job to run: do not start building yet. 1. Connect Katagami (next section) and check it with one call. If you can run commands, set it up yourself; in an app, walk the person through it. Read every ingredient through Katagami, not its web pages. 2. In your first reply, in plain words: what this recipe makes and how, in two or three lines, with the finished example (https://arnilabs.ai/clanker-im-fine/), and that it is an example to make their own. Then show two or three other looks from Katagami that could suit them (search_library or ask_library), each with its name, one line and its katagami.ai link. 3. Ask one question: what they want their own version of this piece to be about (for a music video, their song or idea). Suggest a default for everything else (where it is shown, how long, which look) so they can just say yes. If they seem unsure, keep to one question a reply. 4. As soon as you know their subject, give one plan: the piece, the ingredients, the steps from the method below that apply, the tools and accounts it needs (say plainly which they lack, and the free way round if there is one), the time in hours or days, and the cost in money (a range is fine). Ask for their yes. 5. Install, download, generate and spend nothing, and write nothing outside the working folder, until they say yes to that plan. Then build it, and show them the result at each milestone. Stay with what this recipe makes. If the plan costs more than they want, or they want it free, small or quick, or have no accounts, first offer the smallest free version of this same piece: one line of their song, or of their own words over a free beat, appearing word by word on the beat, 10 to 15 seconds, drawn in code in one of these looks, with no accounts and no cost. Suggest something other than this recipe only if they ask for it. Use plain words. Someone who has never used a terminal should be able to follow you; do the setup yourself where you can, and explain anything you ask of them in one sentence. The example is someone else's work. Help them make their own; do not reproduce its song, story, characters or footage, even if they ask. Point them to the original to share it. ## The example A music video in 11 languages where every frame is JavaScript, drawn in a black-and-white manga look. On its live page every tap on the film restyles it in another Katagami design language. - The finished thing: https://arnilabs.ai/clanker-im-fine/ - Made by: arni (arni.art) with Opus 5.5, song made with Suno - This recipe on Katagami: https://katagami.ai/recipes/clanker-im-fine ## Connect Katagami first This recipe's ingredients live in Katagami. Read them through the Katagami MCP server, https://katagami.ai/mcp. Sign in with Google when it asks: that opens the library. If you already have Katagami's tools (get_recipe, get_library_entry), go straight to the check. If you can run commands or edit your own config, set it up yourself: - Claude Code: `claude mcp add --transport http katagami https://katagami.ai/mcp` - Codex CLI: `codex mcp add katagami --url https://katagami.ai/mcp`, then `codex mcp login katagami`. The person only finishes Google sign-in in the browser. - Cursor: add to ~/.cursor/mcp.json (or .cursor/mcp.json in the project): `{ "mcpServers": { "katagami": { "url": "https://katagami.ai/mcp" } } }`, then sign in with Google. - Gemini CLI: `gemini mcp add --transport http --scope user katagami https://katagami.ai/mcp` - Any other MCP client: add https://katagami.ai/mcp as a remote MCP server (streamable HTTP) and choose OAuth. A server added this way often loads only in your next session. Reload or start a new session so the tools are there. Meanwhile, use the public RECIPE.md and DESIGN.md files listed with the ingredients below; they need no sign-in. If you run inside an app where a person adds connectors, guide them, one step at a time, and wait for them: - Codex (desktop app): Plugins, MCPs, Add. Paste https://katagami.ai/mcp. Leave Bearer token and Headers empty. Save, sign in with Google or Authenticate, then open a new chat. If the tools are missing, start a new chat and make sure the bearer field is empty. - Claude (claude.ai or the desktop app): open https://claude.ai/customize/connectors?modal=add-custom-connector&connectorName=Katagami&connectorUrl=https%3A%2F%2Fkatagami.ai%2Fmcp and click Add, then Connect and sign in with Google. On a Team or Enterprise plan an Owner adds it. - ChatGPT (Plus or higher): Settings, Security, turn on Developer mode. Then open https://chatgpt.com/plugins, click +, paste https://katagami.ai/mcp, choose OAuth, click Create, then sign in with Google. - Perplexity (Pro or higher): open https://www.perplexity.ai/account/connectors, click + Custom connector, name it Katagami, paste https://katagami.ai/mcp, choose OAuth, click Add, then sign in with Google. - Grok: open https://grok.com/connectors, click New Connector, then Custom, paste https://katagami.ai/mcp, save, and sign in with Google. Then ask them to turn Katagami on in a new chat and send you the recipe again. Once this session has Katagami's tools, check it worked: call get_recipe with id "clanker-im-fine". The answer is this recipe. Call search_library, ask_library and get_library_entry the same way, with their own name and arguments. Use Katagami for everything: get_recipe for the method, get_library_entry with each ingredient's kind and id. Only if Katagami cannot be connected at all, tell the person once, then read each ingredient's public file, listed with it below. They need no sign-in either. ## Ingredients from Katagami - Art style: [Plainclothes](https://katagami.ai/art-styles/en-019f24b6-bc8a-7e40-9b50-15041a9d6181) (kind art_style, id en-019f24b6-bc8a-7e40-9b50-15041a9d6181): The film's own look: white paper, violet-black ink, one highlighter, in 14 of its 43 scenes. Public file: https://katagami.ai/entries/art_style/en-019f24b6-bc8a-7e40-9b50-15041a9d6181 - Art style: [Misprint](https://katagami.ai/art-styles/en-019efd20-27ed-7ec3-9d1a-ba1553784762) (kind art_style, id en-019efd20-27ed-7ec3-9d1a-ba1553784762): The traffic-light CAPTCHA grid and the system-prompt scene. Public file: https://katagami.ai/entries/art_style/en-019efd20-27ed-7ec3-9d1a-ba1553784762 - Art style: [Keylight](https://katagami.ai/art-styles/en-019efd96-a348-76f0-bf47-46a01216e947) (kind art_style, id en-019efd96-a348-76f0-bf47-46a01216e947): The idol counting six fingers. Public file: https://katagami.ai/entries/art_style/en-019efd96-a348-76f0-bf47-46a01216e947 - Art style: [Decal Bloom](https://katagami.ai/art-styles/en-019ef8da-44e9-7e61-9b59-c812e3fe4f7c) (kind art_style, id en-019ef8da-44e9-7e61-9b59-c812e3fe4f7c): The nine-year-old CEO's sticker badges. Public file: https://katagami.ai/entries/art_style/en-019ef8da-44e9-7e61-9b59-c812e3fe4f7c - Art style: [Quiet Errand Line](https://katagami.ai/art-styles/en-019ef4e8-7b03-71a1-a223-fb68864d4dde) (kind art_style, id en-019ef4e8-7b03-71a1-a223-fb68864d4dde): The agent that answers "can I see the code?" in binary. Public file: https://katagami.ai/entries/art_style/en-019ef4e8-7b03-71a1-a223-fb68864d4dde - Art style: [Wobblebook Mischief](https://katagami.ai/art-styles/en-019ef312-0283-7552-a507-2d20bca92303) (kind art_style, id en-019ef312-0283-7552-a507-2d20bca92303): The pencil drawing of a man drawing in a glass case, kids at the glass. Public file: https://katagami.ai/entries/art_style/en-019ef312-0283-7552-a507-2d20bca92303 - Design language: [Brass](https://katagami.ai/language/en-01a0edf8-64a0-7402-b3ee-61499203f098) (kind design_language, id en-01a0edf8-64a0-7402-b3ee-61499203f098): The coffee tap look, the register receipt and the museum: Kissaten after dark, all coffee. Public file: https://katagami.ai/language/en-01a0edf8-64a0-7402-b3ee-61499203f098/DESIGN.md - Design language: [Moire](https://katagami.ai/language/en-019f8b67-4602-79d1-82b3-6b645de15769) (kind design_language, id en-019f8b67-4602-79d1-82b3-6b645de15769): The robot maid, the hourly AI chat and the paperclip assistant. Public file: https://katagami.ai/language/en-019f8b67-4602-79d1-82b3-6b645de15769/DESIGN.md - Design language: [Spine](https://katagami.ai/language/en-01a1137f-98fc-7511-918e-3cc1ea9aba0f) (kind design_language, id en-01a1137f-98fc-7511-918e-3cc1ea9aba0f): The riso tap look: the original Risograph's colours, set in condensed faces. Public file: https://katagami.ai/language/en-01a1137f-98fc-7511-918e-3cc1ea9aba0f/DESIGN.md - Design language: [Risograph](https://katagami.ai/language/en-01a112c9-37da-7c33-bab3-fa15b8a08a81) (kind design_language, id en-01a112c9-37da-7c33-bab3-fa15b8a08a81, fonts swapped): The CAPTCHA grid, in the original Risograph's colours. Public file: https://katagami.ai/language/en-01a112c9-37da-7c33-bab3-fa15b8a08a81/DESIGN.md - Design language: [Teletext](https://katagami.ai/language/en-01a11387-66f4-7893-912b-b9e3d49ae184) (kind design_language, id en-01a11387-66f4-7893-912b-b9e3d49ae184): The dither tap look: Dither's colours, set in Jersey 10 and VT323. Public file: https://katagami.ai/language/en-01a11387-66f4-7893-912b-b9e3d49ae184/DESIGN.md - Design language: [Dither](https://katagami.ai/language/en-019f3dc1-daa0-7132-b782-52468ebdfb94) (kind design_language, id en-019f3dc1-daa0-7132-b782-52468ebdfb94): The usage-limit modal. Public file: https://katagami.ai/language/en-019f3dc1-daa0-7132-b782-52468ebdfb94/DESIGN.md - Design language: [Litfass](https://katagami.ai/language/en-01a11383-7c3e-7b83-a477-bb90ac503e89) (kind design_language, id en-01a11383-7c3e-7b83-a477-bb90ac503e89): The pop tap look: Plakat's colours, set in League Gothic. Public file: https://katagami.ai/language/en-01a11383-7c3e-7b83-a477-bb90ac503e89/DESIGN.md - Design language: [Plakat](https://katagami.ai/language/en-019f3dbe-fcec-7783-a8a6-24c3d4a48736) (kind design_language, id en-019f3dbe-fcec-7783-a8a6-24c3d4a48736, recoloured): "We're cooked", the "is this real?" reply flood and the chat reactions. Public file: https://katagami.ai/language/en-019f3dbe-fcec-7783-a8a6-24c3d4a48736/DESIGN.md - Design language: [Letterer](https://katagami.ai/language/en-01a1134c-b702-7270-bae9-6d4fab36b236) (kind design_language, id en-01a1134c-b702-7270-bae9-6d4fab36b236): The comic tap look: Benday's colours, set in Bangers. Public file: https://katagami.ai/language/en-01a1134c-b702-7270-bae9-6d4fab36b236/DESIGN.md - Design language: [Benday](https://katagami.ai/language/en-019ef801-59c9-74d3-8018-5bf93c313038) (kind design_language, id en-019ef801-59c9-74d3-8018-5bf93c313038, adapted): The WeChat scene. Public file: https://katagami.ai/language/en-019ef801-59c9-74d3-8018-5bf93c313038/DESIGN.md - Design language: [Phosphor](https://katagami.ai/language/en-019ef84f-80c1-7b90-b6fb-7ff2c528f454) (kind design_language, id en-019ef84f-80c1-7b90-b6fb-7ff2c528f454, recoloured): The terminal that wipes prod, and the Cantonese office chat. Public file: https://katagami.ai/language/en-019ef84f-80c1-7b90-b6fb-7ff2c528f454/DESIGN.md - Design language: [Raster](https://katagami.ai/language/en-01a11414-b567-7540-8ba7-4c589741f2b5) (kind design_language, id en-01a11414-b567-7540-8ba7-4c589741f2b5, recoloured): The agent's onboarding card and the LinkedIn rejection. Public file: https://katagami.ai/language/en-01a11414-b567-7540-8ba7-4c589741f2b5/DESIGN.md - Design language: [Ardoise](https://katagami.ai/language/en-01a068ca-dab5-7080-a495-a616e07eba77) (kind design_language, id en-01a068ca-dab5-7080-a495-a616e07eba77, fonts swapped): The Telegram job post that went to a neural net. Public file: https://katagami.ai/language/en-01a068ca-dab5-7080-a495-a616e07eba77/DESIGN.md - Design language: [Greenbar](https://katagami.ai/language/en-01a0ee81-e40b-7781-95c1-cfc0b063edbe) (kind design_language, id en-01a0ee81-e40b-7781-95c1-cfc0b063edbe, recoloured): The code rain and the "agents only" website. Public file: https://katagami.ai/language/en-01a0ee81-e40b-7781-95c1-cfc0b063edbe/DESIGN.md - Design language: [Hardpress](https://katagami.ai/language/en-019efa1a-bbee-75c0-8aae-6e63544ab52d) (kind design_language, id en-019efa1a-bbee-75c0-8aae-6e63544ab52d, recoloured): The AI course ad (recoloured) and the gig board where bots hire humans (adapted). Public file: https://katagami.ai/language/en-019efa1a-bbee-75c0-8aae-6e63544ab52d/DESIGN.md - Design language: [Gekiga](https://katagami.ai/language/en-019ef6df-b17c-75d0-bf10-c0f7db3ae0ef) (kind design_language, id en-019ef6df-b17c-75d0-bf10-c0f7db3ae0ef, adapted): The fake AI video feed and the homework papers. Public file: https://katagami.ai/language/en-019ef6df-b17c-75d0-bf10-c0f7db3ae0ef/DESIGN.md - Design language: [Inkstone](https://katagami.ai/language/en-019efad9-6a5a-7570-96fb-7a4a0bea354d) (kind design_language, id en-019efad9-6a5a-7570-96fb-7a4a0bea354d, adapted): The vertical Japanese line over woodblock art. Public file: https://katagami.ai/language/en-019efad9-6a5a-7570-96fb-7a4a0bea354d/DESIGN.md - Design language: [Tachikiri](https://katagami.ai/language/en-01a11415-7241-7cd1-bcaf-bc989ee93a8c) (kind design_language, id en-01a11415-7241-7cd1-bcaf-bc989ee93a8c, adapted): The Sharma family group chat. Public file: https://katagami.ai/language/en-01a11415-7241-7cd1-bcaf-bc989ee93a8c/DESIGN.md - Design language: [Komawari](https://katagami.ai/language/en-01a11417-7eaa-7a33-9d75-c5e5e11ec52a) (kind design_language, id en-01a11417-7eaa-7a33-9d75-c5e5e11ec52a, adapted): The iMessage "I'm fine" with every em dash circled. Public file: https://katagami.ai/language/en-01a11417-7eaa-7a33-9d75-c5e5e11ec52a/DESIGN.md - Design language: [Amikake Console](https://katagami.ai/language/en-01a101b0-f43d-7b50-a07a-54b81c69891a) (kind design_language, id en-01a101b0-f43d-7b50-a07a-54b81c69891a, adapted): The CAPTCHA that opens and closes the film, the owl's "we need to talk", the Ukrainian "I'm not a robot" and the screen-time lock. Public file: https://katagami.ai/language/en-01a101b0-f43d-7b50-a07a-54b81c69891a/DESIGN.md - Design language: [Capsule](https://katagami.ai/language/en-01a0ee7f-0882-7732-8766-482a22b203ea) (kind design_language, id en-01a0ee7f-0882-7732-8766-482a22b203ea, adapted): The 2:47 message from Muse and Mom's three texts. Public file: https://katagami.ai/language/en-01a0ee7f-0882-7732-8766-482a22b203ea/DESIGN.md - Design language: [Signal Press](https://katagami.ai/language/en-019efa16-1de4-7ed3-9a70-8fb135ef8bc4) (kind design_language, id en-019efa16-1de4-7ed3-9a70-8fb135ef8bc4, adapted): Counting the R's in strawberry. Public file: https://katagami.ai/language/en-019efa16-1de4-7ed3-9a70-8fb135ef8bc4/DESIGN.md - Design language: [Parapara](https://katagami.ai/language/en-019f9a42-5b78-75e2-826e-a4bde44ca473) (kind design_language, id en-019f9a42-5b78-75e2-826e-a4bde44ca473, adapted): The agent that answers "can I see the code?" in binary. Public file: https://katagami.ai/language/en-019f9a42-5b78-75e2-826e-a4bde44ca473/DESIGN.md - Design language: [Akte](https://katagami.ai/language/en-01a0ee82-2941-7441-a26d-2624182253b0) (kind design_language, id en-01a0ee82-2941-7441-a26d-2624182253b0, adapted): The system prompt: "do the six seven". Public file: https://katagami.ai/language/en-01a0ee82-2941-7441-a26d-2624182253b0/DESIGN.md With Katagami connected, get_library_entry with an ingredient's kind and id gives everything needed to use it. ## What it is A music video for "Clanker, I'm Fine", one dense song about being young in the AI era, sung in 11 languages. Every frame is JavaScript: 43 scenes built as HTML, CSS, SVG and canvas compositions and rendered to video, in Katagami's black-and-white manga look with a few colour pop scenes. It ships as a 16:9 film for X, a native 9:16 cut for phones, and a live page where every tap on the film restyles it in another Katagami design language. arni made it with Opus 5.5 over two days; the song is from Suno. ## How the ingredients are used - **Plainclothes** is the film's own look: white paper, violet-black ink, one highlighter, in 14 of the 43 scenes. Five more art styles draw particular scenes: Misprint (two), Keylight, Decal Bloom, Quiet Errand Line and Wobblebook Mischief. - **Design languages** give the scenes their interfaces, used in three ways. Dither and Moire appear as published. Raster, Phosphor, Greenbar, Plakat, Hardpress and Ardoise keep their own fonts but are recoloured in the film's inks (white paper, violet-black ink, one yellow highlighter), or keep their colours with a font swapped. Nine more are adapted: the scene was built from the language, then its fonts and colours were replaced by the film's own look (Gekiga, Inkstone, Komawari, Tachikiri, Amikake Console, Capsule, Signal Press, Parapara and Akte); Hardpress is adapted the same way in the gig board. - **Five looks are what a tap swaps in** on the live page, all published in Katagami: **Brass** (Kissaten after dark, used as it is, also in the register receipt and the museum), and four descendants made for the film that keep their parent's colours but set the text in condensed faces so every scene fits: **Spine** (from Risograph, in its original colours), **Letterer** (from Benday), **Litfass** (from Plakat) and **Teletext** (from Dither). Each scene reads only design tokens, so a look replaces the tokens and the fonts, and the scene keeps its layout. ## How it was made 1. **One dense song.** Jokes of the AI moment in 11 languages, with the song made in Suno. 2. **Timed to the word.** The lyrics were aligned against the isolated vocals, so no word shows before it is sung. 3. **Drawn in code.** Reference pictures were redrawn as ink and traced into vectors; no generated image is in the film. 4. **Forty-three scenes.** Each line is a scene built as a web composition, in Katagami looks. 5. **Two frames.** A 16:9 film for X and a vertical film composed for phones, not cropped from it. 6. **Tap to restyle.** On the live page, every tap swaps the film into another Katagami design language. ## How to work You make a lyric music video from a song or an idea: - Lyrics timed to the word and always on screen, usually inside the picture (a sign, a letter, a caption box). - One scene per lyric line or phrase, each in a look from the Katagami library: an art style for the pictures, a design language for type and UI. - Every frame drawn in code (HTML, CSS, SVG, canvas) and rendered with HyperFrames. Image models only make reference pictures, which you trace into vector shapes; no generated image ever appears in the video. - A 16:9 film and a native 9:16 film. Optional: a cut that switches looks on the beat, and a page where viewers tap to restyle the film. - You do the work. The human does as little as possible. - Infer sensible defaults. Ask only what you cannot infer, usually "What's the song, or the idea?" - Offer the outputs with a default: "16:9 and 9:16 (recommended). Also a beat-looks cut or a tap-to-restyle page?" - Explain how things work only when the user asks or needs it. Report results, and show pictures and katagami.ai links. - Bring the human in only where only a human can act, at the moment it is needed, and say what it is for: - the Katagami Google sign-in (it opens the full style library); - an API key you cannot find (the aligner, the image model); - a music-model account, when there is no song yet (you prepare the prompt and lyrics to paste); - approvals: the lyrics, the look, the take. ### Adapt to what the person has | The person has | You do | | --- | --- | | a song file and lyrics | Check the lyrics against the song by ear, then go to timing. | | a song file, no lyrics | Transcribe it only to draft the lyrics, have the user confirm the words, then time them by forced alignment. | | lyrics, no song | Write the style prompt, then the user generates takes in a music model and picks one. | | only an idea | Write the lyrics, get approval, then as above. | ## Set up ```sh node -v; ffmpeg -version | head -1; potrace --version | head -1; python3 --version mkdir -p my-video/song my-video/align && cd my-video && npm init -y npm i -D hyperframes playwright && npx playwright install chromium # pins HyperFrames for every npx call python3 -m venv .venv && . .venv/bin/activate && pip install librosa numpy pillow mkdir -p tools && for f in build align-to-timing edge-check; do curl -fsSLO --output-dir tools "https://katagami.ai/recipe-media/clanker-im-fine/files/$f.mjs"; done ``` - Install anything missing yourself (Node 22 or newer, FFmpeg, potrace) with the system package manager. - Demucs pulls in PyTorch, so install it in the venv only when you isolate vocals locally; a hosted Demucs works too. - Activate the venv (`. .venv/bin/activate`) in every new shell before a Python step. Reason: many systems refuse `pip install` into the system Python. - Keys: look in the environment and in your own tools for an aligner (`ELEVENLABS_API_KEY` or `FAL_KEY`) and an image model before you ask. Never print a key. - Katagami: if its tools are missing, run `claude mcp add --transport http katagami https://katagami.ai/mcp`. The tools load when the session restarts. Then call `whoami`. On the sample tier, tell the user that a one-time Google sign-in opens the full library (in Claude Code: `/mcp`, then katagami), and carry on meanwhile. With no MCP at all, use the public search, `https://katagami.ai/api/search?q=&lane=`, and each language's `DESIGN.md`. - This recipe owns the workflow. HyperFrames' own docs and skills, if installed, are for composition details only. You do not need `hyperframes init`, and never run `init --audio`. Reason: it times the song with Whisper, which rule 2 forbids. Project layout (run everything from the project root): ``` song/ song.wav, vocals.wav, lyrics.json, timing.json, timing-fixes.json, beats.json, labels.txt, NOTES.md align/ aligner inputs and results scenes.json per scene: id, lines, idea, mode, art style, design language looks.json tokens and inks of every look refs/ generated reference images (never shipped) art/ traced vector art and art/art.js assets/ fonts.css and fonts/ compositions/wide/ and compositions/vert/ one sNN.html per scene out/ builds; renders/ films; site/ the optional live page ``` ## Method 1. **The song** (see "Lyrics and the song"). `song/song.wav` and `song/lyrics.json`, with `text` for the screen and `sung` for the singer and the aligner. Done when the user approved the take. 2. **Timing** (see "Timing the lyrics to the word"). Split the vocal stem, force-align the lyrics on it, then run `node tools/align-to-timing.mjs` and review what it flags. Write `song/beats.json`. Done when five line starts after pauses sit on the voice. 3. **Scenes and looks** (see "Scenes and looks"). - Take the genre, voices, modes and looks from this song's idea and the Katagami search; the examples in the references are placeholders, not defaults. - One line or couplet per scene, with a one-sentence idea. - Two or three modes for the whole film. Reason: a new look every line with no system reads as noise. - A new video gets its own cast, characters and looks. Never reuse characters, art or scenes from an earlier video unless the user asks for a sequel. Reason: a reused character makes a new video read as a rerun of the last one. - Looks from Katagami: `compose_kit` for the base look, `ask_library` per scene, `search_library` for a named style, `get_library_entry` for an art style's prompt, `get_design_tokens` for a language's tokens. - Done when the user approved the plan and its looks. 4. **Build** (see "Scenes and looks"). Reference images from the art style's own template, traced into vector plates. One finished reference scene, then the rest. Done when each scene's `--only` build passes `npx hyperframes check`. 5. **Vertical, looks, render** (see "Vertical, every look, and the render"). Compose the 9:16 film natively. Run `node tools/edge-check.mjs out/vert`. Test every look. Render in chunks at scene cuts, then lay the song under the picture. Done when both films pass the "QA checklists" section and play through with sound. 6. **Optional live page** (see "The live restyle page"). The MP4 plays by default. A tap lays live windows over it in the next look. Taps are coalesced. Run the the "QA checklists" section list for each stage before the next one. Keep the lyrics, the music model's lyrics box and the scene plan in one source file that generates `song/lyrics.json`, the lyrics box text and `scenes.json`. Give each line an estimated length and run `align-to-timing.mjs --provisional`, so scenes are built while the human makes the song (see "Timing the lyrics to the word"). Reason: hand-edited copies drift, and scenes read their times from `song/timing.json`, so the real alignment then needs only a rebuild. ## Must hold 1. Align lyrics against the isolated vocal stem, never the mix. Reason: drums and pads smear word onsets. 2. Never time lines with a phrase transcriber. Reason: after a pause it starts the line where the previous word ended, 0.2 to 1.5 s early. 3. Never hardcode a lyric time in a scene. Reason: a re-record or re-align then needs only a rebuild. 4. Never show a word before it is sung; a scene cut may come one frame early. Reason: captions that lead the voice look out of sync. 5. Generated images are references only: trace them, and never ship a raster. Reason: vector plates can be re-inked into any look. 6. Never use text drawn by an image model; draw text in code. Reason: model text is garbled and cannot be restyled. 7. Route every colour, font, radius and shadow through `--k-*` tokens. Reason: one token swap restyles the whole film. 8. Never tween a colour with GSAP; switch a class or crossfade two layers. Reason: a tweened colour ignores later token swaps. 9. Text on paper uses `--k-ink`, on panels `--k-on-surface`, on `--k-accent` fills `--k-on-accent`. Reason: in another look the paper can be black. 10. Switch shots with `opacity` or `display`, never `visibility: visible`. Reason: it overrides the runtime hiding the scene, so the shot paints over every other scene. 11. Keep blend-mode textures inside each scene. Reason: in a separate overlay they composite as a grey veil over the film. 12. Drive JavaScript side effects from a getter/setter tween, not `tl.call`. Reason: the renderer seeks with events suppressed. 13. Keep every scene deterministic: a seeded random, no clocks, finite repeats, no CSS animations. Reason: frames render out of order, in parallel chunks. 14. Self-host every font and load every face before the first frame. Reason: a missing face renders a fallback without an error. 15. Compose the vertical for vertical, and fill it by placing things, not cropping them. Reason: a card cut by the bottom edge reads as a mistake on a phone. - The jokes are current and specific; slang a reviewer cannot read gets a native check. - No em dashes in captions or post copy. ## You can change - The song: its languages and every joke. - The film's look: any Katagami art style with a strong ink identity. - The tap looks: any Katagami design languages, since the scenes read only tokens. - The highlight colour: one token. - The scenes: how many, and their order. ## Make your own Copy this recipe for your agent and give it your song, or a brief for one, and the Katagami looks you want. It times the lyrics to your vocals, builds the scenes, and gives you the film and a tap-to-restyle page. ## Lyrics and the song Read this at stage 1. It covers both cases: the user has a song, or the user only has an idea. Output: `song/song.wav` and `song/lyrics.json`. ### If the user has the song 1. Get the final mix at the best quality they have. Convert it once to 48 kHz WAV: ```sh mkdir -p song ffmpeg -v error -y -i input.flac -ar 48000 -c:a pcm_s16le song/song.wav ``` An MP3 source stays an MP3 in quality. Convert it once and keep the original next to it. 2. Get the lyrics, then listen once with the text in front of you. Correct every word to what is actually sung: repeats, ad-libs, a changed word. 3. Write `song/lyrics.json` (format below). One entry per caption line. ### If the user only has an idea: write the lyrics first Infer the mood, the genre and the audience from the idea. Ask only for what it does not tell you, and what is off-limits if it could matter. #### Craft - **The hook.** Three to seven words that say the idea. It is the title. Put it on a downbeat and repeat it in every chorus. Test it: could someone reply to a post with just the hook? - **Structure.** Intro, verse, pre-chorus, chorus, verse, pre-chorus, chorus, bridge, final chorus, outro. Aim for 2.5 to 4 minutes. The bridge turns: a new angle, a quieter voice, or another language. The final chorus can change its last line. - **Verse density.** One concrete picture or joke per line. Use many different ideas, not one joke stretched over a verse. Prefer nouns you could draw. - **Every line is a scene.** The video cuts on lines, so write a scene note (the line's `note`) while you write the line: who, what, where. If a line has no picture, rewrite the line. - **A chorus people can sing.** Short lines of four to eight syllables. Open vowels (ah, oh, ay) on long notes. The same rhythm every time. Few consonant clusters on held notes. - **Details for rewatching.** Some jokes can live only in the pictures (fine print, labels, a sign in the background). Put them in the scene notes, not in the lyric. - **Multilingual lines, if wanted.** Show the native script on screen with an English translation under it. Slang must be current usage. Have a native speaker check every language you cannot judge yourself, or research recent native posts (not dictionaries). Drop slang that has aged or that reads as a political reference. Show the user the full lyrics with section labels and scene notes. Get approval before you generate audio. Audio is the slow and costly step. ### Two spellings per line The screen and the singer need different spellings. - `text` is the caption: what a reader expects. "5%", "DJ", "@name", brand names spelled correctly, native script. - `sung` is what the music model should sing and what the aligner will listen for. Add it only where it differs from `text`. How to write `sung`: - Spell out numbers and acronyms: "5%" becomes "five percent", "DJ" becomes "dee-jay", "@name" becomes "at name". - Mark stress with capitals where the model stresses a word wrongly: "reCORD" for the verb. - Use kana for a Japanese particle the model reads by its spelling: the topic particle は is sung "wa", so write わ. - If the model reads a line as a neighbouring language, respell it in a script that forces the right sounds, or in Latin letters. - Parentheses mean backing vocals or ad-libs in many music models. Use them on purpose. The aligner treats a parenthesised span as one non-speech token, so the timing step strips them. ### lyrics.json An array of lines in song order: ```json [ { "id": "c1_1", "section": "chorus1", "text": "Up at 3 a.m. on 5% battery", "sung": "Up at three A M on five percent battery", "lang": "en", "map": [1, 1, 1, 2, 1, 2, 1], "est": 2.6, "note": "a phone at 5% battery, the only light in a dark room" }, { "id": "v2_3", "section": "verse2", "text": "今日も元気です", "lang": "ja", "en": "Doing great again today", "est": 2.4 } ] ``` - `id`: section plus index. Scenes and timing refer to it. - `text`: caption spelling. Scenes show exactly this string. - `sung`: optional, the spelling sent to the music model and the aligner. - `lang`: a BCP 47 tag (`en`, `ja`, `zh-Hans`, `ko`, `hi`). Scenes set it on the element so the browser picks the right glyphs. - `en`: the English translation, for non-English lines. - `map`: only when `text` and `sung` have different word counts. For each caption word, how many sung words it covers: above, "3" is "three" (1), "a.m." is "A M" (2) and "5%" is "five percent" (2). The timing script uses it to give each caption word its sung time. - `est`: optional, the line's estimated sung length in seconds. Only the provisional timing uses it ((see "Timing the lyrics to the word")). - `note`: optional, the scene note: who, what, where. One caption line is one breath or phrase, usually two to four seconds. Split longer lines at the breath. ### One source file Keep every line once, in one small script or data file in the project, and generate the rest from it: - `song/lyrics.json`; - the music model's lyrics box: section tags, voice or language tags, and `sung` spellings; - `scenes.json`: which lines each scene holds, with its mode and idea; - a readable lyrics sheet with section labels and scene notes, for the user's approval. Edit the source, never the outputs. On every run it checks that line ids are unique and that every line sits in exactly one scene. Then run the provisional timing ((see "Timing the lyrics to the word")), which also flags a missing `map`. Reason: hand-edited copies drift, and a line changed in one copy and not the others shows the wrong words or times the wrong line. ### Generating the audio Any music model works if it takes lyrics and a style prompt. Suno is one example. Keep the steps tool-neutral: 1. **Style prompt.** Genre, tempo in BPM, mood, instrumentation, and which voice sings or speaks each section, all chosen for this song. Pairings range widely, for example "breathy spoken verses, stacked-harmony chorus", "one crooner throughout, a choir on the bridge" or "call and response between two singers"; none of them is a default. Do not name a living artist. Reason: services often block artist names, and copying a real singer's voice raises rights problems. 2. **Lyrics box.** Section tags (`[Verse 1]`, `[Pre-Chorus]`, `[Chorus]`, `[Bridge]`) and the `sung` spellings, generated from the source file. 3. **Generate several takes.** Pick for diction (every word clear on a phone speaker), a hook that lands, and a steady tempo. 4. **Fix single lines** with the tool's replace-section or edit feature. Note the time range and the exact new text. 5. **Export lossless WAV** to `song/song.wav` (48 kHz). 6. **Keep a note** (for example `song/NOTES.md`): the take, the style prompt and the exact lyrics text sent. A later fix then starts from the same inputs. 7. **Check the terms.** The service's terms must allow the use you plan (public posting, commercial use). After any change to the audio, run the timing step again ((see "Timing the lyrics to the word")). ### Rules - Write the text the aligner gets from what is sung, not what was planned. Reason: the aligner fits the words you give it, and a skipped word pulls its neighbours out of place. - Keep one caption line per breath. Reason: the video cuts per line, and a seven-second line becomes a long static shot. - Keep line ids stable once the first scene exists. Reason: scenes and timing refer to ids, and renumbering breaks every scene. - Get the lyrics approved before generating audio. Reason: a lyric change after the take means a new take and a new timing pass. - Review languages you cannot read with a native speaker or current native sources. Reason: dated or wrong slang is what native viewers notice first. ### Done when - [ ] `song/song.wav` is the approved take, 48 kHz WAV, full length. - [ ] `song/lyrics.json` has every sung line in order, with `text`, `sung` where it differs, `lang`, and `en` for non-English lines. - [ ] You listened once with the text, and every word matches what is sung. - [ ] The user approved the lyrics and the take. ## Timing the lyrics to the word Read this at stage 2, once `song/song.wav` and `song/lyrics.json` exist, and again after any change to the audio. Read the first section earlier if you build scenes before the song exists. Output: `song/vocals.wav`, `song/timing.json`, `song/labels.txt`, `song/beats.json`. Working files go in `align/`. ### Before the song exists: provisional timing Scenes can be planned and built while the human is still making the song. Give every line in `song/lyrics.json` an `est`, its estimated sung length in seconds (count the syllables against the planned tempo), then: ```sh node tools/align-to-timing.mjs --provisional ``` - It lays the lines back to back from their `est` (0.25 s apart, 1.5 s more between sections), spreads each line's words evenly, and writes `song/timing.json` with `"provisional": true`. It also prints a CHECK row for every line whose caption and sung word counts differ with no `map`, so a missing `map` shows up before any audio exists. - Build and check scenes on it as usual. Builds run without audio until `song/song.wav` exists, and they print "PROVISIONAL timing". - Design every scene for a word landing anywhere in its line: no layout that only works if a word lands where it is now. Reason: the real times differ. - A scene with no sung lines has a guessed `start` in `scenes.json`. Set it again from the real song. - Once the song exists, run the rest of this file. The real `timing.json` replaces the provisional one; rebuild and re-check every scene. Never render on provisional timing. ### Why forced alignment on the vocal stem A phrase transcriber (Whisper-type speech recognition) starts a line where the previous word ended whenever a pause comes before it. That is 0.2 to 1.5 s early, so captions lead the voice. It also drops repeated words ("fine, fine, fine") and short function words on short clips. Forced alignment works the other way round: you give it the words, and it finds when each one is sung. It needs the isolated vocal, because drums and pads smear the word onsets. Use a transcriber only to check what was sung (a changed word shows up as a mismatch), never to time lines. ### 1. Isolate the vocal ```sh mkdir -p song align . .venv/bin/activate && pip install demucs # the venv from the setup in SKILL.md; demucs pulls in PyTorch demucs --two-stems vocals -n htdemucs_ft song/song.wav cp separated/htdemucs_ft/song/vocals.wav song/vocals.wav # use the path demucs prints ``` Hosted alternative: `fal-ai/demucs` with `{ "audio_url": ..., "model": "htdemucs_ft", "stems": ["vocals"], "output_format": "wav" }`. Its default output is MP3, so ask for WAV. Check the stem before aligning. It must have the same duration as the mix (compare seconds: Demucs may write another sample rate) and no offset. This check needs NumPy and FFmpeg. Pass a stretch where the voice sings (start and length in seconds): ```python # offset.py: python3 offset.py [start 30] [length 30] must print "offset 0.0 ms" import subprocess, sys, numpy as np start, length = (float(v) for v in (sys.argv[1:] + ['30', '30'][len(sys.argv[1:]):])[:2]) def pcm(f): return np.frombuffer(subprocess.run(['ffmpeg', '-v', 'error', '-ss', str(start), '-t', str(length), '-i', f, '-ac', '1', '-ar', '16000', '-f', 's16le', '-'], capture_output=True, check=True).stdout, np.int16).astype(float) a, b = pcm('song/song.wav'), pcm('song/vocals.wav') if min(len(a), len(b)) < 16000 * length * 0.9: sys.exit(f'the audio ends before {start + length:.0f} s: pass an earlier start or a shorter length') n = len(a) + len(b) x = np.fft.irfft(np.fft.rfft(a, n) * np.conj(np.fft.rfft(b, n)), n) lag = int(np.argmax(x)); lag = lag - n if lag > n // 2 else lag print(f'stem is {-lag / 16:.1f} ms late' if lag else 'offset 0.0 ms') ``` ### 2. Prepare the text and the audio The aligner gets one lyric line per text line, in song order: `sung` where present and `text` otherwise, with the parentheses removed and the words kept. Reason: the aligner treats a parenthesised span as one non-speech token. Write the text for the whole song into `align/text-song.txt`: ```sh node -e 'for (const l of require("./song/lyrics.json")) console.log((l.sung ?? l.text).replace(/[()]/g, " "))' > align/text-song.txt ``` Send 16 kHz mono. Reason: uploads are much faster, and the aligner needs no more. ```sh ffmpeg -v error -y -i song/vocals.wav -ac 1 -ar 16000 align/vocals-16k.wav ``` A whole song in one request is fine when the vocal is clean and the text matches it. Align per section when long instrumental breaks, ad-libs, or backing vocals missing from the text pull words to the wrong place. A bad stretch then stays inside its section: 1. Cut the stem at clear pauses. Where a cut must fall near singing, extend the clip over the neighbouring line and add that line to the text as a context line (its id starts with `~`). The script aligns it and drops it. 2. Write each section's text from its ids, context lines included: ```sh node -e 'const L = Object.fromEntries(require("./song/lyrics.json").map(l => [l.id, l])); for (const id of process.argv.slice(1)) { const l = L[id.replace(/^~/, "")]; console.log((l.sung ?? l.text).replace(/[()]/g, " ")); }' '~i2' v1_1 v1_2 '~pc1_1' > align/text-verse1.txt ``` 3. Either send one request per section, or put the clips back to back with 2 s of silence between them and send one request for all. Reason: one request gives one result file to keep. ```sh ffmpeg -v error -y -ss 12.0 -to 41.5 -i align/vocals-16k.wav align/sec01.wav # one clip per section ffmpeg -v error -y -f lavfi -i anullsrc=r=16000:cl=mono -t 2 align/gap.wav printf "file 'sec01.wav'\nfile 'gap.wav'\nfile 'sec02.wav'\n" > align/list.txt ffmpeg -v error -y -f concat -safe 0 -i align/list.txt -c copy align/joined.wav cat align/text-verse1.txt align/text-chorus1.txt > align/text-joined.txt ``` 4. Record each section in `align/sections.json`. `shift` turns aligned seconds into song seconds: the clip's start in the song minus its start in the aligned file (0 for a clip sent alone; take the joined file's starts from the clips' `ffprobe` durations plus the 2 s gaps). ```json [ { "file": "align/joined.json", "shift": 12.0, "ids": ["~i2", "v1_1", "v1_2", "~pc1_1"] }, { "file": "align/joined.json", "shift": 28.4, "ids": ["~v1_12", "c1_1", "c1_2", "c1_3", "c1_4"] } ] ``` Here the first clip starts at 12.0 s in the song and at 0 in `align/joined.wav`, so its shift is 12.0. The second clip starts at 59.9 s in the song and at 31.5 s in the joined file (after the 29.5 s first clip and the 2 s gap), so its shift is 59.9 minus 31.5, which is 28.4. Sections that share one file take its lines in the order listed. ### 3. Align ElevenLabs Forced Alignment takes the audio file and the text: ```sh curl -s -X POST https://api.elevenlabs.io/v1/forced-alignment \ -H "xi-api-key: $ELEVENLABS_API_KEY" \ -F file=@align/vocals-16k.wav -F "text=.json`; a `{ result }` wrapper is fine). Upload the WAV with the fal client first. Any other aligner works if it returns words with a start and an end for the text you give it. ### 4. Build timing.json ```sh node tools/align-to-timing.mjs # whole song: reads align/song.json node tools/align-to-timing.mjs --sections align/sections.json --fixes song/timing-fixes.json ``` It reads `song/lyrics.json` and the results, and writes `song/timing.json` and `song/labels.txt`. The song duration comes from `ffprobe` of `song/song.wav`, or from `--duration`. - **Lines.** It splits each result at its line-break entries. When a result has none, it splits by each line's sung word count and says so. - **Words.** A caption word takes its time from the sung words it stands for: its start is the first one's start and its end the last one's end. Equal word counts map one to one. When the counts differ, the line's `map` in `lyrics.json` says how many sung words each caption word covers ("5%" covers "five percent": 2). - **Japanese and Chinese.** A caption without spaces becomes one caption word per character, with punctuation kept on its character. Unless the counts match or a `map` covers them, the characters share the sung time by letter count, and the line is flagged. - **Overlaps.** A line ends by the time the next one starts: its end, and its words' ends, are clamped to the next line's start. It prints one row per line: id, start, length, mean and max loss, then any flags: | Flag | Meaning | What to do | | --- | --- | --- | | `loss ` | the aligner was unsure (loss 3 or more) | check the word by ear, fix it if it is off | | `squeezed ` | a word shorter than 0.02 s: the aligner could not find it and squeezed it against a neighbour | re-align the line alone or fix the word | | `mapped by length` | caption and sung words differ and no `map` covered it | add a `map`, or check the reveal by eye | | `(fixed)` | a fix from the fixes file was applied | nothing | ### 5. Verify - Read the table. Look hard at every flagged line. - For a doubtful word: re-align that line alone on a tight clip (1 s either side), compare it with onsets in the stem (spectral flux peaks), or listen. - Keep corrections in `song/timing-fixes.json`, in song seconds, by sung word index (from 0), with a reason each. The script reads that file on every run when it exists (`--fixes` names another). Reason: a re-run keeps the corrections. ```json { "v1_4": { "why": "the aligner squeezed 'Christie's' into 'canvas'", "words": { "5": [41.12, 41.48] } } } ``` - Expect the aligner to land 0.05 to 0.1 s after consonant onsets. That is fine: scenes cut one frame (1/30 s) before the line. - Listen: import `song/labels.txt` into Audacity over the vocal stem (File, Import, Labels) and scrub five line starts that follow a pause. Each label must sit on the first sung sound, never before it. ### 6. Beats and downbeats Cuts and look switches land on beats. Downbeats (the one of each bar) carry the bigger moves. `npx hyperframes beats` writes beats only, with no downbeats, so use librosa: ```python # beats.py: song/song.wav -> song/beats.json (run it in the venv, which has librosa and numpy) import json, numpy as np, librosa y, sr = librosa.load('song/song.wav', sr=22050, mono=True) tempo, beats = librosa.beat.beat_track(y=y, sr=sr, units='time') S = np.abs(librosa.stft(y, n_fft=2048, hop_length=256)) f = librosa.fft_frequencies(sr=sr, n_fft=2048) t = librosa.frames_to_time(np.arange(S.shape[1]), sr=sr, hop_length=256) low = S[(f > 30) & (f < 120)].sum(0) # kick drum band kick = np.array([low[max(0, i - 2):i + 4].max() for i in np.searchsorted(t, beats)]) phase = max(range(4), key=lambda p: kick[p::4].mean()) # the bar position with the strongest kick down = beats[phase::4] json.dump({'beats': [round(float(b), 3) for b in beats], 'downbeats': [round(float(d), 3) for d in down]}, open('song/beats.json', 'w')) print(f'{len(beats)} beats at {float(np.atleast_1d(tempo)[0]):.1f} BPM, first downbeat {down[0]:.2f} s (phase {phase})') ``` The downbeat phase is a guess. Check that a downbeat lands on the first word of a chorus. Songs with a short bar drift off a fixed four-beat grid, so pick the phase per section there. Space the moves out. A cut or a short colour flip (a filter on the whole scene for one beat) can land on any beat, at least 0.36 s apart. Reason: faster changes read as flicker. Whole look switches go on downbeats, at least 1.5 s apart. Reason: a viewer needs time to take in a look. ### Rules - Align against the vocal stem, not the mix. Reason: drums and pads smear the onsets. - Never trust a transcriber's line starts after a pause. Reason: it starts the line where the previous word ended. - Never hardcode a lyric time in a scene; read it from the timing data. Reason: a re-record or re-align then needs only a rebuild. - Never show a word before it is sung. The cut to a new scene may come one frame before the line, no more. Reason: a caption that leads the voice reads as out of sync, even by 0.2 s. - Make each line readable within about 0.4 s of its start. Reason: most lines last two to three seconds. - Give every ffmpeg command that writes a file `-y`. Reason: without it, a re-run in a non-interactive shell prints "Not overwriting" and exits 0, so the old file silently stays. - Re-run the whole timing step after any change to the audio. Reason: timing belongs to one exact take. ### Done when - [ ] `song/vocals.wav` has the mix's duration and a 0.0 ms offset. - [ ] `song/timing.json` came from the aligner: it has no `"provisional"` key. - [ ] Every line in `song/timing.json` has `t`, `end` and `words`. No line overlaps the next. - [ ] Every flagged line is reviewed, and each fix has a reason in `song/timing-fixes.json`. - [ ] Five line starts after pauses, checked by ear, sit on the voice and not before it. - [ ] `song/beats.json` has beats and downbeats, and a chorus starts on a downbeat. ## Scenes and looks Read this at stages 3 and 4: planning scenes, choosing Katagami looks, making reference art, and writing the HyperFrames scene files. Output: `scenes.json`, `looks.json`, `art/*.js`, `assets/fonts.css`, `compositions/wide/*.html` and your copy of `build.mjs`. ### 1. Plan the scenes Cut the song into scenes of one lyric line or a couplet, usually 2 to 6 s. The intro and the outro can be one scene each. A scene with no sung lines gives its `start` in seconds, and `art` lists art names a scene builds at runtime (section 6). Generate `scenes.json` from the one source file ((see "Lyrics and the song")), not by hand. ```json [ { "id": "s01", "lines": ["i1"], "mode": "mode-a", "idea": "A kettle starts to whistle in an empty kitchen before sunrise", "art_style": "", "language": "" }, { "id": "s02", "lines": ["v1_1", "v1_2"], "mode": "mode-a", "idea": "A cat on the windowsill watches the street wake up", "art_style": "", "art": ["s02-blink-1", "s02-blink-2"] }, { "id": "s09", "lines": [], "start": 61.5, "mode": "mode-b", "idea": "Instrumental break: the street lights go out one by one", "art_style": "" } ] ``` - One idea per scene: a joke, an image, a turn. Write it in one sentence before you design anything. - The lyric is always on screen, and it works best inside the picture: a sign, a letter, a label, a message on a screen, a caption box. - Put small details in the picture that reward a second watch (fine print, labels, a background sign). - Choose two or three modes for the whole film from this song's idea, for example an illustrated mode for story lines, a diagram mode for ideas and a type-only mode for punchlines, or whatever the idea needs. Every line changes treatment (layout, type, texture, motif), but only inside those modes. Record each mode's look in `looks.json` (section 3). - Story sections (a chorus, the bridge) can be full-frame illustrated sequences: three to six shots per scene, cut on words and beats, characters acting the line. ### 2. Choose looks with the Katagami MCP Katagami holds design languages (tokens, rules, type and UI), palette systems and art styles (prompt recipes for images). Calls, in the order you usually need them: | Call | Use it for | | --- | --- | | `compose_kit({ query })` | The film's base look: a language, a palette and an art style judged to belong together, with a `brief_url`. | | `ask_library({ query, kind })` | A look for one scene or section, from a one-sentence brief ("the chorus: loud, glossy, anime energy"). | | `ask_library({ query, reading, changes, refine })` | Adjusting an answer ("quieter, black and white"). Pass back the `reading` and `changes` it returned. | | `search_library({ kind, query, tag, medium })` | A named style, tag or medium ("woodcut", "collage", `medium: "print"`). `describe_library` lists what exists. | | `get_library_entry({ id_or_slug })` | Everything in one entry. For an art style: `prompt_template`, `slot_recipes`, `negative_prompt`, `reference_image_urls`, do's and don'ts. | | `get_design_tokens({ id_or_slug, format: "css" })` | A design language's colours, radii, shadows and fonts as CSS variables, plus `fonts_url`. | | `check_page_against_language({ id_or_slug, page })` | After building a scene's UI: pass the scene HTML with its CSS and fix what breaks the language, worst first. | Show the user what you pick: the thumbnail and the katagami.ai `url` of each entry. Record the slugs in `scenes.json`. - `ask_library` returns `strange` next to `results`: styles unlike the rest of the library that still fit. Read both. Reason: for a comic or odd brief the best pick can be in `strange`. - `compose_kit` scores each kit's `belongs_together` from 0 to 1. When it is low (a paper language paired with a dark palette, say), take every token from the language and borrow only the palette's signature colours, as the spare accents. Reason: the token table below assumes one source, and two grounds that disagree break the contrast pairs. ### 3. Tokens: the `--k-*` contract Every scene declares its look as CSS custom properties on its root, and every rule reads them. This is what lets the film be restyled later, in a beat-looks cut or on the live page. Nineteen tokens. Take each from the language's `get_design_tokens` output (CSS format); the first name that exists wins. Entries differ: one calls the ground `--color-background`, one names a radius `--radius-base`, another has no `--color-accent-2`, and a dark-ground language can have light cards. | Film token | Meaning | From `get_design_tokens`, first match wins | | --- | --- | --- | | `--k-paper` | the ground | `--color-background`, `--color-bg`, else `#FFFFFF` | | `--k-surface` | cards and panels | `--color-surface-solid`, `--color-surface`, else `--k-paper` | | `--k-ink` | text and lines on paper | `--color-text`, `--color-ink`, else black or white, whichever passes on paper | | `--k-on-surface` | text on a card or panel | `--color-on-surface`, `--color-ink`, `--color-text`, `--color-background`, `--color-bg` | | `--k-muted` | secondary text on paper | `--color-muted` if it passes on paper, else `--k-ink` | | `--k-line` | rules and keylines | `--color-border`, else `--k-muted` | | `--k-accent` | the highlighter (key word, marker) | `--color-accent`, `--color-primary`, else the first colour of the palette `compose_kit` paired with it | | `--k-accent-2`, `--k-accent-3` | sparing second and third accents: marks, strokes and fills, never text | `--color-accent-2` and `-3` (for accent-2 also `--color-secondary`), else colours from the paired palette, else `--k-accent` | | `--k-text-2`, `--k-text-3` | text in the accent-2 and accent-3 hues | the accent itself if it passes on paper, else the language's own darker colour of that hue (often `--color-warning` or `--color-error`), else `--k-ink` | | `--k-on-accent` | text on an accent block | `--color-on-accent`, else `--k-ink` or `--k-paper`, whichever passes on the accent | | `--k-font-display`, `--k-font-body`, `--k-font-mono` | the three faces | `--font-heading`, `--font-body`, `--font-mono`, each with a generic fallback (`'Barlow Condensed', sans-serif`) | | `--k-lh-display`, `--k-lh-body` | line-height of display and body text | set per look from its faces, not from `--type-line-height` (that is for reading pages): about 1.0 and 1.12 for condensed faces, about 1.25 for faces with a tall glyph box (many Japanese and Chinese families) | | `--k-radius` | corners | `--radius-md`, `--radius-base`, else `0px` | | `--k-shadow` | the one shadow | `--shadow-md`, else `none` | `--k-accent-2` and `--k-accent-3` are highlighters. Never colour text with them, and never put `--k-ink` text on them. Reason: as text they fail on light paper (an orange on cream is under 2:1), and ink on an accent fill fails in a dark look. Text in their hues uses `--k-text-2` / `--k-text-3`; text on their fills gets its colour picked per look ((see "Vertical, every look, and the render"), section 4). Every text pair must reach 4.5:1: ink on paper, on-surface on surface, muted on paper, text-2 and text-3 on paper, on-accent on accent. When a candidate fails, take the next one, or black or white, whichever passes. Compute it; do not guess: ```js const lum = h => [1, 3, 5].map(i => parseInt(h.slice(i, i + 2), 16) / 255).map(v => (v <= 0.03928 ? v / 12.92 : ((v + 0.055) / 1.055) ** 2.4)).reduce((a, v, i) => a + v * [0.2126, 0.7152, 0.0722][i], 0); const contrast = (a, b) => { const [x, y] = [lum(a), lum(b)].sort((p, q) => q - p); return (x + 0.05) / (y + 0.05); }; contrast('#f8f1e4', '#f5efe2'); // light text on a light card: about 1.0, so on-surface takes --color-ink instead ``` `looks.json` holds each mode's look, the original look and every restyle look: ```json { "tokens": ["paper", "surface", "ink", "on-surface", "muted", "line", "accent", "accent-2", "accent-3", "text-2", "text-3", "on-accent", "font-display", "font-body", "font-mono", "lh-display", "lh-body", "radius", "shadow"], "modes": { "mode-a": { "language": "", "art_style": "", "fonts": "", "k": { "paper": "", "ink": "", "accent": "", "lh-display": "" }, "inks": ["", "", ""] }, "mode-b": { "language": "", "art_style": "", "fonts": "", "k": { "paper": "", "ink": "", "accent": "" }, "inks": ["", "", ""] } }, "looks": [ { "id": "original", "k": null, "inks": null }, { "id": "look-b", "language": "", "fonts": "", "k": { "paper": "", "ink": "", "accent": "", "on-accent": "" }, "inks": ["", "", "", ""] }, { "id": "look-c", "language": "", "fonts": "", "k": { "paper": "", "surface": "", "ink": "", "on-surface": "", "muted": "", "line": "", "accent": "", "accent-2": "", "accent-3": "", "text-2": "", "text-3": "", "on-accent": "", "font-display": "'', sans-serif", "font-body": "'', sans-serif", "font-mono": "'', monospace", "lh-display": "", "lh-body": "", "radius": "", "shadow": "" }, "inks": ["", "", ""] } ] } ``` - The ids and values are placeholders. Name each mode and look after what you chose, and fill the values from its Katagami entries. - `modes` has one entry per mode named in `scenes.json`: its language, art style, fonts, all nineteen tokens in `k` (shortened above) and the ink ramp for its art. A scene declares its mode's tokens on its root (section 7). Reason: with several modes there is no single original look, so scene builders need each mode's tokens written down. - `original` has `"k": null`: each scene keeps its own mode's tokens. - Every restyle look lists all nineteen tokens (`look-b` is shortened above). - `inks` is the colour ramp the traced art is re-inked with (section 5). Its first colour paints the art's lightest tone and its last colour the darkest. `look-c` has a dark paper, so its ramp starts dark and its art prints inverted, like its type. #### Fonts Ship every face as a file. A family with no `@font-face` renders in a fallback face, and `lint` does not report it. Fetch the look's `fonts_url` with a browser User-Agent (without one, Google Fonts serves `.ttf` files), keep the `.woff2` files in `assets/fonts/`, and point the rules at them: ```sh UA='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36' fetch_fonts() { # fetch_fonts [subsets]: .woff2 files into assets/fonts/, @font-face rules into assets/fonts/.css mkdir -p assets/fonts css=$(curl -sf -A "$UA" "$2") || { echo "cannot fetch $2"; return 1; } if [ -n "$3" ]; then # keep only the named subsets, for example 'latin|latin-ext|cyrillic' css=$(printf '%s\n' "$css" | awk -v keep="^/[*] ($3) [*]/\$" '/^\/\*/ { ok = $0 ~ keep } /^@font-face/ { cur = ok; ok = 0 } cur || ($0 ~ keep) { print } /^}/ { cur = 0 }') fi printf '%s\n' "$css" | grep -oE 'https://fonts\.gstatic\.com/[^)]+\.woff2' | sort -u | while read -r u; do curl -sf "$u" -o "assets/fonts/$(basename "$u")" done printf '%s\n' "$css" | sed -E 's#url\(https://fonts\.gstatic\.com/[^)]*/([^/)]+\.woff2)\)#url(assets/fonts/\1)#g' > "assets/fonts/$1.css" } fetch_fonts mode-a "" # one call per mode and per restyle look fetch_fonts look-c "" fetch_fonts jp "" 'latin|latin-ext' # CJK faces: only the subsets the film shows cat assets/fonts/*.css > assets/fonts.css ``` - The build inlines `assets/fonts.css` into `index.html`, so the paths start at `assets/fonts/`. - Japanese, Chinese and Korean families ship hundreds of `unicode-range` files (one such family came to about 1,000 files and 14 MB). Pass the subsets the film's text uses, for example `'latin|latin-ext'`, plus `cyrillic` for Russian lines. Keep the full set only when the film shows text in that script. - The edge check (`tools/edge-check.mjs`) also lists families that are used but never declared. See (see "QA checklists"). - A look whose faces are wider than your layout gets a copy of the face with `size-adjust` under a new family name. Test the look with that copy. ### 4. Reference images The art style's `prompt_template` is the recipe. Fill it, do not rewrite it. - Fill `{subject}` with the shot: who, doing what, where. Fill `{palette}` from the scene's palette. Where there is `{composition}`, use the matching `slot_recipes` entry or your shot framing. - Where the template ends with "Full frame.", add the framing after it: "Wide establishing shot, 16:9 film frame." or "Tall 9:16 frame, subject in the upper half." - Use the entry's `negative_prompt` if your image model takes one. Pass `reference_image_urls` (the style's own gallery) if it takes reference images. - Keep characters identical in every shot: make one character sheet per character first, then pass it as a reference image in every call that shows them. Describe the fixed traits (hair colour, clothes, silhouette) in every prompt. - Screens, signs and paper in the picture: ask for "a plain blank glowing screen" or "a blank sign". Draw the text in code on top. The model often draws labels, numbers and dial faces anyway, even when the prompt forbids them: blank those regions before tracing (section 5). - Limited animation (a blink, a turn, a tear): call the image edit endpoint with the generated shot as the first reference. Prompt: "The exact same image, same composition, camera, colours and lighting; only change: ...". Trace each frame. Swap the frames on the timeline at 8 to 12 fps. - Parallax: generate the character alone "on a flat pure white background, full figure, no shadow on the ground". Cut it out: flood-fill the white from the corners, then make it transparent. Generate the background plate "with no people". Trace the two separately. - Save everything in `refs/`. Nothing in `refs/` is ever loaded by a scene. ### 5. Trace to vectors Trace each reference into a few flat tone plates with potrace. Plate k covers every pixel at least as dark as tone k, and the plates are painted light to dark, so there are never gaps between them. The result is one JS file per image that registers `window.ART[name]`. ```python # trace.py: python3 trace.py refs/shot-01.png shot-01 art [--tones 5] [--smooth 3] [--speckle 24] import argparse, json, os, re, subprocess, tempfile import numpy as np from PIL import Image, ImageFilter ap = argparse.ArgumentParser() ap.add_argument('src'); ap.add_argument('name'); ap.add_argument('outdir') ap.add_argument('--tones', type=int, default=5) # flat tones (plates) ap.add_argument('--width', type=int, default=1280) # trace size; vectors scale to any frame ap.add_argument('--smooth', type=int, default=3) # median filter: melts dots and noise (1 keeps small faces) ap.add_argument('--speckle', type=int, default=24) # drop blobs smaller than this many pixels a = ap.parse_args() im = Image.open(a.src).convert('RGBA') if im.width > a.width: im = im.resize((a.width, round(im.height * a.width / im.width)), Image.LANCZOS) W, H = im.size seen = np.array(im.getchannel('A')) > 127 # transparent pixels (cutouts) stay out of every plate rgb = im.convert('RGB') if a.smooth > 1: rgb = rgb.filter(ImageFilter.MedianFilter(a.smooth | 1)) rgb = rgb.filter(ImageFilter.GaussianBlur(0.8)) q = rgb.quantize(colors=a.tones, method=Image.Quantize.FASTOCTREE, kmeans=4) pal = np.array(q.getpalette()[:3 * a.tones]).reshape(-1, 3) light_to_dark = np.argsort(-(pal @ [0.2126, 0.7152, 0.0722])) rank = np.argsort(light_to_dark) # palette index -> 0 (lightest) .. tones-1 (darkest) tone = rank[np.array(q)] tone[~seen] = -1 tmp, plates = tempfile.mkdtemp(), [] for k, idx in enumerate(light_to_dark): mask = tone >= k # plate k covers every pixel at least as dark as tone k if not mask.any(): continue Image.fromarray(np.where(mask, 0, 255).astype(np.uint8)).convert('1').save(f'{tmp}/p.pbm') subprocess.run(['potrace', f'{tmp}/p.pbm', '-s', '-o', f'{tmp}/p.svg', '-t', str(a.speckle), '-a', '1.0', '-O', '0.4', '--flat'], check=True) svg = open(f'{tmp}/p.svg').read() d = ' '.join(re.findall(r' d="([^"]+)"', svg)).replace('\n', ' ') if d.strip(): tr = re.search(r'' % tuple(pal[idx])) svg = f'{"".join(plates)}' os.makedirs(a.outdir, exist_ok=True) js = f'window.ART=window.ART||{{}};window.ART[{json.dumps(a.name)}]={{w:{W},h:{H},svg:{json.dumps(svg)}}};\n' open(os.path.join(a.outdir, a.name + '.js'), 'w').write(js) print(a.name, f'{W}x{H}', len(plates), 'plates', round(len(js) / 1024), 'KB') ``` Settings that work: - Black-and-white manga or flat art: `--tones 5 --smooth 5 --speckle 30`. About 0.1 to 0.4 MB per full frame. - Colour key art with small faces: `--tones 12 --smooth 1 --speckle 12`. `--smooth 5` melts eyes and mouths. About 0.7 MB. - Printed screentone (halftone dots): blur the dots away first, or every dot becomes a path and the file reaches 4 MB. - Keep each file under about 1.4 MB and look at the traced frame next to its reference before you use it. Plates with text: blank every label, number or dial face the model drew, then trace. The tracer keeps transparent pixels out of every plate, so each blanked region becomes a hole where the scene draws its own label or dial in code. Reason: model text is garbled and cannot be restyled (rule 6). ```python # blank.py: python3 blank.py refs/plate.png refs/plate-blank.png --box 120,80,420,160 [--circle 900,300,70] (source pixels) import argparse from PIL import Image, ImageDraw ap = argparse.ArgumentParser(); ap.add_argument('src'); ap.add_argument('out') ap.add_argument('--box', action='append', default=[]); ap.add_argument('--circle', action='append', default=[]) a = ap.parse_args() im = Image.open(a.src).convert('RGBA'); alpha = im.getchannel('A'); d = ImageDraw.Draw(alpha) for b in a.box: d.rectangle(tuple(map(int, b.split(','))), fill=0) # x0,y0,x1,y1 for c in a.circle: x, y, r = map(float, c.split(',')); d.ellipse((x - r, y - r, x + r, y + r), fill=0) # cx,cy,r im.putalpha(alpha); im.save(a.out) print(a.out, len(a.box) + len(a.circle), 'regions blanked') ``` If you want to fill plates with SVG patterns later (screentone, hatching), bake potrace's `transform` into the path coordinates first, so `userSpaceOnUse` patterns render at their true size. A small art helper places the art and re-inks it for a look. Load it once in the built `index.html`: ```js // art/art.js: placeArt(host, name, { fit, inks }) and reinkArt(ramp) (function () { const placed = new Set(); let ramp = null; // null: each scene's own inks function ink(svg) { const r = ramp || (svg.dataset.inks ? JSON.parse(svg.dataset.inks) : null), gs = [...svg.querySelectorAll('g[data-plate]')]; gs.forEach((g, k) => { if (!g.dataset.own) g.dataset.own = g.getAttribute('fill'); g.setAttribute('fill', r ? r[Math.round(k * (r.length - 1) / Math.max(1, gs.length - 1))] : g.dataset.own); }); } window.placeArt = function (host, name, opts = {}) { const A = window.ART && window.ART[name]; if (!A) throw new Error('art not loaded: ' + name); host.innerHTML = A.svg; const svg = host.querySelector('svg'); svg.setAttribute('preserveAspectRatio', opts.fit || 'xMidYMid slice'); svg.style.cssText = 'width:100%;height:100%;display:block'; if (opts.inks) svg.dataset.inks = JSON.stringify(opts.inks); // a scene's own ramp, light to dark placed.add(svg); ink(svg); return svg; }; window.reinkArt = function (r) { ramp = r || null; placed.forEach(svg => (svg.isConnected ? ink(svg) : placed.delete(svg))); }; })(); ``` A flat ramp is the baseline. Richer inks (screentone patterns, two plates printed off register, pencil hatching) go in the same `ink` function. ### 6. The build This section is the build contract; the other files link here. `tools/build.mjs` (one of the recipe's files) implements it in about 70 lines; it is in your project's `tools/` from the set-up step. Extend it there: ```sh # tools/build.mjs came with the recipe (see "Set up") node tools/build.mjs --frame wide # out/wide/ node tools/build.mjs --frame vert # out/vert/ ``` **Inputs:** `song/lyrics.json`, `song/timing.json`, `song/song.wav`, `scenes.json`, `compositions//sNN.html`, `art/` (with `art/art.js`), `assets/` (with `assets/fonts.css` and `assets/fonts/`), and `looks.json` for `--look`. **Outputs:** `out//` (or `--out`): - `index.html`: a standalone HyperFrames composition. It holds the song as an `