How to Make an Anime with AI

I made a four-minute anime short film, The Lightning Fox, on my laptop. Every picture, animation, voice and music cue was generated locally in Promptus, with the Playground set to Local. This page is the full instruction manual: the models and settings, the character descriptions I pasted into every prompt, all 38 shot prompts, the negative prompt, the 12 music cues, and the mistakes I made and how I fixed them. Copy anything you like.

The short version: write one simple story, design reusable character sheets, make one key still per shot with the characters passed in as reference images, animate each still with a short motion prompt, then add voices, sound and music in layers.

Watch the full tutorial, with the complete film at the end:

What you need

• Promptus desktop app with ComfyUI running locally (download). Check that the Playground switch says Local before every Generate, so jobs run on your own graphics card and not online.
• A graphics card with plenty of VRAM. I used one laptop with an NVIDIA RTX 5090 Laptop GPU (24 GB).
• Four models, all run locally: FLUX.2 Klein 4B for pictures, LTX-2.5 Image to Video for animation, Chatterbox Turbo for character voices, and ACE-Step 1.5 for music.
• A video editor to cut the shots together and mix the sound.

Models and settings at a glance

• Key stills: FLUX.2 Klein base 4B, 1280×720, 20 steps, CFG 5, euler sampler. Each character in the shot is passed in as a reference image (VAE-encoded, ReferenceLatent on the positive and negative). Two seeds per shot; keep the cleaner one.
• Animation: LTX-2.5 Image to Video (distilled), 960×544, 5 seconds at 24 fps (121 frames), with the key still as the first frame.
• Voices: Chatterbox Turbo voice clone, one short reference clip per character.
• Music: ACE-Step 1.5, instrumental, one cue per scene.
• Real waits on my laptop: about 1 minute 9 seconds for a reference still, about 1 minute 49 seconds for a 5-second LTX-2.5 clip, and about 28 seconds for an ACE-Step cue.

Step 1: Write one simple story

My first two versions looked good but confused people. They had too many made-up rules and flashbacks. So I started again with one idea anyone gets in fifteen seconds: a girl finds a tiny fox made of lightning, hunters are chasing it, and she has until dawn to get it home.

• One goal, one deadline, one climax, one goodbye. If you can't say all four in a sentence, simplify.
• Draw the route before writing dialogue. The fox falls beside a rooftop tank; its family runs through a ring of lightning above the tower. When Rin says "that's her family out there", the audience has already seen it.
• Let actions define characters. Sora, Rin's best friend, draws the drones away so she can finish the climb. His action tells us who he is.

Step 2: Set up Promptus to run locally

1. Open CosyTemplates and filter to Available Locally. A CosyTemplate is a ready-made workflow for one job.
2. Open the image template and check which model it expects (FLUX.2 Klein 4B for pictures). Open LTX-2.5 Image to Video for animation and ACE-Step 1.5 for music.
3. Remember what each needs: the picture model needs a written picture prompt; the video model needs a starting image plus a description of movement.
4. Switch the Playground to Local and send one job at a time.

Step 3: Design reusable character sheets

Make one sheet per character: front, side, back and a few expressions on a plain background. Then paste the same character description into every prompt where that character appears, even when the camera is behind them.

Character descriptions (pasted into every prompt)

• Rin: a 19-year-old young woman, short black bob hair with one electric-blue streak, grey eyes, orange hooded jacket with reflective stripes, blue crossbody satchel.
• Sora: a friendly 20-year-old young man, long white hair tied in a high ponytail, violet eyes, bare face fully visible, black jacket with cyan piping, empty-handed.
• Kite: exactly one small glowing cyan-blue fox made of soft lightning, about the size of a house cat, big round dark eyes, tall pointed ears, fluffy tail, two small translucent wings, one bent wing.

Style prefix (start of every picture prompt)

2D anime film still, clean bold linework, cel shading, cinematic 16:9 composition, rain-soaked neon mountain city at night, electric blue and warm orange lighting, high detail.

Step 4: Make consistent key frames

For each shot, combine three things: the style prefix, the descriptions of the characters in that shot, and a single picture description. Pass each character's sheet in as a reference image, then press Generate. Make two seeds and keep the cleaner one.

• Write the framing into the prompt. AI still gets fingers wrong, so most of my prompts say "arms and hands out of frame" or hide hands behind a ladder or a jacket. I threw away any clip that got hands wrong.
• One action per shot. Big movements are where a small character falls apart.

Step 5: Animate each still with LTX-2.5

In image to video, the key still supplies the design. The motion prompt only describes what changes. The template comes with a sample prompt; replace it with what moves in this shot, for example: "The beam sweeps. The fox flattens its ears."

Negative prompt for LTX-2.5

The template's original negative prompt included the word "cartoon", which fights an anime style. I took it out and added the faults I kept seeing:

pc game, console game, video game, childish, ugly, deformed hands, extra fingers, extra limbs, mask, face covering, weapon, contact sheet, broken face, morphing, melting, distorted animal, two heads, duplicate fox

The clips' native audio was screened with Whisper and muted if it contained speech.

Step 6: Give each character a voice

Chatterbox Turbo takes a short voice reference and a line of text. Rin and Sora each use their own short reference clip. Keep lines short in action scenes ("Hold on tight!", "Sora, the cable!") and put them on wide shots or the back of a head, so nobody has to watch the lips.

Step 7: Build the sound in layers

• Four layers: a continuous rain and wind bed, spot effects for the action, music, and dialogue.
• Distance changes sound. The same hunter drone is a low motor across the alley and a short alarm when its red light finds Rin.
• Silence before the big moment. I left exactly one second of silence before the lightning strike.
• Duck the music about 9 dB under every line of dialogue.

Music cues (ACE-Step 1.5)

One cue per scene, all with the prefix anime film score, orchestral, instrumental, no vocals and the lyrics field set to [Instrumental]. The chase cue is 120 bpm, so every two-second cut lands on a downbeat. A small music-box melody returns whenever Kite matters.

• Discovery (70 bpm, D minor): mysterious and gentle, soft string pad, delicate piano arpeggios like raindrops, celesta, harp
• Hunters (120 bpm): tense suspense, low taiko pulse, dark synth bass drone, staccato low strings
• Wonder (80 bpm, D major): soaring strings, wordless choir pad, harp glissando, chimes, crescendo to a hit
• Friends (100 bpm, G major): warm and hopeful, light piano, pizzicato strings, soft acoustic guitar
• Chase (120 bpm, E minor): driving taiko and rock drums, electric guitar riff, sixteenth-note string ostinato, brass stabs
• Tram (72 bpm): tender, solo piano, soft strings, music box
• Ambush (140 bpm): aggressive taiko, low brass hits, tremolo strings
• Climb (90 bpm): sorrowful strings building into a heroic theme
• Standoff (100 bpm): heartbeat drum, rising tremolo crescendo (played reversed into the strike)
• Transformation (120 bpm, D major): huge orchestral hit, full choir, heroic brass, thundering taiko
• Goodbye (70 bpm): bittersweet piano and strings, soft choir
• Dawn (80 bpm): peaceful sunrise, warm strings, piano, acoustic guitar, final held chord

Mistakes I made (and the fixes)

• The fox melted in about a third of my shots. In my first cut, Kite was smeared, grey or floating. Fix: I went back to Promptus and made two stills for each of those shots with the fox's reference image next to Rin's, picked the cleaner one, and animated it with a small, slow movement. Eleven shots came back fixed in well under an hour.
• A confusing story. My first two versions had too many rules and flashbacks. Fix: one goal, one deadline, one climax, one goodbye.
• A masked, armed character. My first Sora had a mask and a sword. Fix: a new sheet from text alone, with "bare face fully visible" and "empty-handed" in every prompt, plus "mask, face covering, weapon" in the negative prompt.
• The word "cartoon" in the negative prompt. It pushed the clips away from the anime look. Fix: remove it.
• Hands and fingers. Fix: frame hands out of shot and discard any clip with bad fingers.
• Starting a job online by accident. Fix: check the Local switch before every Generate.

The full shot list: picture prompt, motion prompt and sound

Each picture prompt below was combined with the style prefix and the descriptions of the characters in the shot. Shot numbers follow the edit; shots not listed were cut or are variations.

1. Shot 01. Picture prompt: Wide establishing view. A fierce blue lightning storm crowns the needle-like Skyline Tower over a neon mountain city. Two distant red searchlights move below the clouds. Clear geography, one tall destination. Motion prompt: Camera drifts toward the spire. Lightning forks once above it; rain crosses the foreground. Sound: heavy rain, rolling thunder.
2. Shot 02. Picture prompt: Wide rooftop, cylindrical water tank struck by lightning. One tiny winged fox has fallen onto the wet roof at the tank's base, brilliant blue against an amber rooftop lamp. Motion prompt: A flash strikes the metal tank. A tiny glowing creature tumbles onto the wet roof and skids to a stop. Sound: close thunder crack, electric fizz, tiny thud.
3. Shot 03. Picture prompt: Chest-up Rin in her orange hood under the edge of a fabric awning. Her eyes catch a reflected blue glow across the rain. Forearms below frame. Motion prompt: Rin turns from shelter to the blue light; rain bounces on the awning, camera pans with her glance. Sound: rain pattering on canvas.
4. Shot 04. Picture prompt: Medium side shot on a wet rooftop at night. Rin kneels and leans down toward Kite the small lightning fox, which is curled up shivering on the wet tiles in front of her, one wing bent, looking up at her with big frightened eyes. Her arms tucked inside her jacket. Motion prompt: She lowers herself slowly to its eye level. The bent wing trembles, one blue spark flickers. Sound: soft electric crackle, rain on tile.
5. Shot 05. Picture prompt: Fox face close-up, frightened round eyes looking upward, tiny bent left wing visible behind its cheek, wet tile at nose level. Motion prompt: Kite blinks up, its ears twitch, a tiny spark runs over its fur. Sound: tiny electric whimper and fizz.
6. Shot 06. Picture prompt: Wide rooftop alley swept by red searchlights. Two black hunter drones with single red eyes hover over the next building. Dark silhouettes and bright rain. Motion prompt: Searchlights sweep left to right while drones rotate toward the roof. Sound: low drone hum, scanning beeps.
7. Shot 07. Picture prompt: Side view chest-up shot of Rin with her orange hood up, pressed back against a dark brick rooftop wall in the rain, glancing up warily. Kite the small lightning fox peeks out of the inside of her hood right beside her cheek. A red searchlight glow sweeps across the wall above them. Arms and hands out of frame. Motion prompt: Kite ducks into the hood. Rin lowers her head below the passing searchlight. Sound: cloth rustle, crackle, pattering rain.
8. Shot 08. Picture prompt: Low angle from rooftops up to the spire. A ring made of jagged blue lightning, with three distant luminous fox silhouettes running in the clouds above Skyline Tower. No foreground characters. Motion prompt: Camera tilts from the needle to the ring; distant fox silhouettes run clockwise through its light. Sound: deep distant thunder, faint bell chimes.
9. Shot 09. Picture prompt: Tight profile of the tiny fox peeking above Rin's orange shoulder, seeing its family in the lightning ring. Rin's eyes follow the same direction. Crop above chest. Motion prompt: The fox looks up from the hood; Rin follows its eyeline to the clouds. Sound: high wind and answering chimes.
10. Shot 11. Picture prompt: Medium low-angle view on the storm roof. A drenched white-haired young man in a black cyan-lined jacket has just landed, out of breath. Face bare, eyes and smile clear. Arms behind torso. Motion prompt: Sora lands in a puddle and straightens, catching his breath; rain drips from his ponytail. Sound: boots splash, short landing thump.
11. Shot 12. Picture prompt: Shoulder-level two-shot on a rainy rooftop at night, both seen from the chest up. Left: Rin with her orange hood up, Kite sitting on her shoulder beside her hood. Right: Sora, wet white ponytail, black jacket with cyan lines, staring at the little fox in amazement. Arms and hands out of frame. Motion prompt: Rack focus from fox ears to Sora's startled eyes. He stops on the spot. Sound: soft crackle, rooftop wind.
12. Shot 13. Picture prompt: Close portrait of Rin with her orange hood up, determined expression, looking toward a distant glowing needle tower. Kite is curled on her shoulder against her neck, its soft blue glow lighting her cheek. Frame ends at the shoulders. Motion prompt: Camera pushes gently toward her eyes as she looks from the injured creature to the tower. Sound: rain on hood, soft electric pulse.
13. Shot 14. Picture prompt: Three-quarter shoulder portrait of Sora, uncovered face, white tied hair and violet eyes. The distant tower glows on the opposite side of the valley. Motion prompt: Sora turns toward the tower; lightning briefly outlines the skyline. Sound: distant thunder, jacket in wind.
14. Shot 15. Picture prompt: Medium orange hooded courier rises from a crouch. Tiny blue fox ears show above the rim of her hood. Arms tucked against her coat. Motion prompt: Rin stands, leans into the wind and starts forward. Sound: wet bootstep, cloth snap.
15. Shot 16. Picture prompt: Close on Sora's bare smiling face, white ponytail and black jacket, kind violet eyes; shoulder-up profile. Motion prompt: He shakes his head with a grin and turns to follow. Sound: soft rain and wind.
16. Shot 17c. Picture prompt: Wide rear three-quarter view of white-haired Sora vaulting over a steaming vent, legs tucked; his arms on the far side of the vent are obscured. Motion prompt: Sora clears the vent in one vault. Camera tracks his boots. Sound: boot scrape and whoosh.
17. Shot 17d. Picture prompt: Wide side silhouettes of Rin and Sora sprinting along a narrow neon roof edge, orange ahead of black and blue. Arms close to their sides, feet splash. Motion prompt: Side tracking shot of the pair running along the roof edge. Sound: running footsteps on wet metal.
18. Shot 17f. Picture prompt: Wide silhouette of Sora jumping a roof gap; cyan-lined coat and tied white hair above city lights, face visible, arms behind torso. Motion prompt: He jumps the gap after Rin and lands in frame. Sound: air rush, metal landing.
19. Shot 18. Picture prompt: Wide alley view from behind the runners. A black hunter drone turns sharply; its red searchlight paints both figures. Motion prompt: The drone banks. Its red beam locks on the pair as they run. Sound: alarm beep, rising siren whoop.
20. Shot 19a. Picture prompt: Black red-eyed hunter drones dive between neon buildings over a street canyon. Motion prompt: Camera dives with the leading drone as the others peel past a sign. Sound: descending drone whine.
21. Shot 19c. Picture prompt: Wide side view, Sora shoulder-checks a black drone off a metal ledge. Arms tight against his jacket; his shoulder, not his hands, meets its shell. Motion prompt: A shoulder charge knocks the drone sideways off the ledge; sparks fall. Sound: metal clang and falling sparks.
22. Shot 21. Picture prompt: Medium shot on the roof of a sky-tram at night, city lights far below. Rin and Sora sit side by side, catching their breath and laughing. Kite sits on Rin's shoulder and nuzzles her cheek. Arms and hands out of frame. Motion prompt: They catch their breath in the wind. Kite brushes Rin's cheek and she smiles. Sound: tram wind, gentle crackle, soft laugh.
23. Shot 22. Picture prompt: Close shot on a sky-tram roof at night. Kite sits on Rin's shoulder beside her face, but its light is very faint, its body pale and semi-transparent with a few weak sparks, ears drooping. Rin looks at it with worry. Arms out of frame. Motion prompt: The fox's light flickers and dims. Rin watches in silence. Sound: weak electric pulse and wind.
24. Shot 24. Picture prompt: Wide tram roof. A drone beam scorches a patch of wet metal; Sora blocks it with his shoulder, moving Rin clear. Their arms are hidden behind their bodies. Motion prompt: Beam strikes; Sora shields Rin with his shoulder and falls to one knee. Sound: electric blast, heavy impact, low grunt.
25. Shot 25. Picture prompt: Medium close, battered Sora on one knee facing Rin; his unmasked face is resolute. His arm gestures beyond frame toward the tower. Motion prompt: He looks up and nods urgently toward the tower, then turns to face the hunters. Sound: drone alarm, rain on metal.
26. Shot 26. Picture prompt: Wide shot from behind: Rin runs alone along a long elevated walkway toward a distant tall needle tower glowing electric blue, in the rain. Far to the right, round black hunter drones with glowing red eyes veer away from her. Only one person in the scene. Motion prompt: Follow Rin's jump as the hunters bank away toward Sora behind her. Sound: leap whoosh and drone engines peeling away.
27. Shot 27. Picture prompt: Wide exterior shot of a tall needle tower in a storm at night. Rin climbs a metal ladder on the outside of the tower; Kite clings to her shoulder, ears flat in the wind. Her hands hidden behind the ladder rails. Motion prompt: Track upward with her climbing boots. The wind tugs the hood, and Kite clings close. Sound: wind howl, metal creak, thunder.
28. Shot 29. Picture prompt: Rin at the tower's edge in profile. The tiny fox sits on her raised forearm, wrapped in her sleeve, facing the ring in the sky. Sleeves and lower arms are below the frame. Motion prompt: She gently lifts the fox toward the clouds; its wings open halfway. Sound: high wind and soft chimes.
29. Shot 30. Picture prompt: Close on the tiny lightning fox, one bent wing, looking back over its shoulder at Rin. Its lightning dims against the storm. Motion prompt: Kite turns back, ears lowered, sparks weakening. Sound: weak crackle in wind.
30. Shot 31. Picture prompt: Wide shot from behind Rin on a narrow metal platform at the top of the tower in a storm. Kite sits on her shoulder looking back. Three round black hunter drones with single glowing red eyes rise out of the clouds, red searchlights fixed on her. Motion prompt: Red lights climb into view and fix on Rin. Sound: drone hum, target-lock beeps.
31. Shot 32. Picture prompt: Low wide shot from behind Rin on the stormy tower top: she stands her ground facing a half circle of round black hunter drones with glowing red eyes. Kite crouches on the metal floor right behind her boots, looking up at her. Motion prompt: She turns to face the hunters. Her boots plant and she holds her ground. Sound: drone pressure, then silence.
32. Shot 33. Picture prompt: The spire fills the composition. A blue-white lightning bolt hits the tip and one tiny fox flares into white light. Motion prompt: One blinding strike hits the spire; blue light expands outward. Sound: largest thunder crack, electric roar.
33. Shot 35. Picture prompt: Medium portrait of Sora, scratched but safe, emerging over the tower edge. Face fully visible, violet eyes lifted at the giant fox; forearms below frame. Motion prompt: His head rises above the lip of the tower. He looks up in awe. Sound: wind and a bootstep on steel.
34. Shot 36. Picture prompt: Intimate side profile. A giant blue lightning fox gently lowers its muzzle to Rin. Rin rests her forehead against its glowing nose. Crop at shoulders, no arms in view, tender silent goodbye. Motion prompt: The fox lowers its glowing nose to her forehead. She closes her eyes; hold the contact. Sound: warm electrical hum and soft chimes.
35. Shot 37. Picture prompt: Wide sky above a tall needle tower and the city at dawn. A single giant glowing blue fox made of lightning leaps upward into a ring of clouds where small fox spirits of light run in a circle. Golden sunrise light pours through. Motion prompt: The giant fox bounds up into the ring, the clouds part behind it. Sound: rising chimes, thunder rolling away.
36. Shot 39. Picture prompt: Medium-wide tower at golden sunrise. Rin and Sora sit together, exhausted, smiling toward the horizon. Hands behind them or in their laps below frame; Sora's face bare. Motion prompt: The pair exchange a tired smile, then look out together. Sound: morning breeze and distant birds.
37. Shot 40. Picture prompt: Shoulder-level portrait at sunrise. One blue spark settles on the orange fabric at Rin's chest and shines against the jacket. Her hands are entirely outside the composition. Motion prompt: The spark drifts down, settles on her jacket, and Rin smiles. Sound: one delicate chime.
38. Shot 41. Picture prompt: Very wide final image. Two tiny seated silhouettes, orange and black, on the Skyline Tower against a golden city. A single faint fox-shaped cloud hangs in the dawn sky; serene, simple composition. Motion prompt: Very slow pullback; wisps of cloud drift through the faint fox outline. Hold four seconds. Sound: morning breeze, resolved chime.

Frequently asked questions

Can you make an anime with AI on one laptop?

Yes. The Lightning Fox, a four-minute short, was made on one laptop with an NVIDIA RTX 5090 Laptop GPU (24 GB), running every model locally in Promptus.

How do you keep AI anime characters consistent between shots?

Use a character sheet per character, paste the same written description into every prompt, and pass each character's sheet in as a reference image when you generate the key still. Then animate with small, slow movements, because big movements are where characters fall apart.

Which AI models did you use?

FLUX.2 Klein 4B for character sheets and key stills, LTX-2.5 Image to Video for animation, Chatterbox Turbo for character voices, and ACE-Step 1.5 for music, all run locally in Promptus.

How long does each generation take?

On my laptop: about 1 minute 9 seconds for a reference still, about 1 minute 49 seconds for a 5-second video clip at 960×544, and about 28 seconds for a music cue. Your times will differ with your GPU.

How do you stop AI from ruining hands?

Write the framing into the prompt so hands are out of shot, hidden behind objects or tucked into a jacket, add "deformed hands, extra fingers, extra limbs" to the negative prompt, and discard any clip that still gets them wrong.

Summary

1. One simple story: one goal, one deadline, one climax, one goodbye.
2. One character sheet and one fixed description per character.
3. One key still per shot, with the characters passed in as references.
4. One action per shot, animated with a short motion prompt.
5. Check faces and fingers, then build the sound in layers.

Try it yourself: download Promptus, switch the Playground to Local, and start with three pictures: the discovery, the decision and the goodbye. Or watch the full tutorial on YouTube.