viralvision. Case Study · Design & Development
ViralVision takes one song and a folder of footage and returns several distinct 9:16 clips, cut to the beat. The artist downloads them. It does not post them.
Musicians sit on hours of live footage, rehearsal tapes, and phone clips, and they still need new verticals every week. Editing each one by hand means finding the beat, picking a shot, cropping 9:16, and laying captions. Most of that work is the same job, repeated. A timeline editor handles one cut at a time. This job is a stack of different cuts from the same source.
ViralVision is our pipeline for that work: connect the footage, add the song, get several on-beat clips back. The posting stays with the artist.
A deployed private-beta app at app.viralvision.quest, and a public marketing site at viralvision.quest. An allowlisted account signs in (Google or email), picks footage from Google Drive, uploads a song, and submits a job. The Python worker on Contabo ingests, plans, renders, and writes 1080×1920 clips to S3. The app shows progress and a download. Invite only, with no public signup and no billing. It never posts to TikTok, Instagram, or YouTube.
The harder engineering is the cut: scene-detect the library, measure the actual beat grid, let an LLM fill a schema whose slot durations are already set, then assemble with FFmpeg so the transitions land on the beat.
How it's built
The worker measures the music. The model only fills a plan that already has beat-sized slots.
Raw video is split with PySceneDetect, windowed, audio stripped, a keyframe pulled, and vision-tagged into a per-artist manifest. The generator picks from that library, not from a raw dump of files.
librosa derives beat times, tempo, and an energy drop. Clip start can be the first beat, the drop, or a set offset. Cut durations come from that grid. Quiet intros do not set the pace.
The LLM assigns footage and in-points to slots that already exist. Client-side jsonschema checks the plan. A config switch picks the provider (Claude CLI or OpenRouter). Domain code never imports a vendor SDK.
Each segment is cropped to 9:16, concatenated, kinetic captions overlaid (Pillow PNGs, not ffmpeg drawtext), song audio faded in and out. ffprobe then checks 1080×1920, duration, and a non-empty file. A clip that fails those checks is not shipped.
A run produces N clips (1–10) with different hooks, song sections, opening shots, and pacing. The same footage and song should not yield near-duplicates.
Next.js 15 on Vercel. Self-hosted Supabase (GoTrue, Postgres, Kong) on Contabo. Google Drive Picker with drive.file scope only. S3 keys are prefixed by user id. The worker deploys on push to main via GitHub Actions over a forced-command SSH key, then systemd restarts it.
PySceneDetect · librosa beat grid · schema-validated LLM · FFmpeg 9:16 + verify · Next.js 15 · self-hosted Supabase · Drive.file ingest.
What this shows
Much of our client work takes messy source files, runs a long job, and returns a result. This pipeline measures beats and scenes, gives the model a structured plan it is not allowed to break, and verifies after FFmpeg.
We designed and built ViralVision: ingest, planning, render, the beta app, and the deploy. Private beta today. It makes varied on-beat clips, and we do not promise virality.
We design and build ingest-to-render pipelines, structured model output, and the apps that queue them, from one studio.
