Story2Board / Engineering

Film Knowledge Base

A shot reference needs more than a caption. Framing, camera movement, dialogue and scene context all change whether it is useful for a storyboard. We built this library to keep those details together.

This is the data description behind Story2Board's film-reference work: what we store, how we build it and where coverage stops. It is not a public dataset download or a public search interface.

Database snapshot: . Counts reported by the Story2Board team, not an independent audit.

Annotated shots
275,346
Films
236
Director style profiles
79
Fields per shot record
48

Database size: 8.7 GB. The 48 stored fields are not 48 separate cinematic judgments: the record also needs identifiers and retrieval data. The field guide below describes 27 annotation and text fields in six groups.

What a shot record describes

These are model-generated descriptions and estimates, not ground truth from a camera report. A field existing in the schema does not mean every shot has a usable value for it. Subtitle availability, for example, is separate from description coverage.

Shot language

shot_size
How much of the subject is in frame, from a wide shot to a close-up.
camera_angle
The viewing angle relative to the subject, such as high, low or eye level.
composition
How the subjects and space are arranged inside the frame.
lighting
The apparent lighting pattern, direction and contrast.
color_palette
The dominant colors visible in the image.
color_scheme
The relationship between those colors, such as complementary or monochromatic.
depth_of_field
How much of the scene appears in focus.
focal_length_mm
An estimated focal length in millimeters, not verified camera metadata.
lens_type
The apparent lens category, inferred from the image.
aspect_ratio
The relationship between the frame width and height.

Camera motion

camera_motion
The estimated movement of the camera during the shot.
camera_motion_confidence
The motion analysis confidence score, not a human quality rating.

Scene

interior_exterior
Whether the action appears to take place indoors or outdoors.
setting
The kind of location shown, such as a room, street or landscape.
time_of_day
The apparent time of day in the scene.
weather
Visible weather conditions when they can be inferred.

Subject

subjects
The people, objects or other subjects described in the frame.
subject_count
The annotated number of visible subjects.
subject_position
Where the subjects sit within the frame.
action
What the subjects appear to be doing.

Narrative

emotional_tone
The inferred mood of the shot, rather than a measured audience response.
narrative_purpose
The inferred storytelling role of the shot in context.

Text

description
A natural-language account of what is visible in the shot.
description_zh
A Chinese-language shot description.
context_prefix
Additional scene context attached to the shot description for retrieval.
subtitle_en
English dialogue aligned to the shot, when available.
subtitle_zh
Chinese dialogue aligned to the shot, when available.

From a film to a searchable shot record

  1. Video decoding

    Decode the source video into frames that the pipeline can process.

  2. Shot boundary detection

    Locate cuts and transitions with a neural detector to define each shot.

  3. VLM annotation

    Ask a vision-language model to describe keyframes using structured cinematic fields.

  4. Shot slicing

    Extract the corresponding video segments for shot-level processing.

  5. Camera motion analysis

    Analyze movement over the segment, which a single still cannot establish.

  6. Context enrichment

    Attach aligned subtitles and surrounding-scene context to the shot record.

  7. Embeddings

    Encode images and available text for similarity search.

  8. Storage

    Store the annotations, media references and vectors together for retrieval.

How one film becomes searchable shot recordsA film is split into roughly 1,167 shots. Each shot becomes one record with 48 fields. Every record gets an image embedding; 42.6 percent also get a text embedding. Retrieval combines vector search and keyword search.INPUT1 film · roughly 2 hoursShot boundary detection — neural, not pixel differenceOne dissolve misread as three cuts is annotated, embedded andretrieved three times. The error compounds down the pipeline.≈ 1,167 shotsthe unit of work for every stage belowKeyframe sampling, then annotation by one vision-language modelqwen3-vl-flash · no human review48 fields / record27 annotation and text fields, 6 groupsEmbedding — this is where coverage stops being completeImage embeddings100%Text embeddings42.6%Retrieval runs both routes and merges themvector search + keyword search → rank fusion → rerankKeyword search still reaches records without a text embedding.
Shots per film is the corpus average (275,346 ÷ 236); individual films vary with cutting rate. Coverage figures are the September 16, 2026 snapshot.

We chose neural shot boundary detection over pixel-difference heuristics because a dissolve can be mistaken for several cuts. Each false boundary becomes another unit to annotate, embed and retrieve. The error costs time at every later stage, not just at detection. The architecture notes explain that trade-off and the rest of the pipeline decisions.

Coverage is not the same as accuracy

These counts measure whether an asset or value is present in a shot record. They do not measure annotation correctness or whether a search result helps a director.

Keyframes
275,346 / 275,346100%
Image embeddings
275,346 / 275,346100%
Chinese descriptions
275,346 / 275,346100%
Text embeddings
117,276 / 275,34642.6%
Snapshot: September 16, 2026. Text embeddings cover 42.6% of shot records; complete image embeddings do not imply complete text retrieval.

Known limitations

  1. One annotation model; no human review. The visual annotations come from a single VLM, qwen3-vl-flash. They have not been human-reviewed, and we do not report an inter-annotator agreement metric. An inferred mood, lens or narrative role can be wrong.
  2. Text embeddings cover only 42.6%. There are 117,276 text embeddings for 275,346 shot records. Do not read the corpus size as the number of shots available to every text-based search.
  3. The film list is private. We publish the aggregate film count, not individual titles. This snapshot does not include a verified decade or region breakdown, so it cannot establish how representative the collection is.

A specific comparison with MovieNet

MovieNet is a movie-understanding research dataset from CUHK, published at ECCV 2020. Its official project site reports 1,100 movies and 60,000 trailers. Its paper, table 4(a), reports 46,857 shots from 7,858 trailers in the cinematic-style benchmark. That subset, not MovieNet as a whole, is the relevant comparison here.

Breadth and annotation depth, comparedMovieNet covers about 4.7 times more films. This corpus produces roughly 79 times more cinematic-style annotation values, because it labels 27 fields per shot instead of 2.BREADTH · films coveredMovieNet1,100 filmsThis corpus236 filmsMovieNet covers about 4.7× more films. That gap is real.DEPTH · cinematic-style annotation valuesMovieNet93,714= 46,857 shots × 2 fieldsThis corpus≈ 7.4M= 275,346 shots × 27 fieldsRoughly 79× more labelled cinematic values, from 4.7× fewer films.
Depth compares cinematic-style labels only. MovieNet also ships character boxes, scene boundaries, action and place tags and aligned descriptions, which this corpus does not have; those are excluded from both bars. Bar lengths are proportional within each row, not across rows.
MovieNet cinematic-style subset and Story2Board's September 2026 snapshot
ScopeMovieNet (2020)Story2Board (2026-09-16)
Films / trailers1,100 films; 60,000 trailers236 films
Cinematic annotations46,857 shots from 7,858 trailers, labeled for scale and movement275,346 automatically annotated shot records
Fields discussed here2 cinematic-style categories: shot scale and movement27 annotation and text fields across 6 groups; per-field completeness varies
Annotation processManual annotations reported in the paperAutomated; no human review or agreement metric

MovieNet also includes character, scene, action and other annotations. Its two cinematic-style categories are not its entire schema. Our broader record format is designed for storyboard reference retrieval; counting fields does not demonstrate better labels, better retrieval, or a larger movie dataset.

Engineering notes

Seven accounts of the decisions and failures behind the library. These are engineering records, not a benchmark or a promise that every generated panel uses a film reference. Use the dated snapshot on this page for current counts and coverage.

Access and use

This page is a public description of a private knowledge base. We do not offer the underlying films, clips, keyframes, annotations or embeddings as a downloadable dataset. Publishing this description does not grant a license to those materials. No public search endpoint is provided here.

For the product workflow, see the AI storyboard generator. Our references and open-source dependencies are documented separately. Questions about the methodology can go through our contact page.