Scene Manager: fixes from the first run against a real Libreflow

- Update / version switch: Blender 5.2 replaces the library on reload
  (renamed after its file, custom properties gone); find it by path and
  restore its name and tags.
- Remove the libraries an asset linked indirectly when it is removed or
  fails to load (remove_unused_libraries), never a managed one.
- Config: real Kitsu asset types for link modes and Shot Viewer groups,
  setdress_low files for bg_location, "environments" collection fallback.
- Dev fake server and test use the real asset types; rock links a node
  group from the essentials to cover the indirect library cleanup.
- README and CLAUDE.md updated with the live findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
NDemaerschalkandClaude Opus 5.5 committed 2026-10-27 11:30:46 +01:00
1 parent 017bdfc2fd
commit 9ee7710479
7 files changed
+109 -35

No files matched your search

+23 -5
View File
@@ -65,14 +65,32 @@ that contains this repo on `sys.path` and importing the repo folder name (see
properties when they resync, so instances are traced back to their library (empty →
`instance_collection.library`, override → `override_library.reference.library`).
- `library.reload()` invalidates every Python reference to linked and override data:
look IDs up again afterwards.
look IDs up again afterwards. In Blender 5.2 it also replaces the Library itself (renamed
after its file, custom properties gone): `library.update()` finds it by path and restores
the name and tags.
- Removing a library leaves the libraries it linked indirectly (e.g. Blender's
`geometry_nodes_essentials.blend`): `remove_unused_libraries()` drops the ones whose data
is no longer used, never a managed library.
- Background requests (`scene_manager.ipc.run`) hand results back through a timer;
in callbacks, find scenes by name (`bpy.context.scene` is not reliable in timers).
## Real Libreflow (first run, 2026-10-27)
- Kitsu asset types: `main_character`, `secondary_character`, `tertiary_character`,
`main_props`, `secondary_props`, `set_props`, `vehicles`, `bg_location`, `bg_set`,
`skies`, `setdress_bank`. The fake server uses the same names.
- Paths come back as Windows paths with backslashes; `comment` can be `null`.
- Set dressing publishes (`setdress_low`) hold a whole layout scene: the set is the
`environments` collection, next to `lights`, `cameras`, `characters`, `_unused`.
- Tested live: connection, context, casting, load (instance and override), version switch
and update, remove, assetize errors. Not tested: an actual assetize publish (it writes
to production).
## Open points
- First run of the Scene Manager against a real Libreflow (only tested with the fake
server): connection, casting, load, update, assetize, Windows paths.
- Asset type names assumed `chars` / `props` / `sets` (`SCENE_MANAGER_LINK_MODES`,
`SHOT_VIEWER_ASSET_TYPE_GROUPS`): confirm with Kitsu.
- Override vs instance per asset type (`SCENE_MANAGER_LINK_MODES`), and the file patterns
of sets (`setdress_low` seen on one asset only): confirm with production.
- The server picks the first file matching a pattern even when it is not on this site,
while another matching file is (e.g. `car_paul`: `modeling/modeling.blend` v001 not
available, `modeling/tes_modeling.blend` v002 available).
- Values marked TODO in `config.py` wait for production specs.
+6 -6
View File
@@ -312,7 +312,7 @@ The bottom of the panel lists the armatures of the scene, grouped into **Charact
- **Shift+click**: adds it to the selection, to pose several rigs at once.
- The rig(s) currently selected are highlighted.
A rig loaded by the [Scene Manager](#scene-manager) is grouped by its Libreflow asset type (`chars` → Characters, `props` → Props, see `SHOT_VIEWER_ASSET_TYPE_GROUPS`). Otherwise, a rig counts as a character or a prop when its name, one of its collections, or the library file it comes from starts with `chr_` / `prp_` (see `SHOT_VIEWER_RIG_GROUPS` in [Configuration](#configuration)). Hidden rigs cannot be selected. Camera rigs (armatures carrying a camera) are not listed.
A rig loaded by the [Scene Manager](#scene-manager) is grouped by its Libreflow asset type (`main_character`, `secondary_character`, `tertiary_character` → Characters; `main_props`, `secondary_props`, `vehicles` → Props; see `SHOT_VIEWER_ASSET_TYPE_GROUPS`). Otherwise, a rig counts as a character or a prop when its name, one of its collections, or the library file it comes from starts with `chr_` / `prp_` (see `SHOT_VIEWER_RIG_GROUPS` in [Configuration](#configuration)). Hidden rigs cannot be selected. Camera rigs (armatures carrying a camera) are not listed.
---
@@ -353,10 +353,10 @@ Lists the assets cast in the shot, with how many instances are loaded out of how
**What gets loaded:**
- **File:** the latest publish **available on this site** of the asset's first file matching `rigging/*.blend`, then `modeling/*.blend`.
- **Collection:** from that file, `COL_<asset>`, else `<asset>`, else the file's only top-level collection.
- **Characters and props** become **library overrides**, so animators can pose them. Other types, like sets, become **collection instances**.
- **Where they go:** each asset type gets its own collection, `COL_chars`, `COL_props`, `COL_sets`…
- **File:** the latest publish **available on this site** of the asset's first file matching `rigging/*.blend`, then `modeling/*.blend` (sets, `bg_location`: `setdress_low/*.blend`).
- **Collection:** from that file, `COL_<asset>`, else `<asset>`, else `environments` (set dressing files), else the file's only top-level collection.
- **Characters, props and vehicles** (`*_character`, `main_props`, `secondary_props`, `vehicles`) become **library overrides**, so animators can pose them. Other types, like sets and set props, become **collection instances**.
- **Where they go:** each asset type gets its own collection, `COL_main_character`, `COL_main_props`, `COL_bg_location`…
All of these can be changed in the [Configuration](#configuration).
@@ -460,7 +460,7 @@ Only markers bound to a camera are shots. Select the marker, select its camera,
Start Libreflow (with the `wtdcmds` extension), then click the refresh icon of the Scene Manager. Its IPC server stops 30 seconds after the last Libreflow session closes.
**Scene Manager: an asset cannot be loaded.**
Its red line says why. "No file matching..." means none of its files matches `SCENE_MANAGER_FILE_PATTERNS`. "No collection named..." means the file has neither `COL_<asset>` nor `<asset>`, and several top-level collections: rename the published collection, or add its name to `SCENE_MANAGER_COLLECTION_NAMES`.
Its red line says why. "No file matching..." means none of its files matches `SCENE_MANAGER_FILE_PATTERNS`. "No collection named..." means the file has none of `COL_<asset>`, `<asset>` or `environments`, and several top-level collections: rename the published collection, or add its name to `SCENE_MANAGER_COLLECTION_NAMES`.
**Scene Manager: the version stays red after Update.**
The newer version is published but not downloaded on this site yet. Wait for the sync, then update again.
+27 -7
View File
@@ -244,7 +244,14 @@ SHOT_VIEWER_RIG_GROUPS = {
"Props": r"(?i)^(?:prp|prop)[_-]",
}
# Rigs loaded by the Scene Manager are grouped by their Libreflow asset type first.
SHOT_VIEWER_ASSET_TYPE_GROUPS = {"chars": "Characters", "props": "Props"}
SHOT_VIEWER_ASSET_TYPE_GROUPS = {
"main_character": "Characters",
"secondary_character": "Characters",
"tertiary_character": "Characters",
"main_props": "Props",
"secondary_props": "Props",
"vehicles": "Props",
}
# Shots added by the shot viewer. TODO: confirm with production.
SHOT_NAME_FORMAT = "sh{:04d}"
@@ -271,13 +278,26 @@ LIBREFLOW_TIMEOUT = 15
# Published file loaded for each asset type ("*" = the others): ordered
# "task/file.ext" glob patterns, the first file of the asset matching one is loaded.
SCENE_MANAGER_FILE_PATTERNS = {"*": ["rigging/*.blend", "modeling/*.blend"]}
# TODO: confirm the set publishes (only setdress_low seen so far).
SCENE_MANAGER_FILE_PATTERNS = {
"*": ["rigging/*.blend", "modeling/*.blend"],
"bg_location": ["setdress_low/*.blend"],
}
# Collection linked from an asset file: the first of these names found ({asset}
# = asset name), else the file's single top-level collection.
SCENE_MANAGER_COLLECTION_NAMES = ["COL_{asset}", "{asset}"]
# How an asset type is brought in: "OVERRIDE" (library override, posable) or
# "INSTANCE" (collection instance). "*" = the other types. TODO: confirm type names.
SCENE_MANAGER_LINK_MODES = {"chars": "OVERRIDE", "props": "OVERRIDE", "*": "INSTANCE"}
# = asset name), else the file's single top-level collection. "environments"
# holds the set in the setdress files.
SCENE_MANAGER_COLLECTION_NAMES = ["COL_{asset}", "{asset}", "environments"]
# How an asset type (Kitsu name) is brought in: "OVERRIDE" (library override, posable)
# or "INSTANCE" (collection instance). "*" = the other types. TODO: confirm with production.
SCENE_MANAGER_LINK_MODES = {
"main_character": "OVERRIDE",
"secondary_character": "OVERRIDE",
"tertiary_character": "OVERRIDE",
"main_props": "OVERRIDE",
"secondary_props": "OVERRIDE",
"vehicles": "OVERRIDE",
"*": "INSTANCE",
}
# Collection gathering the loaded assets of a type, under the scene collection.
SCENE_MANAGER_TYPE_COLLECTION = "COL_{asset_type}"
# Default task of the assetize dialog.
+7 -7
View File
@@ -2,8 +2,8 @@
python dev/fake_libreflow.py <workdir>
Serves a project with one shot casting bob (chars, x2), tree (sets), rock (props),
lamp (props, nothing published) and ghost (not in the flow). The asset files come
Serves a project with one shot casting bob (main_character, x2), tree (bg_set), rock (main_props),
lamp (main_props, nothing published) and ghost (not in the flow). The asset files come
from dev/make_test_assets.py (<workdir>/lib). Writes <workdir>/kabaret/server.json:
point config LIBREFLOW_SERVER_FILE to it (see dev/test_scene_manager.py).
@@ -26,15 +26,15 @@ def rev(name, path, available=True, comment=""):
"available": available, "date": time.time(), "user": "nero", "comment": comment}
ASSETS = { # asset oid -> (type, {file_oid: (label, [revisions oldest first])})
f"{PROJECT}/asset_types/chars/assets/bob": ("chars", {f"{PROJECT}/asset_types/chars/assets/bob/tasks/rigging/files/rig_blend": ("rigging/rig.blend", [
f"{PROJECT}/asset_types/main_character/assets/bob": ("main_character", {f"{PROJECT}/asset_types/main_character/assets/bob/tasks/rigging/files/rig_blend": ("rigging/rig.blend", [
rev("v001", f"{ROOT}/lib/bob/v001/rig.blend"), rev("v002", f"{ROOT}/lib/bob/v002/rig.blend", comment="two meshes")])}),
f"{PROJECT}/asset_types/sets/assets/tree": ("sets", {f"{PROJECT}/asset_types/sets/assets/tree/tasks/modeling/files/model_blend": ("modeling/model.blend", [
f"{PROJECT}/asset_types/bg_set/assets/tree": ("bg_set", {f"{PROJECT}/asset_types/bg_set/assets/tree/tasks/modeling/files/model_blend": ("modeling/model.blend", [
rev("v001", f"{ROOT}/lib/tree/v001/model.blend")])}),
f"{PROJECT}/asset_types/props/assets/rock": ("props", {f"{PROJECT}/asset_types/props/assets/rock/tasks/modeling/files/rock_blend": ("modeling/rock.blend", [
f"{PROJECT}/asset_types/main_props/assets/rock": ("main_props", {f"{PROJECT}/asset_types/main_props/assets/rock/tasks/modeling/files/rock_blend": ("modeling/rock.blend", [
rev("v001", f"{ROOT}/lib/rock/v001/rock.blend")])}),
f"{PROJECT}/asset_types/props/assets/lamp": ("props", {}),
f"{PROJECT}/asset_types/main_props/assets/lamp": ("main_props", {}),
}
CASTING = [("bob", "chars", 2), ("tree", "sets", 1), ("rock", "props", 1), ("lamp", "props", 1), ("ghost", "chars", 1)]
CASTING = [("bob", "main_character", 2), ("tree", "bg_set", 1), ("rock", "main_props", 1), ("lamp", "main_props", 1), ("ghost", "main_character", 1)]
CALLS = []
def file_data(file_oid):
+9 -3
View File
@@ -4,7 +4,8 @@
bob: a rig (COL_bob: armature + meshes), v001 has 1 mesh, v002 has 2.
tree: COL_tree, plus an unrelated top-level collection.
rock: a single top-level collection not named after the asset (fallback case).
rock: a single top-level collection not named after the asset (fallback case), its mesh
using a node group linked from Blender's essentials (an indirect library).
"""
import os
@@ -21,7 +22,7 @@ def mesh_object(name):
return bpy.data.objects.new(name, mesh)
def write(path, collection_name, meshes=1, rig=False, extra=False):
def write(path, collection_name, meshes=1, rig=False, extra=False, node_group=False):
bpy.ops.wm.read_homefile(use_empty=True)
scene = bpy.context.scene
collection = bpy.data.collections.new(collection_name)
@@ -39,6 +40,11 @@ def write(path, collection_name, meshes=1, rig=False, extra=False):
ob = mesh_object(f"GEO_{collection_name}_{index}")
ob.parent = parent
collection.objects.link(ob)
if node_group:
essentials = os.path.join(bpy.utils.system_resource("DATAFILES"), "assets", "nodes", "geometry_nodes_essentials.blend")
with bpy.data.libraries.load(essentials, link=True) as (_data_from, data_to):
data_to.node_groups = ["Smooth by Angle"]
ob.modifiers.new("Smooth", "NODES").node_group = data_to.node_groups[0]
if extra:
scene.collection.children.link(bpy.data.collections.new("WIP_other"))
os.makedirs(os.path.dirname(path), exist_ok=True)
@@ -49,5 +55,5 @@ lib = os.path.join(WORKDIR, "lib")
write(os.path.join(lib, "bob", "v001", "rig.blend"), "COL_bob", meshes=1, rig=True, extra=True)
write(os.path.join(lib, "bob", "v002", "rig.blend"), "COL_bob", meshes=2, rig=True, extra=True)
write(os.path.join(lib, "tree", "v001", "model.blend"), "COL_tree", extra=True)
write(os.path.join(lib, "rock", "v001", "rock.blend"), "rock_geo")
write(os.path.join(lib, "rock", "v001", "rock.blend"), "rock_geo", node_group=True)
print("Test assets written in", lib)
+6 -5
View File
@@ -32,7 +32,7 @@ shot_viewer = sys.modules[PACKAGE + ".shot_viewer"]
SERVER_FILE = os.path.join(WORKDIR, "kabaret", "server.json")
config.LIBREFLOW_SERVER_FILE = SERVER_FILE
SHOT = "/melvile/films/f01/sequences/s010/shots/p010"
BOB_FILE = "/melvile/asset_types/chars/assets/bob/tasks/rigging/files/rig_blend"
BOB_FILE = "/melvile/asset_types/main_character/assets/bob/tasks/rigging/files/rig_blend"
FAILED = []
@@ -102,7 +102,7 @@ check("bob: 2 instances of v002", state.get("bob") == ("v002", ["COL_bob", "COL_
check("tree: 1 collection instance", state.get("tree") == ("v001", ["tree"]), str(state.get("tree")))
check("rock: single top-level collection fallback", state.get("rock") == ("v001", ["rock_geo"]))
check("lamp and ghost reported", "lamp" in message() and "ghost" in message(), message())
check("type collections", {"COL_chars", "COL_sets", "COL_props"} <= set(scene.collection.children.keys()))
check("type collections", {"COL_main_character", "COL_bg_set", "COL_main_props"} <= set(scene.collection.children.keys()))
check(
"chars are posable overrides",
all(ob.override_library for ob in bpy.data.collections["COL_bob"].all_objects),
@@ -148,6 +148,7 @@ bpy.ops.wtd_scene.remove(selected=True)
check("selected instance removed", loaded()["tree"][1] == [])
bpy.ops.wtd_scene.remove("EXEC_DEFAULT", library="rock")
check("whole asset removed", "rock" not in bpy.data.libraries)
check("its indirect library removed too", set(bpy.data.libraries.keys()) == {"bob", "tree"}, str(bpy.data.libraries.keys()))
bpy.ops.wtd_scene.load()
pump()
check("load restores what is missing", {k: len(v[1]) for k, v in loaded().items()} == {"bob": 2, "rock": 1, "tree": 1})
@@ -155,7 +156,7 @@ check("load restores what is missing", {k: len(v[1]) for k, v in loaded().items(
print("Ambiguous file")
try:
lib.link_asset(
{"name": "zzz", "oid": "x", "asset_type": "props", "file": {"file_oid": "f"}},
{"name": "zzz", "oid": "x", "asset_type": "main_props", "file": {"file_oid": "f"}},
{"name": "v1", "path": os.path.join(WORKDIR, "lib", "bob", "v001", "rig.blend")},
SHOT,
)
@@ -170,7 +171,7 @@ local.objects.link(bpy.data.objects.new("GEO_lamp", bpy.data.meshes.new("GEO_lam
check("only free local collections offered", [c.name for c in lib.assetize_candidates()] == ["COL_lamp"])
sm.STATE.assets = ipc.call("wtd_list_assets", None, oid=SHOT)
bpy.ops.wtd_scene.assetize(
"EXEC_DEFAULT", collection="COL_lamp", asset_type="props", asset_name="lamp", task="modeling", comment="test"
"EXEC_DEFAULT", collection="COL_lamp", asset_type="main_props", asset_name="lamp", task="modeling", comment="test"
)
pump()
check("published and replaced", "replaced" in message(), message())
@@ -178,7 +179,7 @@ check("lamp linked", loaded().get("lamp") == ("v001", ["COL_lamp"]), str(loaded(
check("local collection kept, renamed", "COL_lamp_local" in bpy.data.collections)
bad = bpy.data.collections.new("COL_bad")
scene.collection.children.link(bad)
bpy.ops.wtd_scene.assetize("EXEC_DEFAULT", collection="COL_bad", asset_type="props", asset_name="bad", task="layout")
bpy.ops.wtd_scene.assetize("EXEC_DEFAULT", collection="COL_bad", asset_type="main_props", asset_name="bad", task="layout")
pump()
check("unknown task reported", "available: modeling, rigging" in message(), message())
+31 -2
View File
@@ -135,6 +135,7 @@ def link_asset(asset, revision, context_oid):
f"{len(roots)} top-level collections in the file ({', '.join(c.name for c in roots)})"
)
bpy.data.libraries.remove(library)
remove_unused_libraries()
raise LoadError(message)
chosen = roots[0].name
@@ -177,14 +178,25 @@ def update(library_name, revision):
"""Point a library to another revision of its file and reload it."""
library = bpy.data.libraries[library_name]
path = _revision_path(revision, library.get("wtd_asset_name", library_name))
tags = dict(library)
library.filepath = path
library.reload()
# The reload replaces the linked data: look the library up again.
library = bpy.data.libraries[library_name]
# The reload replaces the library (named after its file, no custom properties):
# find it by its path, then give it back its name and tags.
library = next((lib for lib in bpy.data.libraries if _same_path(lib.filepath, path)), None)
if library is None:
raise LoadError(f"{library_name}: {os.path.basename(path)} could not be reloaded")
library.name = library_name
for key, value in tags.items():
library[key] = value
library["wtd_revision"] = revision["name"]
return library
def _same_path(a, b):
return os.path.normcase(os.path.normpath(bpy.path.abspath(a))) == os.path.normcase(os.path.normpath(b))
def remove_instance(instance):
if isinstance(instance, bpy.types.Object):
bpy.data.objects.remove(instance)
@@ -204,6 +216,23 @@ def remove_library(library):
for instance in instances(library):
remove_instance(instance)
bpy.data.libraries.remove(library)
remove_unused_libraries()
def remove_unused_libraries():
"""Remove the libraries none of whose data is used any more, e.g. the ones a removed
asset linked indirectly (node group assets...). Managed libraries are kept."""
while True:
used = {
id.library.name
for id in bpy.data.user_map()
if id.library is not None and (id.users or id.use_fake_user)
}
unused = [lib for lib in bpy.data.libraries if lib.name not in used and not lib.get(ASSET_OID)]
if not unused:
return
for lib in unused:
bpy.data.libraries.remove(lib)
# --------------------------------------------------------------------------- #