diff --git a/CLAUDE.md b/CLAUDE.md index 41f86e8..c471fca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -65,14 +65,32 @@ that contains this repo on `sys.path` and importing the repo folder name (see properties when they resync, so instances are traced back to their library (empty → `instance_collection.library`, override → `override_library.reference.library`). - `library.reload()` invalidates every Python reference to linked and override data: - look IDs up again afterwards. + look IDs up again afterwards. In Blender 5.2 it also replaces the Library itself (renamed + after its file, custom properties gone): `library.update()` finds it by path and restores + the name and tags. +- Removing a library leaves the libraries it linked indirectly (e.g. Blender's + `geometry_nodes_essentials.blend`): `remove_unused_libraries()` drops the ones whose data + is no longer used, never a managed library. - Background requests (`scene_manager.ipc.run`) hand results back through a timer; in callbacks, find scenes by name (`bpy.context.scene` is not reliable in timers). +## Real Libreflow (first run, 2026-10-27) + +- Kitsu asset types: `main_character`, `secondary_character`, `tertiary_character`, + `main_props`, `secondary_props`, `set_props`, `vehicles`, `bg_location`, `bg_set`, + `skies`, `setdress_bank`. The fake server uses the same names. +- Paths come back as Windows paths with backslashes; `comment` can be `null`. +- Set dressing publishes (`setdress_low`) hold a whole layout scene: the set is the + `environments` collection, next to `lights`, `cameras`, `characters`, `_unused`. +- Tested live: connection, context, casting, load (instance and override), version switch + and update, remove, assetize errors. Not tested: an actual assetize publish (it writes + to production). + ## Open points -- First run of the Scene Manager against a real Libreflow (only tested with the fake - server): connection, casting, load, update, assetize, Windows paths. -- Asset type names assumed `chars` / `props` / `sets` (`SCENE_MANAGER_LINK_MODES`, - `SHOT_VIEWER_ASSET_TYPE_GROUPS`): confirm with Kitsu. +- Override vs instance per asset type (`SCENE_MANAGER_LINK_MODES`), and the file patterns + of sets (`setdress_low` seen on one asset only): confirm with production. +- The server picks the first file matching a pattern even when it is not on this site, + while another matching file is (e.g. `car_paul`: `modeling/modeling.blend` v001 not + available, `modeling/tes_modeling.blend` v002 available). - Values marked TODO in `config.py` wait for production specs. diff --git a/README.md b/README.md index bdd7ab7..2b89958 100644 --- a/README.md +++ b/README.md @@ -312,7 +312,7 @@ The bottom of the panel lists the armatures of the scene, grouped into **Charact - **Shift+click**: adds it to the selection, to pose several rigs at once. - The rig(s) currently selected are highlighted. -A rig loaded by the [Scene Manager](#scene-manager) is grouped by its Libreflow asset type (`chars` → Characters, `props` → Props, see `SHOT_VIEWER_ASSET_TYPE_GROUPS`). Otherwise, a rig counts as a character or a prop when its name, one of its collections, or the library file it comes from starts with `chr_` / `prp_` (see `SHOT_VIEWER_RIG_GROUPS` in [Configuration](#configuration)). Hidden rigs cannot be selected. Camera rigs (armatures carrying a camera) are not listed. +A rig loaded by the [Scene Manager](#scene-manager) is grouped by its Libreflow asset type (`main_character`, `secondary_character`, `tertiary_character` → Characters; `main_props`, `secondary_props`, `vehicles` → Props; see `SHOT_VIEWER_ASSET_TYPE_GROUPS`). Otherwise, a rig counts as a character or a prop when its name, one of its collections, or the library file it comes from starts with `chr_` / `prp_` (see `SHOT_VIEWER_RIG_GROUPS` in [Configuration](#configuration)). Hidden rigs cannot be selected. Camera rigs (armatures carrying a camera) are not listed. --- @@ -353,10 +353,10 @@ Lists the assets cast in the shot, with how many instances are loaded out of how **What gets loaded:** -- **File:** the latest publish **available on this site** of the asset's first file matching `rigging/*.blend`, then `modeling/*.blend`. -- **Collection:** from that file, `COL_`, else ``, else the file's only top-level collection. -- **Characters and props** become **library overrides**, so animators can pose them. Other types, like sets, become **collection instances**. -- **Where they go:** each asset type gets its own collection, `COL_chars`, `COL_props`, `COL_sets`… +- **File:** the latest publish **available on this site** of the asset's first file matching `rigging/*.blend`, then `modeling/*.blend` (sets, `bg_location`: `setdress_low/*.blend`). +- **Collection:** from that file, `COL_`, else ``, else `environments` (set dressing files), else the file's only top-level collection. +- **Characters, props and vehicles** (`*_character`, `main_props`, `secondary_props`, `vehicles`) become **library overrides**, so animators can pose them. Other types, like sets and set props, become **collection instances**. +- **Where they go:** each asset type gets its own collection, `COL_main_character`, `COL_main_props`, `COL_bg_location`… All of these can be changed in the [Configuration](#configuration). @@ -460,7 +460,7 @@ Only markers bound to a camera are shots. Select the marker, select its camera, Start Libreflow (with the `wtdcmds` extension), then click the refresh icon of the Scene Manager. Its IPC server stops 30 seconds after the last Libreflow session closes. **Scene Manager: an asset cannot be loaded.** -Its red line says why. "No file matching..." means none of its files matches `SCENE_MANAGER_FILE_PATTERNS`. "No collection named..." means the file has neither `COL_` nor ``, and several top-level collections: rename the published collection, or add its name to `SCENE_MANAGER_COLLECTION_NAMES`. +Its red line says why. "No file matching..." means none of its files matches `SCENE_MANAGER_FILE_PATTERNS`. "No collection named..." means the file has none of `COL_`, `` or `environments`, and several top-level collections: rename the published collection, or add its name to `SCENE_MANAGER_COLLECTION_NAMES`. **Scene Manager: the version stays red after Update.** The newer version is published but not downloaded on this site yet. Wait for the sync, then update again. diff --git a/config.py b/config.py index 680fd15..19e2f26 100644 --- a/config.py +++ b/config.py @@ -244,7 +244,14 @@ SHOT_VIEWER_RIG_GROUPS = { "Props": r"(?i)^(?:prp|prop)[_-]", } # Rigs loaded by the Scene Manager are grouped by their Libreflow asset type first. -SHOT_VIEWER_ASSET_TYPE_GROUPS = {"chars": "Characters", "props": "Props"} +SHOT_VIEWER_ASSET_TYPE_GROUPS = { + "main_character": "Characters", + "secondary_character": "Characters", + "tertiary_character": "Characters", + "main_props": "Props", + "secondary_props": "Props", + "vehicles": "Props", +} # Shots added by the shot viewer. TODO: confirm with production. SHOT_NAME_FORMAT = "sh{:04d}" @@ -271,13 +278,26 @@ LIBREFLOW_TIMEOUT = 15 # Published file loaded for each asset type ("*" = the others): ordered # "task/file.ext" glob patterns, the first file of the asset matching one is loaded. -SCENE_MANAGER_FILE_PATTERNS = {"*": ["rigging/*.blend", "modeling/*.blend"]} +# TODO: confirm the set publishes (only setdress_low seen so far). +SCENE_MANAGER_FILE_PATTERNS = { + "*": ["rigging/*.blend", "modeling/*.blend"], + "bg_location": ["setdress_low/*.blend"], +} # Collection linked from an asset file: the first of these names found ({asset} -# = asset name), else the file's single top-level collection. -SCENE_MANAGER_COLLECTION_NAMES = ["COL_{asset}", "{asset}"] -# How an asset type is brought in: "OVERRIDE" (library override, posable) or -# "INSTANCE" (collection instance). "*" = the other types. TODO: confirm type names. -SCENE_MANAGER_LINK_MODES = {"chars": "OVERRIDE", "props": "OVERRIDE", "*": "INSTANCE"} +# = asset name), else the file's single top-level collection. "environments" +# holds the set in the setdress files. +SCENE_MANAGER_COLLECTION_NAMES = ["COL_{asset}", "{asset}", "environments"] +# How an asset type (Kitsu name) is brought in: "OVERRIDE" (library override, posable) +# or "INSTANCE" (collection instance). "*" = the other types. TODO: confirm with production. +SCENE_MANAGER_LINK_MODES = { + "main_character": "OVERRIDE", + "secondary_character": "OVERRIDE", + "tertiary_character": "OVERRIDE", + "main_props": "OVERRIDE", + "secondary_props": "OVERRIDE", + "vehicles": "OVERRIDE", + "*": "INSTANCE", +} # Collection gathering the loaded assets of a type, under the scene collection. SCENE_MANAGER_TYPE_COLLECTION = "COL_{asset_type}" # Default task of the assetize dialog. diff --git a/dev/fake_libreflow.py b/dev/fake_libreflow.py index 0f08fd5..33d0873 100644 --- a/dev/fake_libreflow.py +++ b/dev/fake_libreflow.py @@ -2,8 +2,8 @@ python dev/fake_libreflow.py -Serves a project with one shot casting bob (chars, x2), tree (sets), rock (props), -lamp (props, nothing published) and ghost (not in the flow). The asset files come +Serves a project with one shot casting bob (main_character, x2), tree (bg_set), rock (main_props), +lamp (main_props, nothing published) and ghost (not in the flow). The asset files come from dev/make_test_assets.py (/lib). Writes /kabaret/server.json: point config LIBREFLOW_SERVER_FILE to it (see dev/test_scene_manager.py). @@ -26,15 +26,15 @@ def rev(name, path, available=True, comment=""): "available": available, "date": time.time(), "user": "nero", "comment": comment} ASSETS = { # asset oid -> (type, {file_oid: (label, [revisions oldest first])}) - f"{PROJECT}/asset_types/chars/assets/bob": ("chars", {f"{PROJECT}/asset_types/chars/assets/bob/tasks/rigging/files/rig_blend": ("rigging/rig.blend", [ + f"{PROJECT}/asset_types/main_character/assets/bob": ("main_character", {f"{PROJECT}/asset_types/main_character/assets/bob/tasks/rigging/files/rig_blend": ("rigging/rig.blend", [ rev("v001", f"{ROOT}/lib/bob/v001/rig.blend"), rev("v002", f"{ROOT}/lib/bob/v002/rig.blend", comment="two meshes")])}), - f"{PROJECT}/asset_types/sets/assets/tree": ("sets", {f"{PROJECT}/asset_types/sets/assets/tree/tasks/modeling/files/model_blend": ("modeling/model.blend", [ + f"{PROJECT}/asset_types/bg_set/assets/tree": ("bg_set", {f"{PROJECT}/asset_types/bg_set/assets/tree/tasks/modeling/files/model_blend": ("modeling/model.blend", [ rev("v001", f"{ROOT}/lib/tree/v001/model.blend")])}), - f"{PROJECT}/asset_types/props/assets/rock": ("props", {f"{PROJECT}/asset_types/props/assets/rock/tasks/modeling/files/rock_blend": ("modeling/rock.blend", [ + f"{PROJECT}/asset_types/main_props/assets/rock": ("main_props", {f"{PROJECT}/asset_types/main_props/assets/rock/tasks/modeling/files/rock_blend": ("modeling/rock.blend", [ rev("v001", f"{ROOT}/lib/rock/v001/rock.blend")])}), - f"{PROJECT}/asset_types/props/assets/lamp": ("props", {}), + f"{PROJECT}/asset_types/main_props/assets/lamp": ("main_props", {}), } -CASTING = [("bob", "chars", 2), ("tree", "sets", 1), ("rock", "props", 1), ("lamp", "props", 1), ("ghost", "chars", 1)] +CASTING = [("bob", "main_character", 2), ("tree", "bg_set", 1), ("rock", "main_props", 1), ("lamp", "main_props", 1), ("ghost", "main_character", 1)] CALLS = [] def file_data(file_oid): diff --git a/dev/make_test_assets.py b/dev/make_test_assets.py index 924199c..e05f8f9 100644 --- a/dev/make_test_assets.py +++ b/dev/make_test_assets.py @@ -4,7 +4,8 @@ bob: a rig (COL_bob: armature + meshes), v001 has 1 mesh, v002 has 2. tree: COL_tree, plus an unrelated top-level collection. -rock: a single top-level collection not named after the asset (fallback case). +rock: a single top-level collection not named after the asset (fallback case), its mesh +using a node group linked from Blender's essentials (an indirect library). """ import os @@ -21,7 +22,7 @@ def mesh_object(name): return bpy.data.objects.new(name, mesh) -def write(path, collection_name, meshes=1, rig=False, extra=False): +def write(path, collection_name, meshes=1, rig=False, extra=False, node_group=False): bpy.ops.wm.read_homefile(use_empty=True) scene = bpy.context.scene collection = bpy.data.collections.new(collection_name) @@ -39,6 +40,11 @@ def write(path, collection_name, meshes=1, rig=False, extra=False): ob = mesh_object(f"GEO_{collection_name}_{index}") ob.parent = parent collection.objects.link(ob) + if node_group: + essentials = os.path.join(bpy.utils.system_resource("DATAFILES"), "assets", "nodes", "geometry_nodes_essentials.blend") + with bpy.data.libraries.load(essentials, link=True) as (_data_from, data_to): + data_to.node_groups = ["Smooth by Angle"] + ob.modifiers.new("Smooth", "NODES").node_group = data_to.node_groups[0] if extra: scene.collection.children.link(bpy.data.collections.new("WIP_other")) os.makedirs(os.path.dirname(path), exist_ok=True) @@ -49,5 +55,5 @@ lib = os.path.join(WORKDIR, "lib") write(os.path.join(lib, "bob", "v001", "rig.blend"), "COL_bob", meshes=1, rig=True, extra=True) write(os.path.join(lib, "bob", "v002", "rig.blend"), "COL_bob", meshes=2, rig=True, extra=True) write(os.path.join(lib, "tree", "v001", "model.blend"), "COL_tree", extra=True) -write(os.path.join(lib, "rock", "v001", "rock.blend"), "rock_geo") +write(os.path.join(lib, "rock", "v001", "rock.blend"), "rock_geo", node_group=True) print("Test assets written in", lib) diff --git a/dev/test_scene_manager.py b/dev/test_scene_manager.py index 5a00482..efd4160 100644 --- a/dev/test_scene_manager.py +++ b/dev/test_scene_manager.py @@ -32,7 +32,7 @@ shot_viewer = sys.modules[PACKAGE + ".shot_viewer"] SERVER_FILE = os.path.join(WORKDIR, "kabaret", "server.json") config.LIBREFLOW_SERVER_FILE = SERVER_FILE SHOT = "/melvile/films/f01/sequences/s010/shots/p010" -BOB_FILE = "/melvile/asset_types/chars/assets/bob/tasks/rigging/files/rig_blend" +BOB_FILE = "/melvile/asset_types/main_character/assets/bob/tasks/rigging/files/rig_blend" FAILED = [] @@ -102,7 +102,7 @@ check("bob: 2 instances of v002", state.get("bob") == ("v002", ["COL_bob", "COL_ check("tree: 1 collection instance", state.get("tree") == ("v001", ["tree"]), str(state.get("tree"))) check("rock: single top-level collection fallback", state.get("rock") == ("v001", ["rock_geo"])) check("lamp and ghost reported", "lamp" in message() and "ghost" in message(), message()) -check("type collections", {"COL_chars", "COL_sets", "COL_props"} <= set(scene.collection.children.keys())) +check("type collections", {"COL_main_character", "COL_bg_set", "COL_main_props"} <= set(scene.collection.children.keys())) check( "chars are posable overrides", all(ob.override_library for ob in bpy.data.collections["COL_bob"].all_objects), @@ -148,6 +148,7 @@ bpy.ops.wtd_scene.remove(selected=True) check("selected instance removed", loaded()["tree"][1] == []) bpy.ops.wtd_scene.remove("EXEC_DEFAULT", library="rock") check("whole asset removed", "rock" not in bpy.data.libraries) +check("its indirect library removed too", set(bpy.data.libraries.keys()) == {"bob", "tree"}, str(bpy.data.libraries.keys())) bpy.ops.wtd_scene.load() pump() check("load restores what is missing", {k: len(v[1]) for k, v in loaded().items()} == {"bob": 2, "rock": 1, "tree": 1}) @@ -155,7 +156,7 @@ check("load restores what is missing", {k: len(v[1]) for k, v in loaded().items( print("Ambiguous file") try: lib.link_asset( - {"name": "zzz", "oid": "x", "asset_type": "props", "file": {"file_oid": "f"}}, + {"name": "zzz", "oid": "x", "asset_type": "main_props", "file": {"file_oid": "f"}}, {"name": "v1", "path": os.path.join(WORKDIR, "lib", "bob", "v001", "rig.blend")}, SHOT, ) @@ -170,7 +171,7 @@ local.objects.link(bpy.data.objects.new("GEO_lamp", bpy.data.meshes.new("GEO_lam check("only free local collections offered", [c.name for c in lib.assetize_candidates()] == ["COL_lamp"]) sm.STATE.assets = ipc.call("wtd_list_assets", None, oid=SHOT) bpy.ops.wtd_scene.assetize( - "EXEC_DEFAULT", collection="COL_lamp", asset_type="props", asset_name="lamp", task="modeling", comment="test" + "EXEC_DEFAULT", collection="COL_lamp", asset_type="main_props", asset_name="lamp", task="modeling", comment="test" ) pump() check("published and replaced", "replaced" in message(), message()) @@ -178,7 +179,7 @@ check("lamp linked", loaded().get("lamp") == ("v001", ["COL_lamp"]), str(loaded( check("local collection kept, renamed", "COL_lamp_local" in bpy.data.collections) bad = bpy.data.collections.new("COL_bad") scene.collection.children.link(bad) -bpy.ops.wtd_scene.assetize("EXEC_DEFAULT", collection="COL_bad", asset_type="props", asset_name="bad", task="layout") +bpy.ops.wtd_scene.assetize("EXEC_DEFAULT", collection="COL_bad", asset_type="main_props", asset_name="bad", task="layout") pump() check("unknown task reported", "available: modeling, rigging" in message(), message()) diff --git a/scene_manager/library.py b/scene_manager/library.py index 76bc704..959418f 100644 --- a/scene_manager/library.py +++ b/scene_manager/library.py @@ -135,6 +135,7 @@ def link_asset(asset, revision, context_oid): f"{len(roots)} top-level collections in the file ({', '.join(c.name for c in roots)})" ) bpy.data.libraries.remove(library) + remove_unused_libraries() raise LoadError(message) chosen = roots[0].name @@ -177,14 +178,25 @@ def update(library_name, revision): """Point a library to another revision of its file and reload it.""" library = bpy.data.libraries[library_name] path = _revision_path(revision, library.get("wtd_asset_name", library_name)) + tags = dict(library) library.filepath = path library.reload() - # The reload replaces the linked data: look the library up again. - library = bpy.data.libraries[library_name] + # The reload replaces the library (named after its file, no custom properties): + # find it by its path, then give it back its name and tags. + library = next((lib for lib in bpy.data.libraries if _same_path(lib.filepath, path)), None) + if library is None: + raise LoadError(f"{library_name}: {os.path.basename(path)} could not be reloaded") + library.name = library_name + for key, value in tags.items(): + library[key] = value library["wtd_revision"] = revision["name"] return library +def _same_path(a, b): + return os.path.normcase(os.path.normpath(bpy.path.abspath(a))) == os.path.normcase(os.path.normpath(b)) + + def remove_instance(instance): if isinstance(instance, bpy.types.Object): bpy.data.objects.remove(instance) @@ -204,6 +216,23 @@ def remove_library(library): for instance in instances(library): remove_instance(instance) bpy.data.libraries.remove(library) + remove_unused_libraries() + + +def remove_unused_libraries(): + """Remove the libraries none of whose data is used any more, e.g. the ones a removed + asset linked indirectly (node group assets...). Managed libraries are kept.""" + while True: + used = { + id.library.name + for id in bpy.data.user_map() + if id.library is not None and (id.users or id.use_fake_user) + } + unused = [lib for lib in bpy.data.libraries if lib.name not in used and not lib.get(ASSET_OID)] + if not unused: + return + for lib in unused: + bpy.data.libraries.remove(lib) # --------------------------------------------------------------------------- #