Working inside of Macros isn't working properly

Jan · February 21, 2025, 2:57pm

Hi,
In our production (big open-world franchise) we haven’t been able to use any WM version past 3029. Mainly this was because the broken tile-importer but also because working inside macros stopped working properly in 4xxx.

Now that the tile stuff seems to be working I’ve spent some time investigating 4045 to see if it is suitable for our production. But unfortunately it is not.

In order to structure our scenes well enough (they get complex) we structure it with macros and output the result down the network. For instance “core shapes” > “underwater” > “cliffs” > “surface detail”. This makes the top level very easy to navigate and understand, and if I need to tweak some settings it’s easy to step inside the macro to do the work.

But… In the 4xxx versions, this isn’t working:

First of all, just selecting a node inside a macro, and then generate to that device (yellow button) forces the entire macro to re-generate. This alone invalidates our scenes, since a macro can be quite intense, and also have nested macros. I can’t wait 20-30 seconds when just tweaking a blur or adjusting an expand. It should only take milliseconds. It’s a blocker.

Secondly, often I only get a 256x256 preview resolution when generating to a device inside a macro. This is not always the case though, and I’m not sure why.

To reproduce: Create a macro in a simple scene. Go inside and set up a network of nodes. Make sure to have some erosion in there. End with a simple blur device or something. Select the blur device and generate to selected device. Observe render time and preview resolution. Now tweak the blur settings and repeat the generation to the device.

Expected result: It should only take milliseconds, and the preview resolution should be the same resolution as your scene/extent.
Actual result: The entire macro is regenerated. And likely the preview resolution is low.

(I’ve been trying to get around this but putting everything in the top level, but the new versions is just too slow with a lot of nodes.)

I think 4045 is a great update overall for simple scenes, but I would really like to be able to use it on our next project. I’m really sad that we can’t take advantage of the speed/quality improvements of the newer versions.

Cheers,
-Jan

WFab · February 21, 2025, 5:35pm

@Jan

As someone who had the same issues working for a well known animation facility almost 15 years back (they relied heavily on macros as well)…

Use “Blueprints” instead of macros for repetitive tasks if you want to keep your complex projects editable.

Macros keep only their outputs after a build, all other data is dumped. Makes your builds slug through their internals every time.

Blueprints are like “graphlets”, you can save a portion of your graph for reuse. This should solve some of your use cases.

For the rest, @Stephen has talked about this topic before. He may elaborate on it further, so do keep following this thread.

Tomatiy · February 21, 2025, 5:50pm

In last update i did read a string about that @Stephen did make macroses to rebuild only edited nodes inside of a macroses. Maybe it means that macro will rebuild only that one device inside, if you will edit it by using parameters interface of macro, and not by direct editing of that device inside of macro by hand?

Jan · February 21, 2025, 6:43pm

Thanks for the reply!

I just want to point out that this is a 4xxx issue and it works fine in 3029 for instance.

It’s like the 4xxx process can’t see inside a macro anymore. When triggering a build-to-device inside a macro and looking at the render progress, one can’t see any of the devices inside the macro, only the macro itself in the context of the top level. So it seems like a macro in 4xxx can’t be partially generated.

In 3xxx when generating a macro from the top level meant that yes, the devices inside wouldn’t keep their individual outputs, which is good and an important part of keeping memory in check. But now when I try to have everything in the top level I’m juggling well over 100gb in memory and it’s just not very efficient.

Blueprints won’t help. We use macros mainly for organisational reasons, as opposed to avoid setting up often-used network snippets. (Our goal is to keep the network clean and readable, to easily swap out and compare sections of the graph, to keep memory and overall editor UI performance in check).

Cheers!

Stephen · February 21, 2025, 8:42pm

Hi Jan,

Thank you for the bug report! I can confirm the macro rebuild situation you describe. Given that 4046 Final was released yesterday, I wish I had learned of this issue months ago But I’ll see what I can do.

Essentially, macros operate in either

Always-conserve-memory mode, or
Whatever the world is set to, when the macro is opened for editing.

Normally the triggering device (white) should only invalidate its following devices, but you can see that the Parameters and Input devices (and their dependency chains) are marked as well for rebuild.

That behavior you’re seeing is designed to ensure correct rebuild behavior in the face of memory conservation mode. It should turn off when the macro is open, but it is not.

Jan · February 24, 2025, 6:22am

Thanks Stephen, it would be awesome if this is fixed! And if you’re active on the development, I can submit some suggestions that would also help us out . (“Instanced Inputs” is probably the main one, that would simplify our work the most. We’re spending a lot of time keeping the network clean and legible. But funneling inputs all through the network is time consuming. It would be a lot nicer to have all our file inputs in one group to the side, and just copy the inputs as instances to where they are needed in the graph. Same thing for macros, but needed to a lesser extent).

For now I’m aborting my evaluation of 4045 and our team will need to go back to 3029. But I hope we can upgrade in time for the next project!

Keep an eye out for our game, for which we used WM a lot on the terrain. It is released on March 20!

cheers,
-Jan

Stephen · March 7, 2025, 7:20am

Just an update that the fix for this is one of the changes in the 4047 bugfix release.

Jan, would love to hear more details about game release - What is the name?

Jan · March 10, 2025, 9:44am

cheers Stephen, I’ve downloaded the 4047 version and will have a look. I’m in the middle of something else now so it might be a few days before I can look at this again. Thanks again…

About the game, we’re not supposed to talk about it before release. But…

Tomatiy · March 10, 2025, 10:30am

i did hear story that Odyssey map was made by using WM by one environment artist girl (or he, or they, sorry)
but now i’m surprised

Jan · March 17, 2025, 3:43pm

Hey Stephen, I had the chance to test it today. It seems that it works but only one level down.

In nested macros (a macro inside a macro) the issue is still there unfortunately. And we do have quite a few nested macros throughout the network in order to be able to switch out components inside bigger components. For instance, a macro that spawns trees (for visualisation only) is a fairly small macro but it is copied a few times inside another macro for different types of trees. But they are all individually tweaked (which is now a terrible job). Other examples are cliff structures, underwater sand patterns, etc, etc…

Is it possible to fix that?

4047 (as long as I don’t find something else) might not be a dealbreaker anymore though. Some macros will become a lot bigger inside, if I have to bring everything into “sublevel 1” from having been in “sublevel 2”, or even “sublevel 3”. But at least the top level should be cleaner and more responsive to work with.

Any thoughts? I would like to avoid having to rebuild everything, and lose flexibility, if I don’t have to!

Cheers!

EDIT: I might have been premature… I’m not sure what is going on but when testing with simple nested macros it seems to work, so I need to investigate a bit more with the content I’m trying to convert. BUT, if you think there might be some issues lingering with nested macros, feel free to let me know!

EDIT2: Some strange stuff is going on. Even just one level down I sometimes get a complete rebuild of the entire macro. But I can’t find any reliable repro case! I’ll spend some time tomorrow as well to see if I can find some consistency in a clean scene without any legacy devices/macros from previous versions.

Stephen · March 17, 2025, 7:14pm

I was going to say - the changes shouldn’t be sensitive to nesting. However, the behavior IS sensitive to if the macro is open or not. For example, if you have the top and a deep-level macro open, but not the intermediate macro - it will trigger a complete rebuild of the intermediate macro (because memory is being conserved within it)

Edit:

I should also mention that you will see a rebuild occur when you first open the macro and then change something, because it does not have internal results from prior builds. Once the macro has been rebuilt once it should do the expected thing.