WM/UI getting really slow with big graphs

[hamzaaa1988]

Hi there,

I am currently working on a pretty big graph and I noticed that after some point using world machine got really slow and clunky and not that fun to work with. I cannot tell you exactly how many nodes are in this graph but if I had to guess it might be around 500 - which I think is not that much comparing to similar programs like houdini or substance designer.
Is that normal behaviour and I need to try to keep graphs smaller or might this be a (known) issue? Is there something I can try to improve performance? Thanks!

[WFab]

May not be a bug, seems like normal behavior.

Depends on how much “data” you have in memory. Of the 500 nodes, how many have data built at high resolution? What’s the state of your main memory during these hanged sessions? Is there some sort of preview window open? Lot of factors to consider. In general, world machine is a lot more efficient at managing graph memory, then say, Houdini. But when you start throwing more raw data then your memory can handle, you’re bound by your disk speed due to disk swapping. Check if your whole system is lagging, or just world world machine.

The question is, what kind of terrain requires you to use 500 memory heavy nodes? There’s something to be said about “creative problem solving” and “optimization” here. Work in “final product resolution” context. No matter how good the tool, in the end the artist is the bottleneck in most cases.

[blattacker]

Was in the middle of typing almost the exact same reply as @WFab. Just want to also add that, especially if you have 500 nodes that are all built at final resolution, comparing to a program like Houdini is pretty unrealistic. When comparing to a full 3D software, a built version of the project file is essentially like a final render stored in RAM. You wouldn’t expect Houdini to be able to have an entire complex scene rendered at final quality within the project viewport, realtime, and not have the entire system lock up, right?

But also, yeah, 500 devices in one project seems a little…off. I might be in the minority here, but I can’t say I’ve ever broken even 100 devices in a project that I can think of off the top of my head.

[hamzaaa1988]

I know the biggest problem is my setup and not world machine and normally I wouldn’t use such a huge graph. It’s just let’s say… it’s production and time constraints that kinda lead me here ^^ I am using pretty high resolution so RAM is maxed out so I already suspected this is a problem. The only thing that I find weird is that moving nodes, shifting slider, entering values etc become extremely slow, too. Do you think this is all due to RAM constraints?

[blattacker]

If your RAM is indeed maxed out, it’s likely that your entire system has become slow, not just World Machine. I’d say, if at all possible, try to split the project into multiple files by either building and exporting the current results at predetermined checkpoints and using a File Input device on a new project, or by using @WFab’s Library Import method. Since a lot of data is being paged to disk anyway, you may as well do it intentionally and free up some RAM.

[hamzaaa1988]

Yeah I suspected as much. I really like @WFab’s tutorial but I often end up needing all kinds of parts of my network so it’s kinda hard to split it up, but I will give it a try

[WFab]

@hamzaaa1988

The only thing that I find weird is that moving nodes, shifting slider, entering values etc become extremely slow, too. Do you think this is all due to RAM constraints?

@hamzaaa1988 Yep, either memory, or GPU fetching data from disk to display.

About splitting projects the way I do, you have to plan the “splitting” in advance. Projects are always built in stages, even if they are within a single graph. Once you get the hang of it, it becomes muscle memory. Very high resolution and tiled builds may break this system, but 8k-16k projects are easy to set up using libraries.

Think of it as similar to “Render>Compositing>Post processing” type of pipeline. Things have to be baked to spec before passing on to the next stage. And with above you have much more data passing through, terrains are quiet simple and linear in comparison.

[WFab]

@blattacker

@blattacker

I might be in the minority here, but I can’t say I’ve ever broken even 100 devices in a project that I can think of off the top of my head.

Building complex geological systems as efficiently as possible is the name of the game! It’s a good thing having less nodes and more organic details, so bravo! I have about 30-50 nodes in the terrain stage, maybe the same in material building and final output building and naming stages. Across 7-10 project files, maybe 150 max.

The recent island project I’m working on is my most ambitious yet, may have gone past that number though. And includes a few “Gaea stages” scattered in between. Those output files tend to add up when doing Gaea builds. Thanks to libraries, world machine data is organized cleaner, though still takes huge storage spaces lol!

[hamzaaa1988]

I reworked my graph a bit and tried to split it up a bit more and I am now using library nodes to cache some stages. I could reduce the memory footprint quite a bit and now WM seems more stable and smooth again. Of course this introduced more steps I need to do and be aware of (that’s why I would love this feature to happen). But in the end you helped me out folks, thanks!

[hamzaaa1988]

I also just realized that background preview building is heavily dragging down program performance so whenever I need to handle around with a lot of nodes and don’t necessarily need a preview, I just turn preview building off (bottom right corner)