Tutorial Project: Caching your terrain to disk using Libraries

Since people keep asking about implementing a Caching device in world machine, I decided to make a demonstration of an alternative using Library i/o devices. The tutorial project works in the latest release build “Artist-Point Build 4031”

Use this thread to ask questions, if you have any. I’ll be editing the project itself to include this forum topic later on.

Library Caching Demo (WFab).zip (52.0 KB)

4 Likes

So if I understood it correctly this is a more convenient way of caching than let’s say writing your outputs to disk as maps and reimporting those into world machine again? Also it let’s you cache data that cannot be written to a map (e.g. water systems)?

Yes, and yes. This method is more convenient to set up, and maintain. It’s essentially a memory dump of all data streams plugged into it, so it can be imported into world machine as a single node package. This is especially useful for complex projects, where you need to build materials and exports in separate graphs (for better distribution of memory, both your PC and your own lol).

And yes, all composite data types are now compatible with libraries, so water and material, and even meshes can be saved as part of a library.

Whenever I’m doing client projects, or personal assets that are uploaded online once finished, it becomes a hassle backing up production data. For non trivial ones, I’m keeping just a single library with all baked data, ready to be adjusted in post if need arises.

1 Like

That sounds really cool. Would be nice if it could be automated, right, so your “trick” with activating/deactivating parts of the network and having to refresh library input nodes etc. would not be necessary :smiley:
When you deactivate a part of a network that has already built nodes in it, will world machine throw away the temp data or do you have to switch resolutions/restart wm to free up the RAM?

1 Like

@hamzaaa1988 Automating “disable” operation, and automatic caching alternative, I’ve made threads about both in the past. Show those topics some love if you want them lol!

Checkpoint caching : Feature request: Checkpoint node disk caching
Disable bit: Add disable/bypass parameter ports to devices - World Machine

@hamzaaa1988 About wm throwing data though.

No, since the data is cached to disk, you can safely disable the “library/cache saving branch” without losing that data on disk. All wm data however that is in memory, is dumped. So RAM is freed somewhat, though new data is written to the memory, with the “disk cache” that’s loaded with the next branch. You do gain loads of memory back, since not all outputs are saved with the library usually. In a nutshell, you only have to keep the data you’re using currently, rest is dumped when you disable a branch.

That said, MAKE SURE you disable the “library output” once its built, along with all nodes that came before. I’ve noticed that outputs save placeholder data even if nothing is connected to them, so your cache may get corrupted on build if it remains enabled, rendering the next branch blank since the source library is overwritten with blank data…

This is why the “disable parameter suggestion” exists in the first place.

1 Like