I’ve been meaning to clarify some things about disk caching, especially working within heightmapping context most terrain tools work with. There have been some misconceptions going on about memory saving and disk caching. Guess this is as good a place as any, also applies to my other feature request about checkpoint caching.
Disk caching is NOT a MEMORY SAVING feature (by itself), it’s a “Time saving” feature, sometimes at the cost of additional memory usage. Especially when talking about terrain mapping tools like World Machine and Gaea.
Let me explain:
There are two different ways disk caching works for heightmap data. “Passive” and “Active”.
Passive
Passive baking is already well built into world machine. This works by decoupling “disc caching” process into a separate node (Library I/O), passively caching or loading data from disk, from and to your main memory. When you save a cache, it essentially dumps your memory to a file on disk. When you load a cache, that dump is again placed back into your memory. So essentially, by itself, it doesn’t save any memory while working within a single graph. The method I described in this tutorial project, shows this problem clearly.
Passive caching however, can be used to split your work into parts, such that only the part you are working on gets loaded into memory. THAT is how memory can be saved, by splitting your graph into a separate projects.
Active
Active Disk caching is exactly what both our feature requests describe (checkpoint, per node, and group caching, to disk). Here the caching and loading process is built into every node. When you cache a node, it’s built to full project resolution, and baked into a memory dump on disk (in case of WM, tml files). This why, when you’re done for the night, you can cache your project, and resume the next day from that memory dump. No more rebuilding your massive projects every time you save and reload a project, thus SAVING TIME. The next time you open a project however, that cache has to be placed back into memory, so that your data can be edited further. NO MEMORY IS SAVED in the process, and unless you find a creative way to split projects, mixing active caching with passive loading, there’s no direct way it’ll save any in the future.
World Machine already has Passive Disk Caching well integrated in the form of “Library input” and “Library output” nodes. Quadspinner Gaea has Active Disk Caching well built in the form of “per node” baking of graphs into a “resource” file on disk.
Both work very well at this point, well integrated to save on “build time”. On their own, both features are NOT designed to SAVE MEMORY
That out of the way, +1 for the second part of your request, quoted below. I don’t know if I’ll be using it frequently, but seems like a natural progression for memory conservation as it stands currently.