Here’s my method of splitting projects to make complex workload manageable, using disk caching. May help with your particular case. I’m outlining a fixed resolution, single tile workflow here.
Tutorial for optimizing workloads using disk caches, using “Library I/O” nodes:
First things first, create a project folder where you’re going to save all your production data. Easier to name and save your project first (with final build resolutions etc), so you can set everything using relative paths. Next, plan “stages” for your terrain in advance, for example: Terrain building, material building, etc. (Depends entirely on your own workflow and taste, you can go as granular as you want). Once all that is done, you can start working in stages right away! Here are the steps.
-
Create your terrain in full. Once done, export it all to a “Library output” node. Set the path to somewhere within your root folder (name it something like “cache 1”, for easier identification). Now connect your terrain, water, and every data mask you want, to the library output (Tip: Skip “selector” based masks like “slope” and “height”, you can recreate that later with just the terrain height). Save your project with a descriptive name. Then build and write to a single library, all the data you want to carry forward to the next stage, that requires rebuilding heavy nodes (like water, erosion data, snow, etc). Make sure you use a single library per stage, makes organization easier. (Tip: You can use a “checkpoint node” to rename your data streams, though be aware of this ONE bug with names)
-
Now create a new project (Tip: To avoid recreating project settings, just copy paste your main terrain project, open and delete all nodes, and save it with a descriptive new name) for your next stage (for example “texturing”). Import your cache using a “library input” node. Make sure you use only this single library file for everything. If you need something from your terrain, only THEN you need to open and edit your terrain project. Otherwise, your data is is already baked at final resolution with this disk dump. Once your texture or material is created in full, you can preview it in final resolution using “scene view” node (Here’s an example, my own work). Once final, you can export another library with your final texturing masks, for the next stage. (Tip: Separate your “exports” into it’s own stage. Makes file format conversions easier).
-
Do the same with the new project (copy paste projects if needed, treat the libraries as individual built files, fixed resolution and scale). Export your files using appropriate outputs.
Once properly setup, you need to edit and open only the specific stage of your project. And you only need to rebuild the stages downstream (only the stages being affected by the edits you did). (Tip: Adding a serial number to TMDs and TMLs filenames helps with the rebuilding part). This way you can also cut down on memory usage, by saving and loading only the data that you need. Build time is also massively reduced, since you’re only rebuilding the parts that you need.
Few things that help cutting down on setup time for this workflow:
- Work at default scale, and extents. Set your final resolution in advance.
- Tiling complicates things significantly, use sparingly. And only when absolutely necessary. You of course need to adjust workflow accordingly.
- The more granularly you want to split your projects, more time you spend setting up your working project. Also, the libraries themselves take up a lot of storage space, so make sure you’re accounting for that in advance.
Hope that helps, at least until something smoother comes along!