2024-03-17T23:00:00Z
In working on a large scale (50000x30000) world, I cannot successfully build the terrain, even as a tiled build.
I am currently attempting to build each node one-by-one, writing that output to disk, and then feeding it back into my project for the next node.
This has worked for the first few nodes which were all shapes devices. However the build consistently fails every time upon reaching my erosion node. There are only two devices that need to build, the file input (which contained the result of all the previous nodes), and erosion.
I first attempted to build the entire world at once, which gave me a fatal memory allocation failure, despite having well over 300GB in storage available for paging to disk. I then tried to perform a tiled build, composed of ~8k resolution tiles, but even then, there was the same failure. The build progress showed multiple tiles successfully building, but eventually one tile would hang and stop nearly instantly, showing a fatal memory allocation failure in the dialog. The tile that failed was not a specific one, it would be random.
I can’t understand why this would be happening. I saw a similar post about these issues that was supposedly solved in an update. However, I am running the most up-to-date version for the time being, and the issue persists.
My specs:
WM Professional edition build 4031.2
16GB RAM
454GB disk space available (for paging)
RTX 3070 Laptop GPU
AMD Ryzen 7 5800H
Here is a screenshot of the current setup, all devices disabled other than the file input, erosion, and height output. Neither a single build nor a tiled build would succeed.
Just checking, but you do know that WM swaps to disk using its own Temp folder, not the windows page file, correct?
Tiling takes up a lot of memory, and by default wm swaps to your C drive, and gets incredibly sluggish when it swaps. I have 128GB RAM on my workstation, and a dedicated 512GB NVME ssd for swap. Tiled terrains are still a marathon of patience for me. Adjust expectation accordingly.
BTW, WM hangs a lot while swapping to disk. So if it hangs on a node, just leave your PC alone (if you have enough space on your C drive).
It’s also a good practice to keep checking your disk space during that build. Once WM crashes, those swap files are deleted, so if you’re not actively checking you won’t know if it crashed cause your disk was full, or if it’s something else.
World Machine doesn’t crash, the build fails because I see the “fatal memory allocation failure” in the dialog. I am also aware that the paging goes to a separate temp folder, which I have already tried moving to my computer’s other drive with even more space. Still have the issue, though.
Alright building a project based on best guesses. 50km/30km, 8k tiles. Trying 4x4 tiles first, then going to do 7^2 as per screenshot. So far memory usage is already past my 128gb. Will make an edit after build completes or fails.
Update: Tiled Build completed successfully in my case, took a while. Files generated successfully. That said, found another bug in the process. Adding a tiled file input to the graph causes an immediate crash. Sent a bug report already about it, but I cannot test the tiles due to that bug.
Seconding this. If I recall correctly, merging the tiles to a single file requires that the project fit into available physical memory, I don’t believe it can page. If that option is enabled, try disabling it and reporting back!
This was already unchecked. It would only perform the build if I reduced the “items at a time” setting in build options to 1. Now that I have done that, there is another problem. The Tiled File Input took 2 DAYS to build when I reimported it. 2 REAL LIFE DAYS.
