Using WM for serious simulation projects

Bug Reports
#1

When we’ve worked on our Valley Benchmark (http://forum.world-machine.com/index.php?topic=909.msg9556#msg9556), World Machine proved itself perfectly in the creation of a fully procedural terrain of relatively small size (8*8 kilometers).

Now we’re working on a new project, where we set the goal of creating much bigger terrain, based on the real setellite data (height maps, color maps and vector data).
In our case, terrain is 262262 kilometers in size, divided into 1616 square tiles with the resulting precision of 8 meters per pixel in heightmaps.
When working with this huge amount of source data in World Machine, we ran into the following problems:

When we use Tiled File Input device with small Extents - everything was ok.
But when we’re increasing Extents to the size of a whole terrain (262*262km), or switching to the Explorer View, World Machine hangs for a long time.
It seems like it’s trying to load all the source files to the memory for no substantial reason. Shouldn’t this data be streamed asynchronously?
Probably, it’s not a big deal, but for the tiled output we have to set Extents to the size of a whole terrain and wait until it finishes to load.
And this is getting even worse as the size of the source data increases and when we place multiple Tiled File Input devices in one World graph.

It turned out that the Layout Generator node works very poorly with SVG files:

  • it works VERY slow with big SVGs (around 100 000 points in dozens of path objects), and no Tiled Layout Generator node available
  • paths are polygonized inadequately (it makes holes in completely closed paths, where they shouldn’t appear)
  • it doesn’t work right with compound paths (that are intended for creation of “donut holes” in shapes), but there’s a workaround for that: two Layout Generator nodes and one Combiner node can be created
  • it can’t open SVGs saved with Global Mapper. Because of that we had to create a python script that converts SVG files saved with Global Mapper to SVG before they could be fed to World Machine
  • it can’t undo any actions performed in Layout Generator editor
  • there’s no way to set common Brush Properties for all shapes in one Layout Generator node. Acquire from Selected/Give to Selected can’t copy/paste these properties from one shape to all selected ones (it can be done only one at a time)
    Although, any vector graphics editor (Inkscape, Xara Photo & Graphic Designer, etc) can handle these tasks very quickly and without any problems.

Problem with coast erosion. The minimal value of 0.05 is too big for beaches given the scale of real-word geo data.
Real beaches have the height range of around 3 meters, but the minimum value of 0.05 gives us the height range of dozens of meters.

For the Blur device with the scale-independent checkbox set, the minimum value of 1 is too big for real-word geo data.
It’ll be ideal if we could set the blur radius in meters.

Minor problems we’ve encountered:

  • Copying device removes connected route points
  • There’s no option to set color values in the numerical format in Colorizer device

Solving this problems will help us a lot in creation of our new proj and we will be very grateful.

If these issues could be solved, that’d be a tremendous step forward for using WM in serious simulation projects. It would allow for creting terrains of extremely big scale that are based on real geographical data. WM is a really great tool that we as technical artists enjoy using, it just needs some features targeted to big-scale projects.

Thanks!

#2

Hi there,

Thanks for the feedback and list of issues! There have been strongly increasing numbers of users who are using World Machine to create huge terrain expanses (dozens-to-hundreds-of-GB datasets), and at that scale there are some problems that crop up.

Regarding #1, the TFI device is creating a cache pyramid of the input files so that only the higher resolution builds need to go to disk. This is very important for full-dataset views (bad enough to sit there and load many gigabytes once, you don’t want to do that every edit…), and although async loading would certainly be nice, the fact that the WM previews themselves are synchronous limits the utilty of this – you need that data before you can show anything. This is a known issue, and one of the most helpful things here would be to have the TFI device save its cache to disk in a session file so that you only have to suffer through the re-cache on the inital load or when the source data actually changes.

The minor bugs and small things you mentioned are small but important fixes – 2.3 is coming out imminently but I will see when they will get squeezed in. More long term, I would love to tackle all of the problems mentioned in your post. Once 2.3 is out I’m going to talk more about the future development path of WM, but in a nutshell I am hoping to do smaller releases more often with focused improvements – a “Huge Worlds” improvement set is definitely in the cards.

#3

Thanks for the answer, Stephen!

I think that implementing Tiled File Output device (similar to TFI device) with the ability to set the Extents and tiled parameters will be a solution in this case.
So we’ll not have to open whole Extents in a preview window and wait until it finishes caching, because using sources with the resolution of 1 meter per pixel in TFI device can take hours to perform this operation.

And I think that the option of setting Extents in Height Output devices may also be useful.
With this option we’ll have the ability to export several heightmaps with different resolutions and extents in one build.

#4