Author Topic: Scaling up detail levels of DEM data  (Read 13622 times)

0 Members and 1 Guest are viewing this topic.

Alexander

  • WM Pro Beta Tester
  • World Machine Veteran
  • ***
  • Posts: 25
    • View Profile
Scaling up detail levels of DEM data
« on: May 08, 2007, 09:58:51 AM »
I am currently experimenting with some alternative ways of creating terrain and I had a question for the community.

Let's say I have a piece of data at 10-30m resolution and I want it to be closer to 1m resolution. What would be be the quickest or the best way to upscale this detail based on that DEM data? I've tried adding noise, combining with perlin noise, and using erosion. Any other thoughts?


nikita

  • Testing Team
  • World Machine Guru
  • *****
  • Posts: 627
  • B.S. in Macro Science
    • View Profile
    • My WM Macros
Re: Scaling up detail levels of DEM data
« Reply #1 on: May 08, 2007, 10:45:31 AM »
I can't think of any other ways to add detail.
Maybe use different perlins for different heights?
"Every time you read or write beyond the end of an array, somewhere a fairy dies."
Richard Buckland

Stephen

  • Author
  • World Machine Guru
  • *****
  • Posts: 2320
    • View Profile
    • http://www.world-machine.com/
Re: Scaling up detail levels of DEM data
« Reply #2 on: May 08, 2007, 10:59:56 AM »
From 10-30m to 1m is a huge jump.. adding noise, combine with a perlin, displacement (distortion), and erosion are all good tools.

(Another one might be to try using the Adv. Perlin device, with the low res data set as the guide channel, and then adjust the scale/persistence until you get something of the appropriate scale extra detaill.)
-- Stephen Schmitt
- Creator of World Machine

monkschain

  • World Machine Guru
  • ****
  • Posts: 707
    • View Profile
Re: Scaling up detail levels of DEM data
« Reply #3 on: May 08, 2007, 11:16:52 AM »
 I think it's probably pretty tricky to do in practice. I think adding individual noise octaves based on the terrain. Say you have a part of the terrain that is rolling hills, you could generally get away with adding the large to mid octaves. If the terrain was very detailed rocks over flat terrain, the you would want to add the smaller octaves only. I reckon you could simplify things if you broke the terrain down into say 3 noise levels (low, mid, high), and added those with care across the terrain- ie via selections.
 'Scale and add' is the usual way to do this procedurally I think- so I heard. Procedurals has got to be the best way. Another way might be to sample the terrain via selections and get fourier signatures corresponding to terrain types: hills, mountains, lowlands, etc, and then scale those to the new terrain? Would the Spectral Selector/Combiner or Advanced Perlin help with this?
 Either way, I don't think a 'one size scaling of noise fits all' is going to work.
Maybe a frequency sampler device that could sample terrain by selected areas (you would set up the selection areas by terrain type). Each device would have it's output settings set differently so as to correctly upscale that noise signature. You could then see presets of different signatures coming along with the device- like macros- 'hills', 'beach', etc.

-just read Stephen's post and it looks the advanced perlin could help. :)

monks

Alexander

  • WM Pro Beta Tester
  • World Machine Veteran
  • ***
  • Posts: 25
    • View Profile
Re: Scaling up detail levels of DEM data
« Reply #4 on: May 08, 2007, 12:16:55 PM »
Wow, that was way better and quicker input than I expected. So far in my tests using the DEM as a shape guide for an advanced perlin is giving some great results. I'll post some results perhaps once I get something a bit more refined.

DGUnreal

  • World Machine Veteran
  • ***
  • Posts: 27
  • IT Pro / Pro Level Designer / Programmer
    • View Profile
    • http://www.lilchips.com/dgunreal/
Re: Scaling up detail levels of DEM data
« Reply #5 on: May 22, 2007, 09:24:19 AM »
Alexander, I would be interested to see comparative screenshots (raw DEM vs enhanced).
I have tried increasing DEM "quality" and almost always lost too much original DEM shape.
DGUnreal Lead Designer and Engineer
Game Designer/Programmer since 1982

pthomas1172

  • WM Pro Beta Tester
  • Community Member
  • **
  • Posts: 5
    • View Profile
    • Ensemble Studios
Re: Scaling up detail levels of DEM data
« Reply #6 on: May 25, 2007, 11:13:24 AM »
It's not possible. Its like trying to unblur a compressed jpg.

I have been makeing terrain in Games for 15 years and have tried every angle.
I have tried using DEM data and adding noise, it helps in SOME cases but isnt a fix all.

The best result I have ever seen is takeing large peices and makeing a collage to represent a much smaller peice. per say using the entire grand canyon as a simple river .25 miles by 2 miles.

WM is hands down our best hope for sucsess. it has taken the whole process from months to days, even hours (sometimes)


My travels...
http://www.mobygames.com/developer/sheet/view/developerId,20679/
Patrick Thomas
http://www.ensemblestudios.com/
Terrain Dude.

DGUnreal

  • World Machine Veteran
  • ***
  • Posts: 27
  • IT Pro / Pro Level Designer / Programmer
    • View Profile
    • http://www.lilchips.com/dgunreal/
Re: Scaling up detail levels of DEM data
« Reply #7 on: May 25, 2007, 11:49:42 AM »
It's not possible. Its like trying to unblur a compressed jpg.

But I've watched CSI and they can do things like add 100x image enhancement to blurred images to see anything they want for detail...   :wink:
Just kidding...

I do agree with you that attempting to do this in WM (or any other software that I have tried to-date) results in mediocre results at best.
As I mentioned in my post, in the past I have tried using external source files with WM and adding various combinations of devices, and it usually changes the underlying source heightmap excessively.

I am developing a non-public game-engine-oriented heightmap tool for Epic/UE Licensees (and eventually any game studio that wishes to license it) that will be using a technique that can non-destructively increase the "resolution detail" of a heightmap.  Along with a massive list of other current and proposed features.
DGUnreal Lead Designer and Engineer
Game Designer/Programmer since 1982

nikita

  • Testing Team
  • World Machine Guru
  • *****
  • Posts: 627
  • B.S. in Macro Science
    • View Profile
    • My WM Macros
Re: Scaling up detail levels of DEM data
« Reply #8 on: May 25, 2007, 12:22:03 PM »
I do agree with you that attempting to do this in WM (or any other software that I have tried to-date) results in mediocre results at best.
As I mentioned in my post, in the past I have tried using external source files with WM and adding various combinations of devices, and it usually changes the underlying source heightmap excessively.

No, it doesn't, unless you tell wm to do so :)
"Every time you read or write beyond the end of an array, somewhere a fairy dies."
Richard Buckland

DGUnreal

  • World Machine Veteran
  • ***
  • Posts: 27
  • IT Pro / Pro Level Designer / Programmer
    • View Profile
    • http://www.lilchips.com/dgunreal/
Re: Scaling up detail levels of DEM data
« Reply #9 on: May 25, 2007, 12:47:49 PM »
I would be interested in seeing an example.   :)

For example, I have used a medium resolution source file (DEM, art dept, etc. provided file), such as 256x256 or 512x512, for which I have wanted to add intermediary detail to increase the apparent resolution to perhaps 1024x1024 or 2048x2048, without modifying or destroying any of the source sample points.
I have been unable to do this with WM.

I would perhaps agree that for simple visualization (ie. straight renderings in WM, Max, etc.) the tools in WM might suffice.
This is not what I am using it for however, and I believe both Alexander and Mr.Thomas (if I can make an assumption) are also using this for video game design.
For example, if I have a terrain that is being used for a video game level where the specific current layout of the source file cannot change because it relates to map asset locations, but I wish to implement what is essentially sub-sampled or tesselated detail enhancement, I have never been able to get WM (or other similar software) to do this.
In my software I have a planned method of actually achieving this that I know will work.  Please don't think that I am trying to promote or otherwise recruit people to my software, as it is not for public use.  Nor am I asking WM to change their current software to include such a feature.

If you can do this in WM, I would be interested to know how.   :)
« Last Edit: May 25, 2007, 12:54:22 PM by DGUnreal »
DGUnreal Lead Designer and Engineer
Game Designer/Programmer since 1982

nikita

  • Testing Team
  • World Machine Guru
  • *****
  • Posts: 627
  • B.S. in Macro Science
    • View Profile
    • My WM Macros
Re: Scaling up detail levels of DEM data
« Reply #10 on: May 25, 2007, 02:52:59 PM »
I see. OK, that's a bit much more difficult ;D
I did some experimenting, but the method I thought would work, actually didn't. My idea was to create a mask that covered the whole terrain but the original pixels. That way, only the interpolated pixels are processed.
"Every time you read or write beyond the end of an array, somewhere a fairy dies."
Richard Buckland

DGUnreal

  • World Machine Veteran
  • ***
  • Posts: 27
  • IT Pro / Pro Level Designer / Programmer
    • View Profile
    • http://www.lilchips.com/dgunreal/
Re: Scaling up detail levels of DEM data
« Reply #11 on: May 25, 2007, 05:16:12 PM »
Thanks for trying.  :)
Yes, I have spent many days messing around with various techniques in WM to achieve a "resolution enhancement".
I have a couple of times just used some low-level added noise along with the height/slope masks and was able to get some return for my effort, but not as much as I had hoped.
WM is an awesome piece of software, but like any single solution it doesn't do absolutely everything that all people may want, that would be a lot of work to program that level of functionality into it without a large team of programmers.   :)
DGUnreal Lead Designer and Engineer
Game Designer/Programmer since 1982

nikita

  • Testing Team
  • World Machine Guru
  • *****
  • Posts: 627
  • B.S. in Macro Science
    • View Profile
    • My WM Macros
Re: Scaling up detail levels of DEM data
« Reply #12 on: May 25, 2007, 05:53:47 PM »
WM is an awesome piece of software, but like any single solution it doesn't do absolutely everything that all people may want
Probably, but WM 2.0 will. :) (to some extent)
"Every time you read or write beyond the end of an array, somewhere a fairy dies."
Richard Buckland

Alexander

  • WM Pro Beta Tester
  • World Machine Veteran
  • ***
  • Posts: 25
    • View Profile
Re: Scaling up detail levels of DEM data
« Reply #13 on: May 26, 2007, 02:07:40 AM »
DGUnreal, in my tests I was able to get results that looked decent at the sacrifice of real-world scale, as pthomas1172 pointed out. Also, after reapplying erosion much of the original character of the DEM file was lost. This is still a very interesting problem for my team and I, so I'm interested to see what others have been able to do.

I don't have side-by-side renders atm, but here is a very coarse 30m heightmap that has been increased frrom 1024x1024 to 2048x2048 by feeding the heightmap into the shape input on the advanced perlin noise and then applying erosion:



In some later tests I used higher detail STRM data of the US and the results were better but by no means accurate. What I would be interested to see is a workflow where we take coarse dem data, and run more advanced natural filters on it to produce an alternative but consistent and more detailed version.

latego

  • WM Pro Beta Tester
  • World Machine Guru
  • ****
  • Posts: 70
    • View Profile
Re: Scaling up detail levels of DEM data
« Reply #14 on: May 26, 2007, 04:19:06 AM »
You might have a look at fractal interpolation. I remember an article I saw at least 10 years ago from 2 japanaese who first created an estimation of the fractal dimension of the terrain (as a map of values) and then used it to run a few steps of midpoint subdivision algorithm which interpolated the existing heightfield values. The results were really good.

Sorry for not being able to give you better references, but I long lost the paper photocopies :-(.

Bye!!!

Stephen

  • Author
  • World Machine Guru
  • *****
  • Posts: 2320
    • View Profile
    • http://www.world-machine.com/
Re: Scaling up detail levels of DEM data
« Reply #15 on: May 26, 2007, 12:41:51 PM »
There are a variety of tools that do this (fractal interpolation), however I've always been somewhat dissatisfied with the results. Fractal interpolation can work well on the small scale (adding another octave or two of noise as detail), and on the very large (taking a small guide map and turning it into terrain). However, in the "moderate" expansion regime, where you have terrain features you want to have stay recognizable, just with "more detail", it tends to fall flat, either not doing enough or modifying the terrain beyond recognition.

It's tough because in that regime we essentially want the magic bullet: Keep all of the existing detail and add more detail that matches the existing. The second part of that statement is the tough one. :)
-- Stephen Schmitt
- Creator of World Machine

latego

  • WM Pro Beta Tester
  • World Machine Guru
  • ****
  • Posts: 70
    • View Profile
Re: Scaling up detail levels of DEM data
« Reply #16 on: May 26, 2007, 01:24:36 PM »
There are a variety of tools that do this (fractal interpolation), however I've always been somewhat dissatisfied with the results. Fractal interpolation can work well on the small scale (adding another octave or two of noise as detail), and on the very large (taking a small guide map and turning it into terrain). However, in the "moderate" expansion regime, where you have terrain features you want to have stay recognizable, just with "more detail", it tends to fall flat, either not doing enough or modifying the terrain beyond recognition.

It's tough because in that regime we essentially want the magic bullet: Keep all of the existing detail and add more detail that matches the existing. The second part of that statement is the tough one. :)

In fact in the article the task was to add tiny details to preexisting DEMs (last 2/3 octaves) and there was very detailed preprocessing phase in which for each "cell" to be filled the local fractal dimension of the terrain was painstakingly computed.

I remember that one of the examples was a volcano caldera and the in flat area kept being flat and the rugged border were rugged at all the levels of detail; the border/bottom transition remained sharp.

I think this technique would be a great way to enhance a 512-ish real terrain DEM into a 2048 heightfield (3 octaves) and nothing more challenging.

By the way: what about adding DEM read/write (no fancy georeferencing stuff, just height data)?

Bye!!!