Crash using any 8k world ?

[Anonymous]

Mmm! Never tried mem conservation before so I thought I'd have a go.

So using the default nodes and a bit of tweaking in normal mem mode produced this....




Incidently, after pressing the OK on this pop up, I once again saw a flash of this again..



before the 3D view appeared. Which I successfully saved to my Terragen folder.  




I then logged off, reopened and changed the mem to this...


 
which I assume is the mem conservation mode Remnant was referring to.  Which resulted in clearly a reduction in memory used


 
compared to the previous build.  However when I tried to write the file to my Terragen folder this pop appeared....



I then loaded the file I first produced back into World Machine and did a normal build

 
 
successfully saving the result to my Terragen folder.

I then repeated the build but altered the mem conservation...




Which unfortunately resulted in..



once again.

Don't know if this helps at all Stephen. But best of luck.

[RedSquare]

Erm! the last post was me  

Which unfortunately resulted in..



once again.

Don't know if this helps at all Stephen. But best of luck.

OH! I forgot the terrain size was 4096x4096.  

[Fil]

The error saving file in Mem Concervation is a known issue. Meanwhile, the only way to save a file with File Output device in Mem. Conc. is using the "write every build" option.

As for the wierd looking screen, I remember having that in my laptop, each time I would use the OpenGl preview after my computer had gone to Stand By or Hibernation. It is something to do with the graphics card drivers, but since no update was available for my graphics card, I have to use WM on a fresh start of XP..

Well as for the 8Kx8K crashing part, at least that one didn't happen in Memory Concervation Mode, right?

[RedSquare]

Fil -

Well as for the 8Kx8K crashing part, at least that one didn't happen in Memory Concervation Mode, right?

Whoops I didn't try that as you can see, if you look above I was editing in the world size whilst you were posting.  But I will have a go at that and let you know, promise no more screen grabs

[Fil]

promise no more screen grabs

  oh-oh!...

[Anonymous]

This is definitely a memory allocation issue -- or more accurately, an out-of-memory issue happening.

Correct - but your app really has to check return values.

Any O/S call can result in a failure - every call needs to be error checked.  (one big allocation is better than many small allocations).

I have a hunch about the problem - are you ever resizing a memory block?

Maybe it's a memory fragmentation issue.

Do you allocate each line for a node or the entire heightmap in one big block?  If it's the latter, that might explain why VM didn't seem to work, and why it crashes sometimes and not others, depending on RAM fragmentation.

The thread has had various things discussed in it; to clarify does it also do this with Mem Conservation switched on?

Yes, atlhough I think some of the crashes also happened when I thought it was on but it was not.

World Machine is an application that would definitely benefit from the transition to 64bit windows. 1.1
Should bring some better error handling and detection of memory-wary areas, but doesn't fix the fundamental problem -- that WM really needs more address space for large nets at high res.

Well true, but an application should NEVER hard crash.  You've got to check the HR / Return codes of EVERYTHING.  

Its sort of a moot point, it looks like even a small erosion of even 8k x 4k takes days ...

I'm doing anyother 4k run and it seemed to work fine - the same world earlier crashed with the 1.3G available.  So I wonder if it's the memory fragmentation?  Did you hunt down where you are doing a resize?

== John ==

[Stephen]

Hi John,

WM never resizes a memory block. All HF memory is allocated once per datapacket and is handled through one set of core functions which include the call to check the size of the memory block being malloc'd/free'd.

1.1 will have better exception handling to cover the case of running out of memory, but this is only a partial solution -- WM won't crash, but in the low memory case the build will still abort. Memory Conservation mode should help greatly with this, which is why I was wondering specifically if your networks were still running out of ram with it on. That's still quite possible but hopefully far less common. (with mem conserve on, at any instant WM needs memory equal to the number of output ports at a given "levelset" in the network.) The Erosion outputs alone will take up a 1gb of RAM at 8192.

Regarding erosion speed; I've eroded a 8192x8192 terrain in an acceptable amount of time, even with some serious disk-swapping going on (I have only 1gb of RAM). Have you tried unchecking the "low priority build thread" option in Preferences? I've found that in some cases where people are finding erosion very slow, it's because something else is (usually needlessly) taking all of the CPU time, and WM defaults to being too nice with its priority. Erosion Simulation is a slow process, but its not THAT slow. Even at full res it shouldn't be taking dozens of hours on a fast computer. I'll look into it further.

[jgwinner]

Crashed again - let me know where to upload the file  

Preferences set to 'Keep only output build results (full conservation), fresh boot, only a 4NT session (cmd) running.

463M of physical RAM avail (out of 2G), 2,800M (2.8G) Swap available, i.e. the page file was less than 1/2.

Here's the code trace:

>   [email protected]()  + 0x1d   
    World Machine.exe!0043dc0e()    
    World Machine.exe!00417fe3()    
    World Machine.exe!00431688()    
    World Machine.exe!004315e8()    
    World Machine.exe!0043163d()    
    CustomDev.dll!02efc8c1()    
    CustomDev.dll!02f12252()    
    World Machine.exe!00433c25()    
    World Machine.exe!0041c59d()    
    World Machine.exe!00469c91()    
    World Machine.exe!0043f12d()    
    [email protected]()  + 0x37   

RtlSizeHeap is the retail (release) version of HeapSize()

HeapSize

The HeapSize function retrieves the size of a memory block allocated from a heap by the HeapAlloc or HeapReAlloc function.

If you turn on debugging symbols, you can map that addess roughly to where the loc is in a release build.  Debug symbols only produce a symbol table, it's still a release build (so optimization may move around lines).

The April 98 MSJ has a great article on this - lets you find source for a release build crash.

http://www.microsoft.com/msj/0498/bugslayer0498.aspx

On the speed issue, WM had 99% of the CPU most of the time, it was the only thing running.  It may be terrain related.  I sent you the world, but the .ter that is the input is 33Meg - anywhere I could put it?  I'll try zipping it and seeing how big that is.

  == John ==

[jgwinner]

Ok, I got a different addr when running WM from the debugger:

The thread '[email protected]' (0x958) has exited with code 0 (0x0).
The thread '[email protected]' (0xdbc) has exited with code 0 (0x0).
First-chance exception at 0x7c96cc82 (ntdll.dll) in World Machine.exe: 0xC0000005: Access violation reading location 0xffffffff.
First-chance exception at 0x004315fd in World Machine.exe: 0xC0000005: Access violation writing location 0x00000000.
Unhandled exception at 0x004315fd in World Machine.exe: 0xC0000005: Access violation writing location 0x00000000.

Actually - something weird is going on here - if you take the world I just emailed you Stephen, then delete the file input, then copy the perlin, then connect the perlin to the splitter, it'll crash WM.

Weird, huh?  If you create a new perlin and connect it to the splitter, no problem, only if you copy and paste (was trying to avoid emailing the 33Meg input terrain).

== John ==

[jgwinner]

I emailed Stephen my world and he suggested I try the newer erosion device.  I've done that, and I tried going to a MUCH simpler world.  I have a file input (from a .ter), going to an erosion device, then to an output.  I also have a mask (.BMP) input going into the erosion device.

THis is on 'full conservation' mode, all other recommendations done.  WM was the only program running other than the Task Manager and background services, and notepad.

I still get a crash in exactly the same place.  At the time of the crash, I had slightly over a Gig of available physical RAM (WM was taking 650M and the O/S about 300), and 2.8G of swap space available.  WM crashed in a call to SizeHeap again (probably called from a CRT lib routine as Stephen isn't calling it directly).

Doing a 4k build works fine (and is faster with the new erosion).

Leveller doesnt' have any problems with 2-3 8k worlds up on the same machine, adn doing erosion.  I don't like the Leveller erosion (HF_LAB) as much, however.  So my work around is to do the work in WM at 4k, then import and do a final erosion pass in 8k in Leveller.

Stephen, if you have an NDA I'll run the source through Boundschecker if you want to track this down

      == John ==

[Stephen]

Hi John,

I've found and fixed several bugs for v1.1, including adding exception handling for the out-of-memory case to prevent crashes, and the perlin bug you ran into earlier in this thread.

Later this week I'm going to buy another gig of RAM so that I can do more thorough large-network testing (testing 8192 sizes with only 1gb of ram is an exercise in swapfile frustration).

[Anonymous]

Great!

I'll Alpha test it if you want.

I still haven't been able to get 'any' 8k world working, swap file or no. It's weird that you can with less RAM.

I have 2 more Gig on the way.

== John ==

[jgwinner]

Great!
I have 2 more Gig on the way.

Doh! It didn't help.    I still can't do even a simple erosion with an 8k world, with a mask.  It blows up in the same spot.  When it blew up I had 1981052 physical RAM available, and had used 1244784 out of 5076040 of the page file.  (I haven't made my swap file really big).

I recreated everything using new nodes from the menu.

I really don't understand this.  Why does it work on systems with less physical ram?

== John ==

[jgwinner]

Well, 1.25 doesn't crash, but it does run out of memory.  So there is still a problem, it just doesn't crash.

From O/S:
Physical Memory available: 1.6Gig.
VM available: 2.75Gig.

Devices visited: 2
Devices processed: 2
Heightfields active: 11
Full conservation: On
Memory used: 832.4Mb

the .tmd consists of:
Input: 4k .TER (Terragen) file
Inverter
Erosion
Output


So then I tried

Input: 4k .TER (Terragen) file
Inverter
Output

and THAT won't work.  

This is very frustrating.  So apparently WM has an issue with .TER's.

Yes, I double checked full conservation it's on.

Any advice?  I don't want to use a TGA format because I will lose precision.  I messed around with RAW files and couldn't find a common set of parameters.  I have Leveller so I can convert to a variety of formats if necessary.

[jgwinner]

Ha!

It was the /3G switch - at least, setting that seemed to help.  Interestingly, I have no more RAM available than I did before, but at least I got a simple 4 node network to generate.  I'll see if it finishes.

Interestingly, WM never went past the 2G limit now, either - it's at about 1.G of VM currently.  1G of physical available and 2.4G of Commit charge left.

    == John ==

[RedSquare]

It was the /3G switch - at least, setting that seemed to help.

I'm probably being even thicker than I normally am. But, what is a 3G switch used for.  Or are we looking at fat fingers?     Ah! you must be talking about RAM settings.  

[Fil]

Actually I was thinking of mobile phones!... But no, it's a character sequence that you put in some specific wierd place (boot.ini file), to tell Windows to boot with a different memory setting the next time..

It's described here:
in this thread about memory issues with large builds..