File input node takes up more memory than input file itself

Bug Reports
1

Upon importing a file using the file input node, I can see that the memory the node takes up is about 5.6 GB. The file itself that is being imported is around 12.1 MB. I am trying to combine multiple of these but it quickly becomes too much for my PC to handle as each file input node takes up such a huge chunk. Why is this happening/is there anything I can do to fix it?

2 Likes
2

That certainly doesn’t seem right. I would love to figure out what’s happening here. Can you provide any more details that might help - How are you determining the memory usage of the device? Does it stay at 5.6 or does it increase over time?

1 Like
3

i did also notice that a lot
but before developer answer above did think that it work as it should

the problem of File Input device, is that it uses not actual file size. But the number of memory that you could see in Project settings - “memory per each heightfield” or so

So actual memory that File Input use is limited by number from project settings. If your world is 16 000 resolution - be ready that you 128kb file will become 1024 mb RAM usage in WM

4

i did discover that while loading of 16kb textures of minecraft blocks into 4k project or so (texture is so little because you know that Minecraft is pixel game)

each of 16KB textures become 256MB file inputs after loading into project, lol

5

Is that what you mean?

For example:

You load a 16x16 pixel RGB texture from disk. That takes up 16x16x3 = 768 bytes on disk.

You have the file input device set to worldspace and build an 8k world.

  • The file input device has a copy of the original data transformed into WM’s internal format
  • It keeps its build results (as long as mem conserve is not turned on).

That means that the device consumes:

  • 16x16 * 6 bytes (converted to half float) = 1.5KB for the asset itself.

But, if not conserving memory, it stores its build results as well for you to inspect later:

The File Input device has 2 primary outputs, the data itself and the alpha mask. A build result will have data for both. It also will store multi-resolution results, so each step in the interactive preview schedule will be retained. For that 8k world, that probably means something like:

Results at:

  • 128px
  • 256px
  • 512px
  • 1024px
  • 2048px
  • 8192px (final)

The memory consumed for an RGB texture resultset will be 10 bytes per pixel, across the sum of all of the above pixels.

If that is what you mean, then this is intended and correct behavior.

If you click on the preview circle in the bottom right corner and click “purge all build results”, you should see the memory consumed by the file input device go down to JUST the asset itself.

1 Like