The instance tiler does not actually allow you to change the rescaling type directly, although it certainly could and should.
You might be trying to adjus the resolution config and hoping that that option is applied to the device’s internal operations. The rescaler chosen there changes the processing resolution of the device independent of the inputs and rescales them using your chosen operator - it does not affect the operation within the device at all.
i can not confirm your words - because when i use custom re-scaling type (under “custom” i mean that one, that different from project default one) - i receive different results in Instance Tiling even without usage of custom resolution set:
if you have macro with Instance Tiling inside of it, you have three different re-scaling types that impact your result:
first one is the project settings one
second one is rescaling type chosed in Intance TIling
third one is rescalng type chosed in macro settings
So you have milion of combinations how your rescaling in Instance Tiling will look - i dont know math how calculate 3*(number of scaling types)=number of possible combinations