Ok, so I come to conclusion that TWRP for Probox2 is incompatible with Nougat.
What I tried was built Android 7 from sources and packed TWRP as recovery.img into update.img using provided instruction. Then I interrupted Khadas boot via USB to serial prompt and executed
Unfortunately it stopped with exactly the same output as when I was trying to run recovery.img from SD card or USB stick.
So it is most likely not the way how TWRP is executed, but the built TWRP itself - I assume it is not compatible with Nougat.
@Gouwa @Terry @balbes150 Any ideas or tips on building TWRP for Khadas VIM Nougat? 