Can't boot from USB HDD unless I do "usb reset" in U-Boot first

Hello, I can’t seem to boot from a USB3 HDD (flashed with the official Ubuntu Server 5.9 mainline image) unless I enter U-Boot and run the following:

usb reset
devnum=0
run usb_boot

Then it boots Ubuntu off the USB drive normally, so this fixes it for some reason. However, if I skip usb reset, and just do devnum=0; run usb_boot, it says:

Device 0: unknown device

usb reset makes it recognize Device 0 as my HDD, and then it can boot.

Here’s the boot log if I do nothing, it just keeps trying for TFTP:

G12B:BL:6e7c85:2a3b91;FEAT:E0F83180:402000;POC:F;RCY:0;EMMC:0;READ:0;0.
bl2_stage_init 0x01
bl2_stage_init 0x81
hw id: 0x0000 - pwm id 0x01
bl2_stage_init 0xc1
bl2_stage_init 0x02

L0:00000000
L1:20000703
L2:00008067
L3:14000000
B2:00402000
B1:e0f83180

TE: 153449

BL2 Built : 15:22:05, Aug 28 2019. g12b g1bf2b53 - luan.yuan@droid15-sz

Board ID = 8
Set A53 clk to 24M
Set A73 clk to 24M
Set clk81 to 24M
A53 clk: 1200 MHz
A73 clk: 1200 MHz
CLK81: 166.6M
smccc: 00029f47
eMMC boot @ 0
sw8 s
DDR driver_vesion: LPDDR4_PHY_V_0_1_18 build time: Aug 28 2019 15:22:01
board id: 8
Load FIP HDR from eMMC, src: 0x00010200, des: 0xfffd0000, size: 0x00004000, part: 0
fw parse done
Load ddrfw from eMMC, src: 0x00060200, des: 0xfffd0000, size: 0x0000c000, part: 0
Load ddrfw from eMMC, src: 0x00038200, des: 0xfffd0000, size: 0x00004000, part: 0
PIEI prepare done
fastboot data load
00000000
emmc switch 1 ok
00000000
emmc switch 2 ok
fastboot data verify
verify result: 265
Cfg max: 4, cur: 1. Board id: 255. Force loop cfg
LPDDR4 probe
ddr clk to 1608MHz
Load ddrfw from eMMC, src: 0x0003c200, des: 0xfffd0000, size: 0x0000c000, part: 0
00000000
emmc switch 0 ok

dmc_version 0001
Check phy result
INFO : End of CA training
INFO : End of initialization
INFO : Training has run successfully!
Check phy result
INFO : End of initialization
INFO : End of read enable training
INFO : End of fine write leveling
INFO : End of Write leveling coarse delay
INFO : Training has run successfully!
Check phy result
INFO : End of initialization
INFO : End of read dq deskew training
INFO : End of MPR read delay center optimization
INFO : End of write delay center optimization
INFO : End of read delay center optimization
INFO : End of max read latency training
INFO : Training has run successfully!
1D training succeed
Load ddrfw from eMMC, src: 0x00048200, des: 0xfffd0000, size: 0x0000c000, part: 0
Check phy result
INFO : End of initialization
INFO : End of 2D read delay Voltage center optimization
INFO : End of 2D read delay Voltage center optimization
INFO : End of 2D write delay Voltage center optimization
INFO : End of 2D write delay Voltage center optimization
INFO : Training has run successfully!

channel==0
RxClkDly_Margin_A0==67 ps 7
TxDqDly_Margin_A0==106 ps 11
RxClkDly_Margin_A1==97 ps 10
TxDqDly_Margin_A1==106 ps 11
TrainedVREFDQ_A0==26
TrainedVREFDQ_A1==26
VrefDac_Margin_A0==29
DeviceVref_Margin_A0==26
VrefDac_Margin_A1==27
DeviceVref_Margin_A1==26


channel==1
RxClkDly_Margin_A0==97 ps 10
TxDqDly_Margin_A0==106 ps 11
RxClkDly_Margin_A1==97 ps 10
TxDqDly_Margin_A1==106 ps 11
TrainedVREFDQ_A0==26
TrainedVREFDQ_A1==26
VrefDac_Margin_A0==29
DeviceVref_Margin_A0==25
VrefDac_Margin_A1==28
DeviceVref_Margin_A1==25

 dwc_ddrphy_apb_wr((0<<20)|(2<<16)|(0<<12)|(0xb0):0004 

soc_vref_reg_value 0x 00000025 00000025 00000027 00000026 00000027 00000027 00000027 00000025 00000027 00000027 00000025 00000025 00000026 00000025 00000026 00000025 00000027 00000024 00000025 00000027 00000025 00000026 00000025 00000022 00000028 00000026 00000027 00000027 00000026 00000026 00000026 00000026 dram_vref_reg_value 0x 00000013
2D training succeed
aml_ddr_fw_vesion: LPDDR4_PHY_V_0_1_18 build time: Aug 28 2019 13:54:19
auto size-- 65535DDR cs0 size: 2048MB
DDR cs1 size: 2048MB
DMC_DDR_CTRL: 00e00024DDR size: 3928MB
cs0 DataBus test pass
cs1 DataBus test pass
cs0 AddrBus test pass
cs1 AddrBus test pass

100bdlr_step_size ps== 431
result report
boot times 0Enable ddr reg access
00000000
emmc switch 3 ok
Authentication key not yet programmed
get rpmb counter error 0x00000007
00000000
emmc switch 0 ok
Load FIP HDR from eMMC, src: 0x00010200, des: 0x01700000, size: 0x00004000, part: 0
Load BL3X from eMMC, src: 0x00078200, des: 0x01768000, size: 0x000a0000, part: 0
0.0;M3 CHK:0;cm4_sp_mode 0
MVN_1=0x00000000
MVN_2=0x00000000
[Image: g12b_v1.1.3390-6ac5299 2019-09-26 14:10:05 luan.yuan@droid15-sz]
OPS=0x10
ring efuse init
chipver efuse init
29 0b 10 00 01 21 08 00 00 0a 35 38 47 47 52 50 
[0.018959 Inits done]
secure task start!
high task start!
low task start!
run into bl31
NOTICE:  BL31: v1.3(release):4fc40b1
NOTICE:  BL31: Built : 15:58:17, May 22 2019
NOTICE:  BL31: G12A normal boot!
NOTICE:  BL31: BL33 decompress pass
ERROR:   Error initializing runtime service opteed_fast


U-Boot 2020.04 (Sep 10 2020 - 14:15:14 +0800) khadas-vim3

Model: Khadas VIM3
SoC:   Amlogic Meson G12B (A311D) Revision 29:b (10:2)
DRAM:  3.8 GiB
MMC:   sd@ffe03000: 0, sd@ffe05000: 1, mmc@ffe07000: 2
Loading Environment from FAT... detect... booted from emmc...
** No partition table - mmc 2 **
In:    serial
Out:   serial
Err:   serial
[i] serial eth mac AA:A0:59:8A:1F:07
fusb302_init: Device ID: 0x91
CC connected in 1 as UFP
fusb302 detect chip.port_num = 0
Net:   eth0: ethernet@ff3f0000
Card did not respond to voltage select!
Card did not respond to voltage select!
** Unrecognized filesystem type **
** No partition table - mmc 2 **
starting USB...
Bus usb@ff500000: Register 3000140 NbrPorts 3
Starting the controller
USB XHCI 1.10
scanning bus usb@ff500000 for devices... Device NOT ready
   Request Sense returned 02 04 01
3 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Setting bus to 0
Hit SPACE in 1 seconds to stop autoboot
Device 0: unknown device
Card did not respond to voltage select!
Card did not respond to voltage select!
switch to partitions #0, OK
mmc2(part 0) is current device
** No partition table - mmc 2 **
Speed: 1000, full duplex
BOOTP broadcast 1
*** Unhandled DHCP Option in OFFER/ACK: 2
*** Unhandled DHCP Option in OFFER/ACK: 23
*** Unhandled DHCP Option in OFFER/ACK: 2
*** Unhandled DHCP Option in OFFER/ACK: 23
DHCP client bound to address 192.168.0.32 (1004 ms)
Using ethernet@ff3f0000 device
TFTP from server 192.168.0.1; our IP address is 192.168.0.32
Filename 'boot.scr.uimg'.
Load address: 0x1000000
Loading: *<0x08>T T