VIM4-NN causing kernel panic on samples

Which system do you use? Android, Ubuntu, OOWOW or others?

Fenix Debian11

Which version of system do you use? Khadas official images, self built images, or others?

Built from Repo - Fenix Debian11 Minimal

Please describe your issue below:

Kernel panics on Demo NN application

I used Debian Buster NN (libadla.so) installer from here

Post a console log of your issue below:

[    0.566031] Amlogic Meson T7 (A311D2) Revision 36:c (1:2) Detected


khadas@Khadas:~/workspace/vim4_npu_applications/mobilenet_v2/build$ sudo ./mobilenetv2 ../data/mobilenetv2_int8.adla ../data/goldfish_224x224.jpg
[sudo] password for khadas:
adla usr space 1.2.0.5
[ADLAU ERROR] adla_platform_wait_with_ioctl: ioctl err. ret=-1
[ADLAU ERROR] adla_platform_wait_with_ioctl: invoke err,return status=1052535544
[ADLAU ERROR] dispatch_hw_op: wait fail,curr_task_index[40],end_index[41]
[ADLAU ERROR] thread_invoke_tophalf: dispatch_tasks failed in "[thread_invoke_tophalf]"
E NN_SDK:[aml_adla_run_network_off:1165]Error: wait network finish fail.
  1: 0.500000
  2: 0.500000
  3: 0.500000
  4: 0.500000
  5: 0.500000
[ADLAU ERROR] adla_platform_free_mem: Failed to free buffer!
[ADLAU ERROR] adla_platform_free_mem: Failed to free buffer!
[ADLAU ERROR] adla_platform_free_mem: Failed to free buffer!

After 2nd run

[  141.487856@3]  Unable to handle kernel paging request at virtual address ffffffc0866cd230
[  141.487902@0]  [ADLAK WARN] adlak_destroy_context() net [2] is busy,so destroy delay!
[  141.497927@3]  Mem abort info:
[  141.497930@3]    ESR = 0x96000005
[  141.497933@3]    EC = 0x25: DABT (current EL), IL = 32 bits
[  141.497936@3]    SET = 0, FnV = 0
[  141.497938@3]    EA = 0, S1PTW = 0
[  141.497940@3]  Data abort info:
[  141.497942@3]    ISV = 0, ISS = 0x00000005
[  141.497944@3]    CM = 0, WnR = 0
[  141.497947@3]  swapper pgtable: 4k pages, 39-bit VAs, pgdp=00000000022d0000
[  141.497950@3]  [ffffffc0866cd230] pgd=0000000000000000, pud=0000000000000000
[  141.497955@3]  Internal error: Oops: 96000005 [#1] PREEMPT SMP
[  141.497961@3]  Modules linked in: binfmt_misc iv009_isp iv009_isp_sensor iv009_isp_lens iv009_isp_iq nft_compat nft_counter nft_chain_nat nf_tables overlay optee_armtz optee adc_keypad dolby_fw exfat gpio_keypad leds_state meson_clk_debug meson_cpufreq meson_gxl meson_ir reg_access audio_data audiodsp snd_soc_dummy_codec snd_soc_aml_t9015 snd_soc_aml_codec_tl1_acodec snd_usb_audio snd_hwdep snd_usbmidi_lib amvenc_multi jpegenc media_sync amvdec_av1_v4l amvdec_av1 amvdec_mavs amvdec_avs2 amvdec_vp9_v4l amvdec_vp9 amvdec_vc1 amvdec_mmpeg4_v4l amvdec_mmpeg4 amvdec_mmpeg12_v4l amvdec_mmpeg12 amvdec_mmjpeg_v4l amvdec_mmjpeg amvdec_h265_v4l amvdec_h265 amvdec_h264mvc amvdec_mh264_v4l amvdec_mh264 amvdec_ports stream_input decoder_common aml_drm firmware media_clock snd_soc video_framerate_adapter zram aml_media ov08a10 imx415 meson_gxbb_wdt mali_kbase aml_spicc snd_soc_notify meson_clk_measure aml_dvb_extern aml_gki_tool meson_saradc adla_core aml_crypto_dma amlcam hifidsp aml_rng reboot dhd
[  141.781950@3]  PC : ffffffe791d9f6c0, PFN:2116b1 V
[  141.795694@3]  SP : ffffffc020203cc0, PFN:211fe5 V
[  141.809294@3]  FAR : ffffffc0866cd230, PFN:***** V
[  141.822780@3]  R6  : ffffffb6747a2f40, PFN:1f47a2 L
[  141.836236@3]  R16 : ffffffe810e847c0, PFN: 1e84 V
[  141.849590@3]  R17 : ffffffe8100a4c94, PFN: 10a4 V
[  141.862695@3]  R19 : ffffffc011dca000, PFN:1fb354 V
[  141.875648@3]  R21 : ffffffc011fa7000, PFN:212500 V
[  141.888495@3]  R23 : ffffffc011dbf068, PFN:2007f5 V
[  141.901214@3]  R26 : ffffffb691480c00, PFN:211480 L
[  141.913798@3]  R29 : ffffffc020203cc0, PFN:211fe5 V
[  141.926210@3]  R30 : ffffffe791d9f65c, PFN:2116b1 V
[  141.938461@3]  CPU: 3 PID: 383 Comm: adlak_kthread_0 Not tainted 5.4.180 #1.5.1
[  141.952585@3]  Hardware name: Khadas VIM4 (DT)
[  141.964810@3]  pstate: 80400005 (Nzcv daif +PAN -UAO)
[  141.976889@3]  pc : adlak_submit_patch_and_exec+0x480/0xae4 [adla_core]
[  141.989993@3]  lr : adlak_submit_patch_and_exec+0x41c/0xae4 [adla_core]
[  142.003094@3]  sp : ffffffc020203cc0
[  142.015334@3]  x29: ffffffc020203cc0 x28: 0000000075700000
[  142.027774@3]  x27: 0000000000000088 x26: ffffffb691480c00
[  142.040322@3]  x25: 0000000000000a10 x24: 0000000000000088
[  142.052965@3]  x23: ffffffc011dbf068 x22: 0000000000020000
[  142.065687@3]  x21: ffffffc011fa7000 x20: 0000000000000302
[  142.078461@3]  x19: ffffffc011dca000 x18: 0000000000000000
[  142.091124@3]  x17: ffffffe8100a4c94 x16: ffffffe810e847c0
[  142.103767@3]  x15: 0000000000000000 x14: 0000000100000000
[  142.116547@3]  x13: 0000000a000ea000 x12: 000e9f0c81010040
[  142.129367@3]  x11: 000e000e01f7010a x10: 0000000000000000
[  142.142267@3]  x9 : 0000000000003000 x8 : 0000000000000000
[  142.155210@3]  x7 : 00000000ffffffb6 x6 : ffffffb6747a2f40
[  142.168265@3]  x5 : 0000000000000000 x4 : 000000001d1c988c
[  142.181434@3]  x3 : 0000000000000018 x2 : 000000000000001e
[  142.194654@3]  x1 : 000000001d1c988c x0 : 0000000000000000
[  142.207907@3]
[  142.207907@3]  PC: 0xffffffe791d9f640:
[  142.234434@3]  f640  6b16031f 1a982080 1a982084 8b000aa0 6b04007f 54002643 97ffe0a2 b98012f4
[  142.251125@3]  f660  d342fe94 0b140374 b9401ae0 52800002 52800303 7100001f 5400036d f94063e6
[  142.268059@3]  f680  b94016e0 0b000040 11000442 9b237c00 8b0000c5 b8a068c1 b9400ca8 f94008a0
[  142.285237@3]  f6a0  d342fc21 0b010301 b94004a7 4b160024 6b0102df 1a819081 b94008a5 9ac80800
[  142.302605@3]  f6c0  b8617aa4 9ac72000 4a000080 0a050000 4a040000 b8217aa0 b9401ae0 6b00005f
[  142.320198@3]  f6e0  54fffd0b b9434f40 51000401 7100043f 54000c88 b94022e1 7100003f 54000c2d
[  142.338025@3]  f700  93407c1b b940b7e7 9100137b 52800019 29559be5 d37ef77b f9403ffc 14000037
[  142.356087@3]  f720  340009a4 f94037e0 b9400000 7100001f 54002bec 52800004 52800002 12800001
[  142.374357@3]
[  142.374357@3]  LR: 0xffffffe791d9f5dc:
[  142.405377@3]  f5dc  b940abe0 6b0202df 0b000081 4b160040 1a829002 4b160020 6b0102df 1a819001
[  142.424346@3]  f5fc  52aea000 2a00039c b8227abc 52aec000 2a000294 b8217ab4 b94087e0 69418ae1
[  142.443551@3]  f61c  0b1b0018 f9404be0 4b160304 d342fc43 0b030303 8b010001 6b0302df 4b160060
[  142.463007@3]  f63c  1a839003 6b16031f 1a982080 1a982084 8b000aa0 6b04007f 54002643 97ffe0a2
[  142.482412@3]  f65c  b98012f4 d342fe94 0b140374 b9401ae0 52800002 52800303 7100001f 5400036d
[  142.501765@3]  f67c  f94063e6 b94016e0 0b000040 11000442 9b237c00 8b0000c5 b8a068c1 b9400ca8
[  142.521066@3]  f69c  f94008a0 d342fc21 0b010301 b94004a7 4b160024 6b0102df 1a819081 b94008a5
[  142.540470@3]  f6bc  9ac80800 b8617aa4 9ac72000 4a000080 0a050000 4a040000 b8217aa0 b9401ae0
[  142.559979@3]
[  142.559979@3]  SP: 0xffffffc020203c40:
[  142.593149@3]  3c40  00000088 00000000 00000a10 00000000 91480c00 ffffffb6 00000088 00000000
[  142.613080@3]  3c60  75700000 00000000 20203cc0 ffffffc0 91d9f65c ffffffe7 20203cc0 ffffffc0
[  142.633091@3]  3c80  91d9f6c0 ffffffe7 80400005 00000000 00000000 00000000 00000000 00000000
[  142.653241@3]  3ca0  ffffffff ffffffff 00000000 00000000 20203cc0 ffffffc0 91d9f6c0 ffffffe7
[  142.673321@3]  3cc0  20203df0 ffffffc0 91d9b824 ffffffe7 91480e28 ffffffb6 91480c00 ffffffb6
[  142.693411@3]  3ce0  00000000 00000000 00000000 00000000 91481000 ffffffb6 91da95b0 ffffffe7
[  142.713439@3]  3d00  91da19b0 ffffffe7 91da19c8 ffffffe7 00000003 00000000 00000000 00000000
[  142.732809@3]  3d20  00000000 00000000 91da9590 ffffffe7 91d97c48 00000084 91480e28 ffffffb6
[  142.751365@3]
[  142.751365@3]  FAR: 0xffffffc0866cd230:
[  142.779995@3]  d230  ******** ******** ******** ******** ******** ******** ******** ********
[  142.796267@3]  d250  ******** ******** ******** ******** ******** ******** ******** ********
[  142.812318@3]  d270  ******** ******** ******** ******** ******** ******** ******** ********
[  142.828333@3]  d290  ******** ******** ******** ******** ******** ******** ******** ********
[  142.844228@3]  d2b0  ******** ******** ******** ******** ******** ******** ******** ********
[  142.860036@3]  d2d0  ******** ******** ******** ******** ******** ******** ******** ********
[  142.875766@3]  d2f0  ******** ******** ******** ******** ******** ******** ******** ********
[  142.891453@3]  d310  ******** ******** ******** ******** ******** ******** ******** ********
[  142.907063@3]
[  142.907063@3]  X6: 0xffffffb6747a2ec0:
[  142.932674@3]  2ec0  747a2d80 ffffffb6 ffffffff 00000001 00080000 00000000 00000070 00000000
[  142.948817@3]  2ee0  ffffffff 00000001 0008014a 00000000 00000000 00000000 00000000 00000000
[  142.965059@3]  2f00  00000001 00000003 00000019 00000000 00000000 00000000 00000000 00000000
[  142.981300@3]  2f20  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[  142.997255@3]  2f40  00000070 00000000 ffffffff 00000001 00080000 00000000 00000070 00000000
[  143.013020@3]  2f60  ffffffff 00000001 0008014a 00000000 00000000 00000000 00000000 00000000
[  143.028724@3]  2f80  8f83fec0 ffffffb6 91761140 ffffffb6 92730c00 ffffffb6 bc00029b 00000006
[  143.044428@3]  2fa0  090400d0 1a9a6f50 00000000 00000000 00000000 00000000 00000000 00000000
[  143.060299@3]
[  143.060299@3]  X16: 0xffffffe810e84740:
[  143.086021@3]  4740  f8408403 f8408424 ca040066 f100058c da9f10c7 b4fffee7 b4000146 dac00cc6
[  143.102148@3]  4760  dac00c63 dac00c84 dac010cb 9acb2063 9acb2084 d378fc63 cb44e060 d65f03c0
[  143.118441@3]  4780  f2400842 54000100 38401403 38401424 f1000442 7a441060 54ffff80 cb040060
[  143.134891@3]  47a0  d65f03c0 d2800000 d65f03c0 00000000 00000000 00000000 00000000 00000000
[  143.151496@3]  47c0  aa0003e6 f100405f 540003c3 cb0103e4 f2400c84 540001c0 cb040042 36000064
[  143.168275@3]  47e0  38401423 380014c3 36080064 78402423 780024c3 36100064 b8404423 b80044c3
[  143.185192@3]  4800  36180064 f8408423 f80084c3 f101005f 5400032a f27c0443 54000140 7100807f
[  143.202240@3]  4820  54000080 540000ab a8c12027 a88120c7 a8c12027 a88120c7 a8c12027 a88120c7
[  143.219461@3]
[  143.219461@3]  X17: 0xffffffe8100a4c14:
[  143.248251@3]  4c14  d2c01002 d2596004 510004c6 9ac620a5 cb050042 eb02009f 540001c2 90008b82
[  143.265957@3]  4c34  92409800 f9409042 8b020000 94001876 a8c17bfd d65f03c0 d5382045 927ae4a5
[  143.283802@3]  4c54  aa050000 d5182040 d5033fdf 17ffffd4 f9409862 cb020000 17fffff5 d4210000
[  143.301880@3]  4c74  a9bb7bfd 90009ba1 9100a021 910003fd a90153f3 d538d080 b8606833 97ff8bcb
[  143.320141@3]  4c94  97ffffaf d503201f d503201f d503201f d5384100 97ff97d2 d000c9c0 f944f001
[  143.338514@3]  4cb4  b4000061 2a1303e0 d63f0020 d000bca0 b9454800 34000480 a94153f3 a8c57bfd
[  143.357078@3]  4cd4  d65f03c0 d000c9c0 f9429400 7211001f 54fffde0 a90363f7 90008b97 90008b98
[  143.375590@3]  4cf4  f90023f9 b000c9b9 f9409af4 a9025bf5 90009175 f9409316 910002b5 f9440b23
[  143.394051@3]
[  143.394051@3]  X19: 0xffffffc011dc9f80:
[  143.424858@3]  9f80  ******** ******** ******** ******** ******** ******** ******** ********
[  143.443363@3]  9fa0  ******** ******** ******** ******** ******** ******** ******** ********
[  143.461624@3]  9fc0  ******** ******** ******** ******** ******** ******** ******** ********
[  143.479668@3]  9fe0  ******** ******** ******** ******** ******** ******** ******** ********
[  143.497478@3]  a000  00000001 00000001 00000020 00000014 00f00000 00000000 00000001 00000002
[  143.515305@3]  a020  00000020 0000000c 00003000 00000000 00000000 00000020 00000010 000f0000
[  143.533141@3]  a040  00000000 00000001 00000002 00000020 0000000a 00000c00 00000001 00000001
[  143.550700@3]  a060  00000020 00000014 00f00000 00000000 00000001 00000002 00000020 0000000c
[  143.568051@3]
[  143.568051@3]  X21: 0xffffffc011fa6f80:
[  143.596503@3]  6f80  ******** ******** ******** ******** ******** ******** ******** ********
[  143.613785@3]  6fa0  ******** ******** ******** ******** ******** ******** ******** ********
[  143.630858@3]  6fc0  ******** ******** ******** ******** ******** ******** ******** ********
[  143.647749@3]  6fe0  ******** ******** ******** ******** ******** ******** ******** ********
[  143.664372@3]  7000  7e000000 74000000 75100000 760003ed 00e00074 00000003 00700039 00000020
[  143.681003@3]  7020  00030309 00000000 01010202 00000101 0000000e 00e00074 0000003a 00000000
[  143.697695@3]  7040  ffffff80 ffffff80 00000001 0003a005 80042cb0 01cd0000 00000024 03ffffff
[  143.714292@3]  7060  00000000 00000000 00000000 0c000000 30000000 03ffffff 00000033 ff800003
[  143.730881@3]
[  143.730881@3]  X23: 0xffffffc011dbefe8:
[  143.757964@3]  efe8  ******** ******** ******** ******** ******** ******** 000026bc 00002000
[  143.774655@3]  f008  000009eb 00000000 000001ec 00000000 00000001 00000000 00000000 00000000
[  143.791434@3]  f028  00000000 ffffffff ffffffff ffffffff 00000001 00000001 00000000 00000001
[  143.808065@3]  f048  00000000 00000001 00000001 00000001 00000001 00000000 00000000 00000002
[  143.824636@3]  f068  00000000 ffffffff 000026bc 00002000 000009eb 000001ec 000001ec 00000001
[  143.841008@3]  f088  00000001 00000000 00000000 00000000 00000000 ffffffff 00000000 ffffffff
[  143.857457@3]  f0a8  00000001 00000001 00000000 00000001 00000000 00000001 00000001 00000001
[  143.873681@3]  f0c8  00000001 00000000 00000000 00000002 00000000 ffffffff 0000103c 00001000
[  143.889984@3]
[  143.889984@3]  X26: 0xffffffb691480b80:
[  143.917123@3]  0b80  da295c00 ffffffff da295c40 ffffffff da295c80 ffffffff da295cc0 ffffffff
[  143.934131@3]  0ba0  da295d00 ffffffff da295d40 ffffffff da295d80 ffffffff da295dc0 ffffffff
[  143.951143@3]  0bc0  da295e00 ffffffff da295e40 ffffffff da295e80 ffffffff da295ec0 ffffffff
[  143.968165@3]  0be0  da295f00 ffffffff da295f40 ffffffff da295f80 ffffffff da295fc0 ffffffff
[  143.985221@3]  0c00  914a2100 ffffffb6 94851400 ffffffb6 94851410 ffffffb6 00000000 00000000
[  144.002164@3]  0c20  91480c20 ffffffb6 91480c20 ffffffb6 00000000 00000000 00000000 00000000
[  144.019133@3]  0c40  11899000 ffffffe8 00000000 00000000 00000002 00000001 91da91c0 ffffffe7
[  144.035964@3]  0c60  91480c90 ffffffb6 8f823ab8 ffffffb6 8f823ab8 ffffffb6 1fb00000 00000001
[  144.052943@3]
[  144.052943@3]  X29: 0xffffffc020203c40:
[  144.081252@3]  3c40  00000088 00000000 00000a10 00000000 91480c00 ffffffb6 00000088 00000000
[  144.098763@3]  3c60  75700000 00000000 20203cc0 ffffffc0 91d9f65c ffffffe7 20203cc0 ffffffc0
[  144.116347@3]  3c80  91d9f6c0 ffffffe7 80400005 00000000 00000000 00000000 00000000 00000000
[  144.133984@3]  3ca0  ffffffff ffffffff 00000000 00000000 20203cc0 ffffffc0 91d9f6c0 ffffffe7
[  144.151595@3]  3cc0  20203df0 ffffffc0 91d9b824 ffffffe7 91480e28 ffffffb6 91480c00 ffffffb6
[  144.169093@3]  3ce0  00000000 00000000 00000000 00000000 91481000 ffffffb6 91da95b0 ffffffe7
[  144.186669@3]  3d00  91da19b0 ffffffe7 91da19c8 ffffffe7 00000003 00000000 00000000 00000000
[  144.204210@3]  3d20  00000000 00000000 91da9590 ffffffe7 91d97c48 00000084 91480e28 ffffffb6
[  144.221726@3]
[  144.236169@3]  Call trace:
[  144.250136@3]  [ffffffc020203cc0+ 304][<ffffffe791d9f6c0>] adlak_submit_patch_and_exec+0x480/0xae4 [adla_core]
[  144.269015@3]  [ffffffc020203df0+ 112][<ffffffe791d9b824>] adlak_dev_inference_cb+0x134/0x560 [adla_core]
[  144.287416@3]  [ffffffc020203e60+   0][<ffffffe810120084>] kthread+0x164/0x170
[  144.289840@3]  cec: bad initiator with self 0x0
[  144.303419@3]  [0000000000000000+   0][<ffffffe810087bb4>] ret_from_fork+0x10/0x18
[  144.333990@3]  Code: 6b0102df 1a819081 b94008a5 9ac80800 (b8617aa4)
[  144.349220@3]  ---[ end trace 0164fedc63997665 ]---
[  144.375590@3]  Kernel panic - not syncing: Fatal exception
[  144.390213@3]  SMP: stopping secondary CPUs
[  144.404879@3]  Kernel Offset: 0x2800000000 from 0xffffffc010000000
[  144.420019@3]  PHYS_OFFSET: 0xffffffcb80000000
[  144.434499@3]  CPU features: 0x00030002,21882004
[  144.448864@3]  Memory Limit: none
[  144.474980@3]  Rebooting in 5 seconds..

EDIT:

Doesnt look like the kernel booted it correctly … from dmesg - this correct?

[    6.938359] [ADLAK WARN] adlak_platform_get_resource() Failed to get adla_axi_clk

[    6.938896] adla kmd version: 0.10.7.2
[    6.938900] adla DDK version: 1.7.1

UPDATE: Sample works under Ubuntu Server from OOWOW

Ubuntu-server has the same error… so that is nothing to do with it.

[    8.863712] [ADLAK WARN] adlak_platform_get_resource() Failed to get adla_axi_clk
[    8.864241] adla kmd version: 0.10.7.2
[    8.864713] adla DDK version: 1.7.1

There is a difference between Ubuntu and the debian11 Fenix package - (ADLA dpkg is missing for one!)

Solved… rebuilt with ‘SERVER’ version of Debian11 and all working…