Broken OpenCL in Fenix for Khadas VIM3

telelaci · January 21, 2022, 10:24pm

Hi all,

If you use fenix,
BRANCH=master
COMMIT=2e74e40cd84c
MESSAGE=“logo: update logo”

,to build a system with the following configuration:

KHADAS_BOARD=VIM3
LINUX=mainline
UBOOT=mainline
DISTRIBUTION=Ubuntu
DISTRIB_RELEASE=focal
DISTRIB_TYPE=server
DISTRIB_ARCH=arm64
INSTALL_TYPE=SD-USB

, you’ll get a faulty image, with broken OpenCL and of course all of it’s dependants.
It will download:
npu_aml-dd7b0a7036d480739c9f8c62bdb51a2daa16e91a.tar.gz
containing:
aml-npu_6.4.6.2.5.3.2_arm64.deb

This package contains the following files:
CONTENTS/lib:
314504 Nov 29 23:40 libArchModelSw.so
1682875 Nov 29 23:40 libGAL.so
330224 Nov 29 23:40 libNNArchPerf.so
151264 Nov 29 23:40 libnnsdk.so
428593 Nov 29 23:40 libOpenCL.so
2689844 Nov 29 23:40 libOpenVX.so
34302 Nov 29 23:40 libOpenVXU.so
3713800 Nov 29 23:40 libovxlib.so
164240 Nov 29 23:40 libVSC_Lite.so

When you run the image on the Khadas VIM3, you run this command:
khadas@Khadas:/usr/lib$ ldd libOpenCL.so
linux-vdso.so.1 (0x0000ffff87a4e000)
libGAL.so => /lib/libGAL.so (0x0000ffff87813000)
libVSC.so => not found
libCLC.so => not found
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff87766000)
librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000ffff8774e000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff875db000)
libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffff875a9000)
libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffff87595000)
/lib/ld-linux-aarch64.so.1 (0x0000ffff87a1e000)

As you can see libOpenCL.so was linked against libVSC.so and libCLC.so, those files are missing, but you have one libVSC_Lite.so instead.

How will this OpenCL work if the libOpenCL.so module is not loadable ?
No one tested OpenCL with npu_sdk_6.4.6.2.5.3.2 till now ?

numbqq · January 22, 2022, 12:55am

Hello @telelaci

OpenCL only supported on 4.9 kernel, but you choose the mainline kernel, please use the 4.9 kernel.

telelaci · January 22, 2022, 6:12am

That makes the mainline system almost unusable, don’t you think ?

OpenCL was supported by the previous npu-aml package: 6.4.4.3AAA
With that package + mainline 5.16 kernel:
clinfo: /usr/lib/aarch64-linux-gnu/libOpenCL.so.1: no version information available (required by clinfo)
Number of platforms 1
Platform Name Vivante OpenCL Platform
Platform Vendor Vivante Corporation
Platform Version OpenCL 1.2 V6.4.4.3.310723AAA
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd
Platform Extensions function suffix viv

Platform Name Vivante OpenCL Platform
Number of devices 1
Device Name Vivante OpenCL Device VIPNano-QI.7120.0000
Device Vendor Vivante Corporation
Device Vendor ID 0x564956
Device Version OpenCL 1.2
Driver Version OpenCL 1.2 V6.4.4.3.310723AAA
Device OpenCL C Version OpenCL C 1.2
Device Type GPU
Device Profile FULL_PROFILE
… so on … it worked …

The problem is not the kernel at all, but the broken npu-aml package. You upgraded Fenix mainline and you upgraded the npu_aml in it and you removed the OpenCL support from it ? It was not worth mentioning ?

It would be supported with a proper Vivante libOpenCL.so up to OpenCL 3.0. (Mali supports only OpenCL 2.1)
You just have to get the missing libVSC.so and libCLC.so belonging to the mentioned libOpenCL.so.
That mentioned libOpenCL has been built somehow somewhere:

khadas@Khadas:/usr/lib$ sha1sum libOpenCL.so
18e615282b5b806282ee9b86f9ba5a2854b6afcf libOpenCL.so

Those missing libVSC.so and libCLC.so do exist (because Vivante couldn’t have built libOpenCL.so without them) at Vivante just it seems you don’t have it, you can’t get it.
Why is that ? Why Amlogic/VeriSilicon/Vivante do not want to support the NPU? I doubt that.
Is it a royaly problem ? Do they want device royalty for it, and you don’t want to pay for that ?
It seems that in the previous Vivante NPU blobs OpenCL was limited at v1.2 Full Profile.
Now they don’t support OpenCL at all ?
You should explain the situation to the customers honestly, they deserve it.

You know kernel 4.9 is a very serious restriction in the year of 2022 even if it gets security patches and fixes.

numbqq · January 23, 2022, 1:01am

Thanks for your feedback we will check this issue.

numbqq · February 10, 2022, 6:40am

Hello @telelaci

You want to use OpenCL with NPU or GPU?

Although clinfo command works with npu-aml package: 6.4.4.3AAA, but the information is from NPU, I’m not sure whether it can work.

root@Khadas:# clinfo 
clinfo: /lib/libOpenCL.so.1: no version information available (required by clinfo)
Number of platforms                               1
  Platform Name                                   Vivante OpenCL Platform
  Platform Vendor                                 Vivante Corporation
  Platform Version                                OpenCL 1.2 V6.4.4.3.310723AAA
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd
  Platform Extensions function suffix             viv

  Platform Name                                   Vivante OpenCL Platform
Number of devices                                 1
  Device Name                                     Vivante OpenCL Device VIPNano-QI.7120.0000
  Device Vendor                                   Vivante Corporation
  Device Vendor ID                                0x564956
  Device Version                                  OpenCL 1.2 
  Driver Version                                  OpenCL 1.2 V6.4.4.3.310723AAA
  Device OpenCL C Version                         OpenCL C 1.2 
  Device Type                                     GPU
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               1
  Max clock frequency                             800MHz
  Device Partition                                (core)
    Max number of sub-devices                     0
    Supported partition types                     (n/a)
    Supported affinity domains                    (n/a)
  Max work item dimensions                        3
  Max work item sizes                             256x256x256
  Max work group size                             256
=== CL_PROGRAM_BUILD_LOG ===
(6:0) : error : syntax error at 'kernel'
  Preferred work group size multiple              <getWGsizes:1200: create kernel : error -45>
  Preferred / native vector sizes                 
    char                                                 4 / 4       
    short                                                4 / 4       
    int                                                  4 / 4       
    long                                                 4 / 4       
    half                                                 0 / 0        (cl_khr_fp16)
    float                                                4 / 4       
    double                                               0 / 0        (n/a)
  Half-precision Floating-point support           <printDeviceInfo:68: get CL_DEVICE_HALF_FP_CONFIG : error -30>
  Single-precision Floating-point support         (core)
    Denormals                                     No
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             No
    IEEE754-2008 fused multiply-add               No
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  No
  Double-precision Floating-point support         (n/a)
  Address bits                                    32, Little-Endian
  Global memory size                              268435456 (256MiB)
  Error Correction support                        Yes
  Max memory allocation                           134217728 (128MiB)
  Unified memory for Host and Device              Yes
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       2048 bits (256 bytes)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        16384 (16KiB)
  Global Memory cache line size                   64 bytes
  Image support                                   Yes
    Max number of samplers per kernel             16
    Max size for 1D images from buffer            65536 pixels
    Max 1D or 2D image array size                 8192 images
    Max 2D image size                             8192x8192 pixels
    Max 3D image size                             8192x8192x8192 pixels
    Max number of read image args                 128
    Max number of write image args                8
  Local memory type                               Global
  Local memory size                               32768 (32KiB)
  Max number of constant args                     9
  Max constant buffer size                        65536 (64KiB)
  Max size of kernel argument                     1024
  Queue properties                                
    Out-of-order execution                        Yes
    Profiling                                     Yes
  Prefer user sync for interop                    Yes
  Profiling timer resolution                      1000ns
  Execution capabilities                          
    Run OpenCL kernels                            Yes
    Run native kernels                            No
  printf() buffer size                            1048576 (1024KiB)

I don’t have much knowledge about OpenCL, but the libOpenCL.so is from npu package not from Arm Mali blob, are you sure it can work well?

For 4.9 kernel, the correct clinfo from Mali GPU is:

Number of platforms                               1
  Platform Name                                   ARM Platform
  Platform Vendor                                 ARM
  Platform Version                                OpenCL 2.0 git.c8adbf9.122c9daed32dbba4b3056f41a2f23c58
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_3d_image_writes cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp16 cl_khr_icd cl_khr_egl_image cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_subgroups cl_khr_create_command_queue cl_arm_core_id cl_arm_printf cl_arm_thread_limit_hint cl_arm_non_uniform_work_group_size cl_arm_import_memory cl_arm_shared_virtual_memory
  Platform Extensions function suffix             ARM

  Platform Name                                   ARM Platform
Number of devices                                 1
  Device Name                                     Mali-G52
  Device Vendor                                   ARM
  Device Vendor ID                                0x72120000
  Device Version                                  OpenCL 2.0 git.c8adbf9.122c9daed32dbba4b3056f41a2f23c58
  Driver Version                                  2.0
  Device OpenCL C Version                         OpenCL C 2.0 git.c8adbf9.122c9daed32dbba4b3056f41a2f23c58
  Device Type                                     GPU
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               2
  Max clock frequency                             750MHz
  Device Partition                                (core)
    Max number of sub-devices                     0
    Supported partition types                     None
    Supported affinity domains                    (n/a)
  Max work item dimensions                        3
  Max work item sizes                             384x384x384
  Max work group size                             384
  Preferred work group size multiple              8
  Preferred / native vector sizes                 
    char                                                16 / 4       
    short                                                8 / 2       
    int                                                  4 / 1       
    long                                                 2 / 1       
    half                                                 8 / 2        (cl_khr_fp16)
    float                                                4 / 1

telelaci · February 10, 2022, 9:20am

Hi numbqq,

Both, if that would be possible, but it is not. So any of them, but I need at least one HW OpenCL device.
It’s necessary in many old-style existing computing programs, including OpenCV.
You can’t upgrade every AI tasks to the NPU sdk.

There are 2 OpenCL devices on the VIM3 board.
1 - Vivante VIPNano NPU - compatible with OpenCL 3.0 Full Profile
2 - Mali G52 GPU - compatible with OpenCL 2.0 Full Profile

I can’t use Mali GPU because I have to use mainline kernel. And I can’t use the Mali kernel drivers with the libMali.so binary blob (which contains OpenCL also) from the year 2018 in 2022 with and 5.16 mainline kernel. I don’t want to use 4.9 kernel, that is nonsense if you support a kernel released in 2016 (!!!), you want me to use it in 2022. I know it’s not your fault but Amlogic’s. They should build a new Mali IP library for the newest kernels. They stopped to support A311D Soc in the year 2018 with a 4.9 system ? That was it ?

So we have to use mainline. The aml-npu sdk would be ok, the VIPNano NPU is much faster compute device than Mali GPU anyway.
But the OpenCL support were discontinued after npu-aml package v6.4.4.3AAA.
And it was supported OpenCL v1.2 anyway, although the NPU is able to do OpenCL v3.0.
I think it was intentional restriction.
And now with the aml-npu_6.4.6.2.5.3.2 the OpenCL is broken totally.
It can be intentional, or just a bug. You shoul ask them, Vivante or Amlogic, whoever provides aml-npu_6.4.6.2.5.3.2 to you. Is there any documentation, information, roadmap about NPU support ?

Sorry to say, but A311D is getting unusable, or at least it has very limited use.
And with mainline kernel, we have a huge problem with video decoding/encoding also.
Even the H.264 decoding is bugged but basically works at least at 1080p.
But we have no 4k decoding at all.
And the AVE-10 device capabilities are: MVC, MPEG-1/2/4, VC-1/WMV, AVS, AVS+, AVS2 RealVideo, MJPEG, H.264, H265-10, VP9 and also JPEG pictures with no size limitation. The independent encoder is able to encode in JPEG or H.265/H.264 up to 1080p at 60fps
We have minimal support in mainline kernel, only H.264 decoding, nothing else in the above list.
Even LibreElec H.265 decoding freezes immediately, it can’t even play 4k movies.

Sorry to say but VIM3 SW support is catastrophy now in 2022, although A311D chip it’s not a bad and not an obsolete chip yet. You shouldn’t bury it already. But the kernel v4.9 will do that, it makes it unusable. That kernel was 2 years already in 2018, when the A311D chip was started. Now its almost 6 years old.
This is basically Amlogic’s fault, not your’s fault either Neil Armstrong’s, but you should push Amlogic harder. They could build libMali.so for 5.15 kernel if they wanted. They could upgrade their buildroot to 2022 or at least 2021 if they wanted.

Have you tesetd and didn’t work ? The clinfo executes standard OpenCL commands on the HW device, so if it’s able to get the device parameters then most probably it works, why wouldn’t ?

I’m not sure of course, but A311D has a Vivante NPU, and that NPU supports OpenCL 3.0.
This is the library what Vivante distributes for i.MX8 boards also. The i.MX8MP has vivante VIP8000 Nano NPU almost looks like the A311D’s NPU, very similar things. I hope it works, why not. And Vivante/Amlogic should support the HW device (NPU) they sold you, or they should talk about it if they don’t. Do they want royalty fee for fully supported NPU ? How much is it, per SoC ? They should say something at least.

Regards,
Laci