OpenCL benchmark

VIM3
1

Does anyone knows the performance of Vim3 in any opencl benchmark (eg. Hashcat, clpeak.)?

My purpose is to see a fear comparison in CPU and GPU against odroid n2.

2

Hello, Not sure if this will be useful for your needs, but user @NicoD posted a video with some N2 performance comparisons.

3

Hi RDFTKV. I checked that video, thanks. He didn’t compare the OpenCL performance.

4

Which bench mark do you want, I will do it for you, also tell me the firmware you want it to be tested on, I have Ubuntu Bionic server 5.5-rc2 if that has no problem.

1 Like
5

Hi, @Electr1
Thanks a lot for your offer. I will ask some tests, but of course it is your choice. I don’t understand why Khadas is not doing this, since he could potentially benefit.
Since I would like to compare the OpenCL performance what would be the best is to directly “mimic” the test1.

test1: hashcat

https://forum.odroid.com/viewtopic.php?t=34020#p290612

What is interesting here?

  • We can compare CPU+GPU (OpenCL both), first with pocl, second with propietary ARM Mali drivers.
  • We can see the clinfo (compute units, vector lengths, max mem,
) of both devices.
  • It has a few benchs, not only one (although are mining-related
)
  • We can compare against Odroid N2 (what I am considering against VIM3)

Would be great Computer Vision and Deep Learning apps, but at least we have something. It is important that we run with the same hashcat version (it seems he runs hashcat (v5.1.0-1778-gc5d2d539)) and pocl version (latest pocl_1.5).

Also, synthetic but interesting, would be this:

test2: clpeak

For both CPU and GPU.

If you have VIM3 configured differently than in the Results.md (2200/1800 MHz, 4.9 Kernel and Bionic Ubuntu), as a general baseline, the test3: sbc bench (but this test is not for OpenCL)

And finally, the test4: glmark2 (but I think you cannot run it without wayland)

https://www.cnx-software.com/2019/03/27/odroid-n2-gpu-drivers-linux-5-0-and-impressive-glmarks-es2-score/ (results) and here example view: https://www.youtube.com/watch?v=jXex4ovAJ6A&feature=youtu.be&t=409

firmware & software

About the firmware, I have no idea. Use what you want, the most stable, up to date and with better known performance. But would be great to know your “config tweaks” or OS modifications and model (VIM3 4GB eMMC32?). OS running from eMMC or NVMe SSD?

temperatures

Also, I would ask if you have suffered any throttle, CPU or GPU heating, power failures while running intensive tasks. They (odroid n2) say it never passed 45Âș (GPU) 75Âș (CPU) in intensive tasks (1h, other test). https://forum.odroid.com/viewtopic.php?t=34020#p247753
Do you have any bad experience regarding Vim3 and throttling and heat dissipation? Can you paste some avg temperatures of the CPU and GPU? (they measure the temperature while doing stress-ng --cpu 6 --cpu-method matrixprod && glmark2-es2-fbdev --off-screen --run-forever as seen here https://forum.odroid.com/viewtopic.php?f=176&t=33781&p=245875&hilit=n2+benchmark+temperature#p245875 in section Thermal characteristics)

The only stuff that is missing is NPU benchmarks, but I have no idea. Also, would be interesting to know how VIM3 behaves if we run NPU + OpenCL in CPU and GPU. But probably it is more complex to measure. I don’t know if the NPU will “steal” computing power from the CPU or GPU.

If I can “assist” in anything to you, please, let me know. I know I asked a lot, but you choose what you want to try. Anything will be welcomed and I am pretty sure will help other users to decide between both devices.

Really thank you in advance.

6

I have my OS in a SD card (SDHC class 10) I don’t think it will affect the performance but if it does you will atleast know you can expect a result a bit better, i am currently waiting for a cooling fan for which I have placed an order for, I will get these results for you ASAP once my fan arrives, expect the results by the coming Friday at max :slight_smile: I hope you have the patience to wait a bit, thank you

1 Like
7

NPU performance has been tested actually, we believe it delivers about the same performance as a Nvidia GTX1060 when running a yolov3 training model

8

@bizcocho85, I never had thermal problems before even when i wasn’t using a heatsink, but my tasks were not to intensive,
but I was getting some reasonable amount of work done (document writing, some programming, Watching a movie at 1980*1080) and not much heat produced , but still any heat produced is bad and that’s why i went ahead and purchased a fan to be on the safe side

9

Really nice, @Electr1. Sure, I can wait :slight_smile: I prefer to have something well done and I know these tests could require some time (eg. building pocl), to compare fairly.

If you don’t mind, I will ask you a few subjective questions:

What are the features do you miss from Vim3 (or Khadas company)?

Apart from the performance/stability, so far, there are only two things that “worry” me:

  • software/firmware support. It is really awful to have something that almost nobody has, or that you have to spend weeks to achieve it (if any). For example, if the board has problems to read the temperature sensors or to compute with OpenCL, that would be really bad for me. I can pay more if the company is really backing the boards they sell.
  • hw design decisions. Design decisions that could affect me, like bundled components. For example, I have no experiences with eMMC in these type of devices, and I am a bit lost what could be the penalties. Someone mentioned that if you “brick” this, you cannot replace because it is not external. I do not plan to do weird things, but maybe you know currently bad decisions.

My purpose it is just to run plenty of compute intensive tasks + monitoring + electronics (external sensors, 220v reles, H-motor drivers, etc) + communications. It will be placed (and “covered”) in the field (cold, sun, rain, dust), powered by car batteries (12v) and solar panels. And I prefer to spend 1 month studying the options if then I will spend 150€ in a board for hobby projects.

Regarding the heatsink. Yes, If I buy the Vim3 I will definitely buy the heatsink+fan.
And the NPU looks great, as you say, but I was thinking about the “interference” when using at the same time the CPU+GPU for other tasks. But this is a corner case.

Thank you

10

Bricking a VIM3 ?, no chance, ask @RDFTKV, you can boot from the eMMC even if the bootloader is damaged, just flash new firmware and its good as new :wink:

Regarding Software support I guess there is no problem with community support as we have plenty of experienced members who will be more than happy to help you, since the board is a bit new, even though it is close to a year of release, We are yet to receive the proper drivers and Things like Wayland and Panfrost support is yet to come, Though I will say With time comes support, take for example the RK3399, Four years later and this thing has solid support but is also a bit outdated.

performance is quite good, far better than an SBC out their, of this size and form factor,

Khadas takes very good care to provide proper support to all their boards, nothing like price and Age distracts the support given to each product.

I Will say using the GPIO is a bit finicky, so I rather suggest connecting something like an Arduino to do the heavy external work like driving H bridge, control relays etc. and communicate to it via UART,
and let the VIM3 do the Heavy data processing,

but ensure your board is properly protected if running on the field without a person nearby,
as these are delicate electronics :slightly_smiling_face:

Regarding Firmware I believe you will be mostly using Linux or Ubuntu for these projects,
latest builds are to be made by yourself with Fenix, if you have problem with using it do not worry, I regularly make builds and am more than happy to make you one, just specify which version and type you want like eMMC version or SD card and Mainline or vendor and bionic or focal

Hope that should answer your questions after all I did literally write a wall :laughing:

3 Likes
11

Fantastic answer, and really nice person @Electr1. Yes, indeed I thought about some of these phrases. Thanks. You took the time to answer carefully, altruistically. This is the sort of community I would like to be part :wink: Let’s see those benchmarks and commands next friday.
Have a good week, à€§à€šà„à€Żà€”à€Ÿà€Šà„

12

à€ à„€à€• à€čà„ˆ, are you from India ?

13

No, I am from Spain. I saw your name and I supposed it. I had some Indian friends when I worked in Germany.

14

Ok, no Probs
p.s I don’t Know Hindi I am actually south Indian :sweat_smile:
And I wouldn’t suggest finding my locality using my name, its quite a mismash
anyways “Muy bien” :grin:

1 Like
15

You cannot run OpenCL in NPU. You can run AI related stuff in NPU like convolutions and stuff. If you have a tensorflow/caffe model you could run it there

16

I was referring to just interfere in pure terms; both using those devices with/without penalty at the same time. Although I read in this forum something like OpenCL for the NPU.

I have reviewed it now and there is something, but maybe too obscure since nobody is using it (I could try in the future).

17

@bizcocho85 I need to check if glmark 2 works, but a similar test which is very simple and tiny is glxgears, I don’t know if its benchtest is useful but I can test it anyway for, is it ok ?

18

Sure, the more the better to have more information. I can see also that benchmark executed in other SBCs, so, we can compare too. Thanks. I am looking forward to see the OpenCL benchs. Take your time :wink:

Example of benchs in Oct 2019 from (I suspect) @NicoD https://forum.armbian.com/topic/11870-comparison-nanopi-m4-rock-pi-4b-odroid-n2-khadas-vim3/ (although there were no info there about N2).

19

If you do not mind once I finish all the benchmarks I will create a new topic and post my results there, I hope there is no problem with that for you :slight_smile:

20

Perfect. Then, I will ask and write you there.