Internal HW Watchdog?

As long as i understand there are internal watchdog in Raspebry PI https://www.domoticz.com/wiki/Setting_up_the_raspberry_pi_watchdog

Are there any similiar options inside Khadas 1 / 2?

Yes, VIM and VIM2 all have internal watchdog.


Thank you. Can you suggest any manual to enable this feature on khadas?

Are you an developer ? You need to build yourself Rom. And what system do you want to use this feature on? BTW, Can you tell me why do you want to use this feature?

I plan to use it under ubuntu mate, (have no time to switch to ubuntu server). I have litlle service running 24x7 inside docker container on khadases wich consumes rabbitmq queue. Unfortunatelly khadases unpredictably hangs one/two times a week, can’t find reason for this maybe some docker issues, so as a temp workaround thought about using watchdog to reboot it when it hangs.

Hi Cinerar,

What’s your ubuntu mate version?

Try to enable watchdog.

For linux 4.9:

diff --git a/arch/arm64/boot/dts/amlogic/mesongxl.dtsi b/arch/arm64/boot/dts/amlogic/mesongxl.dtsi
index 580ee91..d7d9fff 100644
--- a/arch/arm64/boot/dts/amlogic/mesongxl.dtsi
+++ b/arch/arm64/boot/dts/amlogic/mesongxl.dtsi
@@ -213,7 +213,7 @@
 
        watchdog {
                compatible = "amlogic, meson-wdt";
-               status = "disabled";
+               status = "okay";
                default_timeout=<10>;
                reset_watchdog_method=<1>; /* 0:sysfs,1:kernel */
                reset_watchdog_time=<2>;

Linux 3.14:

diff --git a/arch/arm64/boot/dts/mesongxl.dtsi b/arch/arm64/boot/dts/mesongxl.dtsi
index 66df452..2e8f2e9 100644
--- a/arch/arm64/boot/dts/mesongxl.dtsi
+++ b/arch/arm64/boot/dts/mesongxl.dtsi
@@ -181,7 +181,7 @@
 
        amlogic-watchdog{
                compatible = "amlogic, gx-wdt";
-               status = "disable";
+               status = "okay";
                default_timeout=<10>;
                reset_watchdog_method=<1>;//0:sysfs,1:kernel
                reset_watchdog_time=<2>;

Rebuild dtb:

$ make ARCH=arm64 kvim_defconfig
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- dtbs

Replace dtb

khadas@Khadas$ sudo cp kvim_linux_new.dtb /boot/dtb/kvim_linux.dtb

Thanks.

Ok, here is what i did

Installed DTC

sudo apt-get install device-tree-compiler

Made backup copy of dtb

   sudo cp /boot/kvim.dtb /boot/kim.bak

Converted dtb to dts

sudo dtc -b 0 -O dts -I dtb -o /boot/kvim.dts /boot/kvim.dtb

Edited dts as stated in the previous message enabling watchdog device with vi

vi /boot/kvim.dts

Compiled it back to dtb

sudo dtc -b 0 -O dtb -I dts -o /boot/kvim.dtb /boot/kvim.dts

rebooted and now if i do

dmesg | grep watchdog

i see

[    3.999372] aml_wdt c11098d0.watchdog: creat work queue for watch dog
[    3.999765] aml_wdt c11098d0.watchdog: AML Watchdog Timer probed done
[   18.638036] dhd_attach(): thread:dhd_watchdog_thread:9ce started

So looks like hw watchdog is enabled.

if i install watchdog

sudo apt-get install watchdog

then i can see this

khadas@Khadas:~$ sudo wd_identify
aml Watchdog

looks like everything is ok till now.

Then i uncomment watchdog-device=/dev/watchdog in /etc/watchdog.conf rebooted and try fork bomb to hang the system fork_bomb.py

#/bin/python
#fork_bomb.py

import os
while True:
    os.fork()

running it
python fork_bomb.py

but unfortunatelly system is not rebooted and just hangs.
If anyone have any ideas i would be very pleased.

update: found some additional info here Enable Watchdog on the C1 - ODROID
but still open for ideas

update2:
found some additional info here en:c2_watchdog_timer [ODROID Wiki]
but still no luck

update3:
more info odroid-c2:application_note:software:watchdog_timer [ODROID Wiki]

update4: i’m not sure but looks like this one should be compiled linux/drivers/watchdog/meson_wdt.c at master · torvalds/linux · GitHub to enable watchdog

update5 so i compiled meson_wdt and still nothing

root@Khadas:/home/khadas# modinfo meson_wdt
filename: /lib/modules/4.9.40/kernel/drivers/watchdog/meson_wdt.ko
description: Meson Watchdog Timer Driver
author: Carlo Caione carlo@caione.org
license: GPL
alias: of:NTCamlogic,meson8m2-wdtC*
alias: of:NTCamlogic,meson8m2-wdt
alias: of:NTCamlogic,meson8b-wdtC*
alias: of:NTCamlogic,meson8b-wdt
alias: of:NTCamlogic,meson8-wdtC*
alias: of:NTCamlogic,meson8-wdt
alias: of:NTCamlogic,meson6-wdtC*
alias: of:NTCamlogic,meson6-wdt
depends:
vermagic: 4.9.40 SMP preempt mod_unload aarch64
parm: timeout:Watchdog heartbeat in seconds (uint)
parm: nowayout:Watchdog cannot be stopped once started (default=0) (bool)

echo 3 > /dev/watchdog doesnt reboot system. today i’m out of ideas will be thankful for any help.

Hi Cinerar,

I’ve tried, it doesn’t work. And I update the watchdog driver, you can find patch here.

Whit this dtb and kernel, you should add watchdog module:

root@Khadas:~# echo gxbb_wdt >> /etc/modules

And do the following things:

root@Khadas:~# mkdir -p /var/log/watchdog

Edit /etc/default/watchdog:

root@Khadas:~# cat /etc/default/watchdog 
# Start watchdog at boot time? 0 or 1
run_watchdog=1
# Start wd_keepalive after stopping watchdog? 0 or 1
run_wd_keepalive=1
# Load module before starting watchdog
Watchdog_module=gxbb_wdt
# Specify additional watchdog options here (see manpage).
watchdog_options="-s -v -c /etc/watchdog.conf"

Edit /etc/watchdog.conf:

root@Khadas:~# cat /etc/watchdog.conf
#ping                   = 172.31.14.1
#ping                   = 172.26.1.255
#interface              = eth0
#file                   = /var/log/messages
#change                 = 1407

# Uncomment to enable test. Setting one of these values to '0' disables it.
# These values will hopefully never reboot your machine during normal use
# (if your machine is really hung, the loadavg will go much higher than 25)
#max-load-1             = 24
#max-load-5             = 18
#max-load-15            = 12

# Note that this is the number of pages!
# To get the real size, check how large the pagesize is on your machine.
#min-memory             = 1
#allocatable-memory     = 1

#repair-binary          = /usr/sbin/repair
#repair-timeout         = 
#test-binary            = 
#test-timeout           = 

watchdog-device = /dev/watchdog

# Defaults compiled into the binary
#temperature-device     =
#max-temperature        = 120

# Defaults compiled into the binary
admin                   = root
interval                = 1
logtick                = 1
log-dir         = /var/log/watchdog

# This greatly decreases the chance that watchdog won't be scheduled before
# your machine is really loaded
realtime                = yes
priority                = 1

# Check if rsyslogd is still running by enabling the following line
pidfile         = /var/run/rsyslogd.pid   

watchdog-timeout        = 5

Enable watchdog service, due to a debian bug? Found here.

root@Khadas:~# ln -s  /lib/systemd/system/watchdog.service /etc/systemd/system/multi-user.target.wants/watchdog.service
root@Khadas:~# systemctl enable watchdog.service
root@Khadas:~# systemctl start watchdog.service

Check watchdog daemon, try to kill watchdog daemon:

root@Khadas:~# killall -9 watchdog
root@Khadas:~# killall -9 wd_keepalive

And the system will reboot in 5 seconds(setting in /etc/watchdog.conf).

Try to crash the kernel manually:

root@Khadas:~# echo c > /proc/sysrq-trigger

And the system will reboot in 5 seconds(setting in /etc/watchdog.conf).

But for your python script fork_bomb.py seems not work. Does it will realy block watchdog daemon to feed watchdog?

Thanks.

Thank you numbqq, do i understand it right: this patch is only for 3.14 kernel?

Hi Cinerar,

I only test it on 3.14 now, but I think it can work on 4.9, too. Because the watchdog driver is for 4.14.y. And I will check on 4.9 later.

Thanks.

Hi numbqq, so i tried to port your instructions to 4.9 but unfortunatelly i m not smart enough to do it.
I succesefully compiled gxbb_wdt and inserted it into the kernel.

khadas@Khadas:~$ lsmod
Module                  Size  Used by
gxbb_wdt               16384  31065126
aufs                  258048  30891806
dhd                   888832  30050446
mali                  274432  0

As long as i understand /dev/watchdog should be created after this, but

khadas@Khadas:~$ ls /dev/watchdog
ls: cannot access '/dev/watchdog': No such file or directory

I installed watchdog service and followed yout instructions for configuring it, but this doesn’t help as long as there are no file /dev/watchdog to write to (as long as i understand watchdog service just writes some data to /dev/watchdog and when data is not written after timeout system reboot)

And of course

root@Khadas:~# echo c > /proc/sysrq-trigger

just hangs the system without reboot

Maybe you have some time to take a look at 4.9. Thank you very much for your time.

UPDATE my mistake

Quite unobivious
Not - working:
compatible = "amlogic, meson-gxbb-wdt"
Working
compatible = "amlogic,meson-gxbb-wdt"

Will try to make pull request

Hi Cinerar,

I will check on linux 4.9.

hi numbqq i tried to make pull request https://github.com/numbqq/linux/pull/1 check it please

Sorry, closed it, need to wrap my head around git ) will open it a bit later again

if someone will need it this is dtb with enabled watchdog for khadas vim with kernel 4.9 https://yadi.sk/d/fRONX19z3WQknq replace in /boot/kvim.dtb
also there is driver gxbb_wdt install it using depmod and echo gxbb_wdt >> /etc/modules to autoload

then install watchdog service sudo apt-get install watchdog
and follow instructions from numbqq post Internal HW Watchdog?

Hope watchdog driver will be included by default in future releases. TY numbqq

2 Likes

I tried to use the watchdog with vim3 and mainline kernel and the patch works.
I added:
File meson-g12-common.dtsi under soc
amlwatchdog: amlogic-watchdog {
compatible = “amlogic,meson-gxbb-wdt”;
status = “disabled”;
reg = <0x0 0xffd0f0d0 0x0 0x10>;
clocks = <&xtal>;
};

and
File meson-g12b-a311d-khadas-vim3.dts at the end
&amlwatchdog {
status = “okay”;
};

The Kernel Module was already activated in 0.9.4
CONFIG_MESON_GXBB_WATCHDOG=y
CONFIG_MESON_WATCHDOG=y

After enabling service watchdog, it works. have a look here.

root@Khadas:~# service watchdog status
● watchdog.service - watchdog daemon
Loaded: loaded (/lib/systemd/system/watchdog.service; enabled; vendor preset:
Active: active (running) since Thu 2020-09-24 15:25:34 CEST; 7s ago
Process: 2731 ExecStartPre=/bin/sh -c [ -z “${watchdog_module}” ] || [ "${watc
Process: 2732 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/wa
Main PID: 2734 (watchdog)
Tasks: 1 (limit: 3573)
Memory: 628.0K
CGroup: /system.slice/watchdog.service
└─2734 /usr/sbin/watchdog -s -v -c /etc/watchdog.conf

Sep 24 15:25:37 Khadas watchdog[2734]: still alive after 3 interval(s)
Sep 24 15:25:37 Khadas watchdog[2734]: was able to ping process 1917 (/var/run/c
Sep 24 15:25:38 Khadas watchdog[2734]: still alive after 4 interval(s)
Sep 24 15:25:39 Khadas watchdog[2734]: was able to ping process 1917 (/var/run/c
Sep 24 15:25:40 Khadas watchdog[2734]: still alive after 5 interval(s)
Sep 24 15:25:40 Khadas watchdog[2734]: was able to ping process 1917 (/var/run/c
Sep 24 15:25:41 Khadas watchdog[2734]: still alive after 6 interval(s)
Sep 24 15:25:41 Khadas watchdog[2734]: was able to ping process 1917 (/var/run/c
Sep 24 15:25:42 Khadas watchdog[2734]: still alive after 7 interval(s)
Sep 24 15:25:42 Khadas watchdog[2734]: was able to ping process 1917 (/var/run/c

root@Khadas:~# echo c > /proc/sysrq-trigger
[ 470.213191] sysrq: Trigger a crash
[ 470.213269] Kernel panic - not syncing: sysrq triggered crash
[ 470.216717] SMP: stopping secondary CPUs
[ 470.220599] Kernel Offset: disabled
[ 470.224045] CPU features: 0x080002,20002004
[ 470.228183] Memory Limit: none
[ 470.231207] —[ end Kernel panic - not syncing: sysrq triggered crash ]—
G12B:BL:6e7c85:2a3b91;FEAT:E0F83180:402000;POC:F;RCY:0;EMMC:0;READ:0;0.▒!,K▒▒х▒▒}▒▒▒с0x01
bl2_stage_init 0x81
hw id: 0x0000 - pwm id 0x01
bl2_stage_init 0xc1
bl2_stage_init 0x02

L0:00000000
L1:20000703
L2:00008067
L3:14000000
B2:00402000
B1:e0f83180

TE: 306839

BL2 Built : 15:22:05, Aug 28 2019. g12b g1bf2b53 - luan.yuan@droid15-sz

Could someone add this to the mainline kernel.

Thank you
Regards Andreas