Internal HW Watchdog?


#1

As long as i understand there are internal watchdog in Raspebry PI https://www.domoticz.com/wiki/Setting_up_the_raspberry_pi_watchdog

Are there any similiar options inside Khadas 1 / 2?


#2

Yes, VIM and VIM2 all have internal watchdog.



#3

Thank you. Can you suggest any manual to enable this feature on khadas?


#4

Are you an developer ? You need to build yourself Rom. And what system do you want to use this feature on? BTW, Can you tell me why do you want to use this feature?


#5

I plan to use it under ubuntu mate, (have no time to switch to ubuntu server). I have litlle service running 24x7 inside docker container on khadases wich consumes rabbitmq queue. Unfortunatelly khadases unpredictably hangs one/two times a week, can’t find reason for this maybe some docker issues, so as a temp workaround thought about using watchdog to reboot it when it hangs.


#6

Hi Cinerar,

What’s your ubuntu mate version?

Try to enable watchdog.

For linux 4.9:

diff --git a/arch/arm64/boot/dts/amlogic/mesongxl.dtsi b/arch/arm64/boot/dts/amlogic/mesongxl.dtsi
index 580ee91..d7d9fff 100644
--- a/arch/arm64/boot/dts/amlogic/mesongxl.dtsi
+++ b/arch/arm64/boot/dts/amlogic/mesongxl.dtsi
@@ -213,7 +213,7 @@
 
        watchdog {
                compatible = "amlogic, meson-wdt";
-               status = "disabled";
+               status = "okay";
                default_timeout=<10>;
                reset_watchdog_method=<1>; /* 0:sysfs,1:kernel */
                reset_watchdog_time=<2>;

Linux 3.14:

diff --git a/arch/arm64/boot/dts/mesongxl.dtsi b/arch/arm64/boot/dts/mesongxl.dtsi
index 66df452..2e8f2e9 100644
--- a/arch/arm64/boot/dts/mesongxl.dtsi
+++ b/arch/arm64/boot/dts/mesongxl.dtsi
@@ -181,7 +181,7 @@
 
        amlogic-watchdog{
                compatible = "amlogic, gx-wdt";
-               status = "disable";
+               status = "okay";
                default_timeout=<10>;
                reset_watchdog_method=<1>;//0:sysfs,1:kernel
                reset_watchdog_time=<2>;

Rebuild dtb:

$ make ARCH=arm64 kvim_defconfig
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- dtbs

Replace dtb

khadas@Khadas$ sudo cp kvim_linux_new.dtb /boot/dtb/kvim_linux.dtb

Thanks.


#7

Ok, here is what i did

Installed DTC

sudo apt-get install device-tree-compiler

Made backup copy of dtb

   sudo cp /boot/kvim.dtb /boot/kim.bak

Converted dtb to dts

sudo dtc -b 0 -O dts -I dtb -o /boot/kvim.dts /boot/kvim.dtb

Edited dts as stated in the previous message enabling watchdog device with vi

vi /boot/kvim.dts

Compiled it back to dtb

sudo dtc -b 0 -O dtb -I dts -o /boot/kvim.dtb /boot/kvim.dts

rebooted and now if i do

dmesg | grep watchdog

i see

[    3.999372] aml_wdt c11098d0.watchdog: creat work queue for watch dog
[    3.999765] aml_wdt c11098d0.watchdog: AML Watchdog Timer probed done
[   18.638036] dhd_attach(): thread:dhd_watchdog_thread:9ce started

So looks like hw watchdog is enabled.

if i install watchdog

sudo apt-get install watchdog

then i can see this

khadas@Khadas:~$ sudo wd_identify
aml Watchdog

looks like everything is ok till now.

Then i uncomment watchdog-device=/dev/watchdog in /etc/watchdog.conf rebooted and try fork bomb to hang the system fork_bomb.py

#/bin/python
#fork_bomb.py

import os
while True:
    os.fork()

running it
python fork_bomb.py

but unfortunatelly system is not rebooted and just hangs.
If anyone have any ideas i would be very pleased.

update: found some additional info here https://forum.odroid.com/viewtopic.php?f=115&t=8984
but still open for ideas

update2:
found some additional info here http://odroid.com/dokuwiki/doku.php?id=en:c2_watchdog_timer
but still no luck

update3:
more info https://wiki.odroid.com/odroid-c2/application_note/software/watchdog_timer

update4: i’m not sure but looks like this one should be compiled https://github.com/torvalds/linux/blob/master/drivers/watchdog/meson_wdt.c to enable watchdog

update5 so i compiled meson_wdt and still nothing

root@Khadas:/home/khadas# modinfo meson_wdt
filename: /lib/modules/4.9.40/kernel/drivers/watchdog/meson_wdt.ko
description: Meson Watchdog Timer Driver
author: Carlo Caione carlo@caione.org
license: GPL
alias: of:NTCamlogic,meson8m2-wdtC*
alias: of:NTCamlogic,meson8m2-wdt
alias: of:NTCamlogic,meson8b-wdtC*
alias: of:NTCamlogic,meson8b-wdt
alias: of:NTCamlogic,meson8-wdtC*
alias: of:NTCamlogic,meson8-wdt
alias: of:NTCamlogic,meson6-wdtC*
alias: of:NTCamlogic,meson6-wdt
depends:
vermagic: 4.9.40 SMP preempt mod_unload aarch64
parm: timeout:Watchdog heartbeat in seconds (uint)
parm: nowayout:Watchdog cannot be stopped once started (default=0) (bool)

echo 3 > /dev/watchdog doesnt reboot system. today i’m out of ideas will be thankful for any help.


#8

Hi Cinerar,

I’ve tried, it doesn’t work. And I update the watchdog driver, you can find patch here.

Whit this dtb and kernel, you should add watchdog module:

root@Khadas:~# echo gxbb_wdt >> /etc/modules

And do the following things:

root@Khadas:~# mkdir -p /var/log/watchdog

Edit /etc/default/watchdog:

root@Khadas:~# cat /etc/default/watchdog 
# Start watchdog at boot time? 0 or 1
run_watchdog=1
# Start wd_keepalive after stopping watchdog? 0 or 1
run_wd_keepalive=1
# Load module before starting watchdog
Watchdog_module=gxbb_wdt
# Specify additional watchdog options here (see manpage).
watchdog_options="-s -v -c /etc/watchdog.conf"

Edit /etc/watchdog.conf:

root@Khadas:~# cat /etc/watchdog.conf
#ping                   = 172.31.14.1
#ping                   = 172.26.1.255
#interface              = eth0
#file                   = /var/log/messages
#change                 = 1407

# Uncomment to enable test. Setting one of these values to '0' disables it.
# These values will hopefully never reboot your machine during normal use
# (if your machine is really hung, the loadavg will go much higher than 25)
#max-load-1             = 24
#max-load-5             = 18
#max-load-15            = 12

# Note that this is the number of pages!
# To get the real size, check how large the pagesize is on your machine.
#min-memory             = 1
#allocatable-memory     = 1

#repair-binary          = /usr/sbin/repair
#repair-timeout         = 
#test-binary            = 
#test-timeout           = 

watchdog-device = /dev/watchdog

# Defaults compiled into the binary
#temperature-device     =
#max-temperature        = 120

# Defaults compiled into the binary
admin                   = root
interval                = 1
logtick                = 1
log-dir         = /var/log/watchdog

# This greatly decreases the chance that watchdog won't be scheduled before
# your machine is really loaded
realtime                = yes
priority                = 1

# Check if rsyslogd is still running by enabling the following line
pidfile         = /var/run/rsyslogd.pid   

watchdog-timeout        = 5

Enable watchdog service, due to a debian bug? Found here.

root@Khadas:~# ln -s  /lib/systemd/system/watchdog.service /etc/systemd/system/multi-user.target.wants/watchdog.service
root@Khadas:~# systemctl enable watchdog.service
root@Khadas:~# systemctl start watchdog.service

Check watchdog daemon, try to kill watchdog daemon:

root@Khadas:~# killall -9 watchdog
root@Khadas:~# killall -9 wd_keepalive

And the system will reboot in 5 seconds(setting in /etc/watchdog.conf).

Try to crash the kernel manually:

root@Khadas:~# echo c > /proc/sysrq-trigger

And the system will reboot in 5 seconds(setting in /etc/watchdog.conf).

But for your python script fork_bomb.py seems not work. Does it will realy block watchdog daemon to feed watchdog?

Thanks.


#9

Thank you numbqq, do i understand it right: this patch is only for 3.14 kernel?


#10

Hi Cinerar,

I only test it on 3.14 now, but I think it can work on 4.9, too. Because the watchdog driver is for 4.14.y. And I will check on 4.9 later.

Thanks.


#11

Hi numbqq, so i tried to port your instructions to 4.9 but unfortunatelly i m not smart enough to do it.
I succesefully compiled gxbb_wdt and inserted it into the kernel.

khadas@Khadas:~$ lsmod
Module                  Size  Used by
gxbb_wdt               16384  31065126
aufs                  258048  30891806
dhd                   888832  30050446
mali                  274432  0

As long as i understand /dev/watchdog should be created after this, but

khadas@Khadas:~$ ls /dev/watchdog
ls: cannot access '/dev/watchdog': No such file or directory

I installed watchdog service and followed yout instructions for configuring it, but this doesn’t help as long as there are no file /dev/watchdog to write to (as long as i understand watchdog service just writes some data to /dev/watchdog and when data is not written after timeout system reboot)

And of course

root@Khadas:~# echo c > /proc/sysrq-trigger

just hangs the system without reboot

Maybe you have some time to take a look at 4.9. Thank you very much for your time.

UPDATE my mistake

Quite unobivious
Not - working:
compatible = "amlogic, meson-gxbb-wdt"
Working
compatible = "amlogic,meson-gxbb-wdt"

Will try to make pull request


#12

Hi Cinerar,

I will check on linux 4.9.


#13

hi numbqq i tried to make pull request https://github.com/numbqq/linux/pull/1 check it please

Sorry, closed it, need to wrap my head around git ) will open it a bit later again

if someone will need it this is dtb with enabled watchdog for khadas vim with kernel 4.9 https://yadi.sk/d/fRONX19z3WQknq replace in /boot/kvim.dtb
also there is driver gxbb_wdt install it using depmod and echo gxbb_wdt >> /etc/modules to autoload

then install watchdog service sudo apt-get install watchdog
and follow instructions from numbqq post Internal HW Watchdog?

Hope watchdog driver will be included by default in future releases. TY numbqq