Use ubuntu @VIM1 to build mainline kernel and install and boot it

ravelo · August 18, 2019, 9:20pm

hi,
my next ambitious project is just that:
upgrade the 4.9.179 kernel of the official ubuntu bionic version june 2019 installed on the EMMC of a VIM1 to v5.3 rc4 (or better), and be able to deploy it locally and to boot ubuntu on top of it.

Cross compilation is good enough, but it did not work so well for me and deb packages built by @numbqq have lead to non booting system for me;

So let me try to generate all the needed files to upgrade my system.

I would welcome anyone who would want to give a hand !

ravelo · August 18, 2019, 9:39pm

here are the steps i followed:

deploy

commit aea90d
apt update, dist-upgrade, install lex fakeroot bison

replace defconfig w/ VIM1’s defconfig from fenix 0.7
apply the patch for TEXT_OFFSET
comment out CONFIG_INITRAMFS_SOURCE in defconfig
make -j4 defconfig
make -j4 vmlinux
make -j4 Image
sudo make -j4 modules_prepare
make -j4 modules
make -j4 dtbs
sudo make -j4 bindeb-pkg
sudo make -j4 dtbs_install

after all these, what I get is…
dtb files in /boot/dtbs/5.3.0-rc4/amlogic
and those deb files:

khadas@Khadas:~/linux-narmstrong-5.3-integ-rebase-chewitt$ ls .. -ltr

-rw-r--r--  1 root   root    10938684 Aug 19 04:45 linux-headers-5.3.0-rc4_5.3.0-rc4-1_arm64.deb
-rw-r--r--  1 root   root     1037608 Aug 19 04:45 linux-libc-dev_5.3.0-rc4-1_arm64.deb
-rw-r--r--  1 root   root    12046012 Aug 19 04:46 linux-image-5.3.0-rc4_5.3.0-rc4-1_arm64.deb
-rw-r--r--  1 root   root        5394 Aug 19 04:46 linux-5.3.0-rc4_5.3.0-rc4-1_arm64.buildinfo
-rw-r--r--  1 root   root        1966 Aug 19 04:46 linux-5.3.0-rc4_5.3.0-rc4-1_arm64.changes`

in a few hours, I will try to figure out what to do with all those shiny new files…

ravelo · August 19, 2019, 9:44am

as i was not so confident with the steps before as the result of my simplistic guesswork,
i’ve found https://github.com/umiddelb/armhf/wiki/How-To-compile-a-custom-Linux-kernel-for-your-ARM-device to be another good guide from our fellow @umiddelb and his github has 5.3rc5 source as of now.
i think i will give it a try, and he clearly indicate how to install a freshly compiled image,headers,uInitrd et all…

ravelo · August 19, 2019, 7:39pm

Ok, after many many retries and almost 8 hours, I managed to run all umiddelb (experimental) steps from start to end, on the VIM1 itself, without error (but I did not use his config file but the one from fenix 0.7), now I double check everything and am ready to switch to new kernel and associated files…
I prepared 2 scripts to select either 4.9.179 or 5.3rc5

ravelo · August 19, 2019, 11:18pm

bad news for now; my rc5 kernel does not boot (from EMMC), but at least I’ve learnt how to manually recover from bootlooping or non booting system;
to be continued…

umiddelb · August 20, 2019, 6:15am

Literally the same kernel boots from emmc on the VIM1, VIM2 (and the C2). You can find a precompiled image here. You will need move the Image, dtb and Initrd to a different directory, since I’ve decided to organize /boot in a clean and tidy way.
Do you get an error message when you try to boot from emmc?

ravelo · August 20, 2019, 8:58am

long time no see Uli, good to have you back here, we interacted back in the days when 4.12 was the mainline kernel version…
I stumbled upon your How-To-compile-a-custom-Linux-kernel-for-your-ARM-device by googling around,
I followed it, and I kinda figured out wher does the Makefile put different files;
the best is to dicover and read the explanations from your self in organize /boot in a clean and tidy way; thanks very much.
Regarding the pre-built images,
I’ll download and probably get back to you with some questions once I detared the archive file…

Tschüß

ravelo · August 20, 2019, 10:24am

@umiddelb
when, in u-boot, I do these steps…

setenv kernel_loadaddr "0x11000000"
setenv dtb_loadaddr "0x1000000"
setenv initrd_loadaddr "0x13000000"
setenv env_loadaddr "0x20000000"
setenv boot_start booti ${kernel_loadaddr} ${initrd_loadaddr} ${dtb_loadaddr}
ext4load mmc 1:5 ${kernel_loadaddr} boot/kernel.d/linux-5.3.0-rc5-gx-gfce02bef9/Image
ext4load mmc 1:5 ${initrd_loadaddr} boot/kernel.d/linux-5.3.0-rc5-gx-gfce02bef9/uInitrd
ext4load mmc 1:5 ${dtb_loadaddr} boot/kernel.d/linux-5.3.0-rc5-gx-gfce02bef9/meson-gxl-s905x-khadas-vim.dtb
run boot_start

then I cannot boot the kernel, it is stuck w/these messages :

 kvim#run boot_start
 [rsvmem] get fdtaddr NULL!
 rsvmem - reserve memory
 
 Usage:
 rsvmem check                   - check reserved memory
 rsvmem dump                    - dump reserved memory
 
 rsvmem check failed
 ## Loading init Ramdisk from Legacy Image at 13000000 ...
    Image Name:   initrd-5.3.0-rc5-gx-gfce02bef9.i
    Image Type:   AArch64 Linux RAMDisk Image (gzip compressed)
    Data Size:    14432998 Bytes = 13.8 MiB
    Load Address: 00000000
    Entry Point:  00000000
    Verifying Checksum ... OK
 load dtb from 0x1000000 ......
 ## Flattened Device Tree blob at 01000000
    Booting using the fdt blob at 0x1000000
 libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
 [rsvmem] fdt get prop fail.
    Loading Ramdisk to 730d7000, end 73e9aae6 ... OK
    Loading Device Tree to 000000001fff6000, end 000000001ffff731 ... OK
 fdt_instaboot: no instaboot image

Starting kernel ...

uboot time: 66683731 us
domain-0 init dvfs: 4

Do you have any clue why please ? tks

or anyone else’ response welcome as usual !

EDIT : in facts it starts to boot, but $bootargs was not set and thus console did not print its usual traces on the serial port!
so I set those additionnal vars:

> setenv condev "console=ttyAML0,115200n8 console=tty0 no_console_suspend consoleblank=0"
> setenv rootdev "LABEL=ROOTFS"
> setenv bootargs "root=${rootdev} rootflags=data=writeback rw ${condev} fsck.repair=yes net.ifnames=0 jtag=disable"

before runing booti
and then I could see …

Starting kernel …

uboot time: 645837623 us
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 5.3.0-rc5-gx-gfce02bef9 (khadas@Khadas) (gcc version 7.4.0 (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1)) #2 SMP Tue Aug 20 02:00:52 CST 2019
[    0.000000] Machine model: Khadas VIM
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000063000000, size 256 MiB

etc…
but again, it gets stuck at :
…
[ 6.964267] xor: measuring software checksum speed
[ 7.056444] 8regs : 2716.800 MB/sec
[ 7.156438] 32regs : 3339.600 MB/sec
[ 7.256438] arm64_neon: 2987.600 MB/sec
[ 7.256457] xor: using function: 32regs (3339.600 MB/sec)
[ 7.295203] Btrfs loaded, crc32c=crc32c-generic

maybe a matter of ROOTFS not getting mounted ?

numbqq · August 20, 2019, 12:31pm

Have you connect the HDMI cable? You will find more information from the screen. Have you enter the ramdisk command line?

ravelo · August 20, 2019, 12:54pm

regarding ramdisk : i’ve done this :

No, I usually never use the HDMI port as my distros are always server type (and headless), but yes, I will connect a monitor to see if there are additionnal error message I can see there,
thanks Nick !

EDIT:
I connected an hdmi display screen,
when boot is stuck @BTRFS step,
I just see the blue/white/red/green/yellow khadas logo in the middle of the screen, and no other text!

ravelo · August 20, 2019, 4:14pm

Literally the same kernel boots from emmc on the VIM1, VIM2 (and the C2). You can find a precompiled image here .

(to dowload it directly from vim, I just did dwonload from a browser, then spied what URL was used, then immediately used it from linux command line as below…
wget https://uc377a57820e4eeeb8e7e3d4849f.dl.dropboxusercontent.com/cd/0/get/Am-ss-egrenHPddBsSqTDe6iJrg2uikuQJzu_hAmJ4NyhsA118tm-1E_8o3pFH64_H7mh59V1OE7XJT3V3oak8u2crH2C9b3Yh-aLNHFZpiLBxFBE1rlUgIoJOJxt_Rvv_4/file?_download_id=60016302036795447291145945849575860037332713636024788816254745&_notify_domain=www.dropbox.com&dl=1

renamed the resulting file and tar xvf the result)

how are we supposed to deploy the resulting files on an official 4.9.179 june 2019 ubuntu bionic distro ?
and then how would we boot them ?
tks

umiddelb · August 20, 2019, 7:24pm

I’m using slightly different settings:

setenv kernel_loadaddr "0x01080000"
setenv dtb_loadaddr "0x01000000"
setenv bootargs "console=ttyAML0,115200 earlyprintk=aml-uart,0xc81004c0 root=UUID=e9e72a23-6ddc-4e7b-95ad-49986ea27cb6 rootwait rw fsck.repair=yes no_console_suspend"

You will need to change the UUID setting appropriately.

umiddelb · August 20, 2019, 7:33pm

The idea is to put the kernel image files in separate directories so that you won’t mess up everything beneath /boot. u-boot is able to follow symbolic links while loading files. Therefore I usually create a symbolic link /boot/kernel.d/default pointing to the real kernel directory, e.g. /boot/kernel.d/linux-5.3.0-rc5-gx-gfce02bef9. The u-boot variables and/or boot.scr only refer to /boot/kernel.d/default/Image etc.

ravelo · August 20, 2019, 8:18pm

I’ve just finished rebuilding all files from your repo, this time using your gx_defconfig;
it took me more than 280 minutes to do the Image and modules on the VIM1 and w/ gcc 7.4.0 !
I’ll soon try again to build the symbolic links and uboot macros according to your recommendations;
and then boot the beast on that;

but before using my own files, I’ll try to use your prebuilt ones that I did not try yet.

umiddelb · August 21, 2019, 4:21am

You should better build the kernel on a vim2 or a vim3 or make use of distcc.

What I didn’t mention before: You are free to relocate the kernel image files wherever you want, they are not restricted to be loaded from a specific path. If you don’t want to adopt this /boot/kernel.d/... thing just adjust the corresponding lines within the build script. But if you adopt it, you will get some flexibility, e.g. have installed different kernels in parallel (very useful if you want to test your freshly compiled mainline kernel while having a well known default kernel) or boot different os versions from the same media.

ravelo · August 21, 2019, 6:54am

i usually build my distros and kernels on x86 + SSD ubuntu that are much faster that those arm boards , but i wanted to give it a “purist” try and i’ll switch back to x86 after a 1st success.
The initial idea still was to be able to choose between multiple kernel version, like we can do on x86 with grub and ukuu as both tools seem to be missing in arm linuxes in general…

ravelo · August 21, 2019, 9:35am

latest(bad) news are:

I now better understand what “does not boot” means in my case, in facts, the system boots but fails to access the rootfs as seen in the xtended logs below (I believe I previously could not see it without the earlyprintk=aml-uart,0xc81004c0)

...
[    7.154247] xor: using function: 32regs (3339.600 MB/sec)
[    7.193139] Btrfs loaded, crc32c=crc32c-generic
Scanning for Btrfs filesystems
done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  /dev/mmcblk1 does not exist.  Dropping to a shell!


BusyBox v1.27.2 (Ubuntu 1:1.27.2-2ubuntu3.2) built-in shell (ash)
Enter 'help' for a list of built-in commands.

be it root=LABEL=ROOTFS
or
root=UUID=eb0ce16e…
or
root=/dev/mmcblk1

the result is always the same : I end up into busibox with the message saying
ALERT! [whatever root value as before] does not exist. Dropping to a shell!

@umiddelb, maybe you know how to fix that ?

umiddelb · August 21, 2019, 1:03pm

Could you please post to output of

blkid

after having been dropped into the busybox shell.

ravelo · August 21, 2019, 1:32pm

[    4.777729] meson-drm d0100000.vpu: bound c883a000.hdmi-tx (ops cleanup_module [meson_dw_hdmi])
[    4.785693] [drm] Initialized meson 1.0.0 20161109 for d0100000.vpu on minor 0
[    4.792527] [drm] Cannot find any crtc or sizes
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOTFS does not exist.  Dropping to a shell!


BusyBox v1.27.2 (Ubuntu 1:1.27.2-2ubuntu3.2) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)
(initramfs) blkid
(initramfs) ls /dev/mmcblk*
/dev/mmcblk1rpmb   /dev/mmcblk1boot1  /dev/mmcblk1boot0  /dev/mmcblk1
(initramfs)

This is the trace from your latest precombiled binaries (they also fall into busibox like mine !)
The issue is really the rootfs (which is the same emmc block 1 partition 5 as the bootfs, as per uboot…)

umiddelb · August 21, 2019, 3:40pm

could you please copy the output of

sudo blkid

after you have booted the vim1 with a working kernel?