VIM3(L) as powerful DSP

Hi Khadas team,
(@numbqq @hyphop @Frank and @Terry )

I’ve started to explore a topic to turn VIM3/VIM3L into powerful signal processor as seems that SBC and Amlogic chip is much better suited for that rather then Raspberry PI4 due onboard NPU and built-in audio-codec.

And it would be much appreciated if you can give some answers and guidance on that long way.

I’m inspired by another projects which really does exists for RasPI (Elk AudioOS https://elk.audio) and GuitarML/NeuralPI https://github.com/GuitarML/NeuralPi

From my perspective if I could make NPU to run the same - it would be really great.

But the target is a bit difficult to reach as there are some steps needs to be taken
1/ Adopt Xenomai and Elk audio project (realtime linux microkernel ) to Amlogic chip
2/ Adapt Amlogic audio drivers to RTDM (realtime driver model) in Xenomai taking other as example - https://github.com/elk-audio/rpi-rtdm-audio-driver
3/ Attach some ADC/DAC hardware to I2S bus.
and etc…

So to just explore possibilities without going into much HW changes - I’m planning to tryout SPDIF IN/OUT ports on VIM3/L SBC and luckily all that available on 40 pin header

SPDIF out is already there, but for SPDIF I wanted to change PWM_F pin into SPDIF IN - could you please let me know what needs to be changed (u-boot, dts, smth else) to change this pin into SPDIF mode ?

Also there is a DAC already on board


Does it only work in balanced mode or can be switched to unbalanced ? Do I need just to get an output signal from TP(3-6) pinounts on a board ?

More questions will follow… :slight_smile:

Many thanks in advance

2 Likes

Here is a patch for linux to enable SPDIF IN on PIN35:

diff --git a/arch/arm64/boot/dts/amlogic/kvim3_linux.dts b/arch/arm64/boot/dts/amlogic/kvim3_linux.dts
index 38d9ec47338d..1dc99e31a50d 100644
--- a/arch/arm64/boot/dts/amlogic/kvim3_linux.dts
+++ b/arch/arm64/boot/dts/amlogic/kvim3_linux.dts
@@ -656,7 +656,7 @@
                                system-clock-frequency = <6144000>;
                        };
                        codec {
-                               sound-dai = <&spdif_codec>;
+                               sound-dai = <&dummy_codec>;
                        };
                };
        };
@@ -1023,7 +1023,7 @@
                interrupt-names = "irq_spdifin";
                pinctrl-names = "spdif_pins",
                                        "spdif_pins_mute";
-               pinctrl-0 = <&spdifout>;
+               pinctrl-0 = <&spdifout &spdifin>;
                pinctrl-1 = <&spdifout_a_mute>;
                status = "okay";
        };
@@ -1232,6 +1232,13 @@
                };
        };
 
+       spdifin: spdifin {
+               mux {/* GPIOH_5 */
+                       groups = "spdif_in_h";
+                       function = "spdif_in";
+               };
+       };
+
        pdmin: pdmin {
                mux { /* gpioa_5, gpioa_6, gpioa_7, gpioa_8, gpioa_9*/
                        groups = "pdm_din0_a",
diff --git a/arch/arm64/boot/dts/amlogic/overlays/kvim3/i2s.dts b/arch/arm64/boot/dts/amlogic/overlays/kvim3/i2s.dts
index 5cf3a162e13e..72b4b0506309 100644
--- a/arch/arm64/boot/dts/amlogic/overlays/kvim3/i2s.dts
+++ b/arch/arm64/boot/dts/amlogic/overlays/kvim3/i2s.dts
@@ -112,7 +112,7 @@
                                        system-clock-frequency = <6144000>;
                                };
                                codec {
-                                       sound-dai = <&spdif_codec>;
+                                       sound-dai = <&dummy_codec>;
                                };
                        };
                };

Check this documentation to upgrade the kernel.

You also need to disable PWM_F, remove pwm_f node in overlays in file /boot/env.txt , and reboot the board.

SPDIF OUT playback:

$ aplay -Dhw:0,4 test.wav

SPDIF IN capture:

$ arecord -Dhw:0,4 -r 48000 -f S16_LE  test.wav
2 Likes

Many thanks for that!

good ,as i think this way may works much better than a market SCOPE.

Dear Khadas Team,

Is it possible to use the SPDIF out GPIO pins with linux mainline kernel also (with VIM3L)?

With 4.9 linux kernel built with Fenix it works fine, but with mainline kernel only sound via hdmi is possible (no sound device with output to spdif is configured). Can this be achived by modifying and compiling the device tree source for the VIM3L board or is there some kind of hardware support still missing in mainline linux?

Many thanks

@hyphop Can you check SDPIF out with mainline kernel?

1 Like

VIM3L same VIM3 VIM2 VIM1

spdif and i2s output for mainline works fine but only with patches

can build with https://github.com/khadas/fenix

or just try build image https://dl.khadas.com/Firmware/Krescue/images/Generic_Ubuntu-minimal-focal_Linux-5.16-rc2_arm64_SD-USB_V1.0.7-211122-develop.img.xz

or install same image it via krescue online scripts

PS: same can ckeck it via Krescue - audio tests (krescue same used mainline kernel source)

1 Like

Thank you both for your quick support. I really appreciate that. :smiley:

Hi @hyphop, thank you for your support. I tried your suggestions, but i have sound over gpio-spdif-pins only with Krescue Audio Tests. Here is what else i have tried:

  • burn the image you linked to sdcard: only sound via hdmi
  • build generic image with fenix, burn to sdcard: only sound via hdmi
  • build vim3l image with fenix (mainline kernel, mainline u-boot), burn to sdcard: only sound via hdmi
  • download linux-mainline 5.16-rc1, patched with patches you linked, configured boot partition of sdcard to use new kernel, new initrd and new dtb.img, which booted successfully, but again: only sound via hdmi

I kindly ask if you have any hint, why sound via spdif-pins is only working in krescue audio tests? :slightly_smiling_face:

Many thanks

you have 3 options

  1. uncomment /* pinctl-0 … */ next blocks - and rebuild dtb in fenix
  2. manualy rebuild by hands dtb just put same pinctl-0 blocks inside right places
  3. creale and applay dt overlays from https://github.com/khadas/khadas-linux-kernel-dt-overlays/
# grep uncomment linux-mainline-5.16-rc2/arch/arm64/boot/dts/amlogic/meson-khadas-vim3.dtsi -B2 -A10

&tdmif_b {
	// uncomment next block  enable I2S output to GPIO HEADERS
	//
	assigned-clocks = <&clkc_audio AUD_CLKID_TDM_SCLK_PAD1>,
			  <&clkc_audio AUD_CLKID_TDM_LRCLK_PAD1>;
	assigned-clock-parents = <&clkc_audio AUD_CLKID_MST_B_SCLK>,
				 <&clkc_audio AUD_CLKID_MST_B_LRCLK>;
	assigned-clock-rates = <0>, <0>;
	/*
	pinctrl-0 = <&tdm_b_fs_pins>, <&tdm_b_dout0_pins>,
		    <&tdm_b_sclk_pins>, <&mclk0_a_pins>;
	*/
--

&spdifout {
	// uncomment next block  enable SPDIF output to GPIO HEADERS
	/*
	pinctrl-0 = <&spdif_ao_out_pins>;
	*/
	pinctrl-names = "default";
	status = "okay";
};

&cec_AO {
	pinctrl-0 = <&cec_ao_a_h_pins>;
	pinctrl-names = "default";

1 Like

Thank you again. I will try this soon.