Khadas are you able to ask Amlogic what is proper pci-e configuration values so we can properly setup big GPU-s?

VIM3
1

Hello Khadas
are you able to ask Amlogic what is proper pci-e configuration so we can use big gpus ?
I am talking about this
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi?h=v5.13.5#n143
“reg” and “ranges”

as with current setup pci-e can’t alocate bar with bigger size then 0x1900000 bytes
so we don’t have enought bars
[ 1.971463][ T7] meson-pcie fc000000.pcie: host bridge /soc/pcie@fc000000 ranges:
[ 1.976985][ T7] meson-pcie fc000000.pcie: IO 0x00fc600000…0x00fc6fffff -> 0x0000000000
[ 1.985772][ T7] meson-pcie fc000000.pcie: MEM 0x00fc700000…0x00fdffffff -> 0x00fc700000
[ 1.994653][ T7] meson-pcie fc000000.pcie: invalid resource
[ 2.000357][ T7] meson-pcie fc000000.pcie: iATU unroll: enabled
[ 2.006469][ T7] meson-pcie fc000000.pcie: Detected iATU regions: 4 outbound, 4 inbound
[ 2.055699][ T7] meson-pcie fc000000.pcie: error: wait linkup timeout
[ 2.086047][ T7] meson-pcie fc000000.pcie: Link up
[ 2.086136][ T7] meson-pcie fc000000.pcie: PCI host bridge to bus 0000:00
[ 2.092466][ T7] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 2.098573][ T7] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff]
[ 2.105472][ T7] pci_bus 0000:00: root bus resource [mem 0xfc700000-0xfdffffff]
[ 2.112997][ T7] pci 0000:00:00.0: [16c3:abcd] type 01 class 0x060400
[ 2.119622][ T7] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[ 2.126969][ T7] pci 0000:00:00.0: supports D1
[ 2.131601][ T7] pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold
[ 2.140767][ T7] pci 0000:01:00.0: [1002:1478] type 01 class 0x060400
[ 2.145270][ T7] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff]
[ 2.152495][ T7] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 2.159024][ T7] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x1 link at 0000:00:00.0 (capable of 126.024 Gb/s with 16.0 GT/s PCIe x8 link)
[ 2.199277][ T7] pci 0000:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 2.202594][ T7] pci 0000:02:00.0: [1002:1479] type 01 class 0x060400
[ 2.209405][ T7] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[ 2.218034][ T7] pci 0000:02:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 2.224618][ T7] pci 0000:03:00.0: [1002:7340] type 00 class 0x030000
[ 2.231010][ T7] pci 0000:03:00.0: reg 0x10: [mem 0x00000000-0x0fffffff 64bit pref]
[ 2.238832][ T7] pci 0000:03:00.0: reg 0x18: [mem 0x00000000-0x001fffff 64bit pref]
[ 2.246666][ T7] pci 0000:03:00.0: reg 0x20: [io 0x0000-0x00ff]
[ 2.252875][ T7] pci 0000:03:00.0: reg 0x24: [mem 0x00000000-0x0007ffff]
[ 2.259771][ T7] pci 0000:03:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
[ 2.267410][ T7] pci 0000:03:00.0: PME# supported from D1 D2 D3hot D3cold
[ 2.274254][ T7] pci 0000:03:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x1 link at 0000:00:00.0 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
[ 2.289960][ T7] pci 0000:03:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[ 2.298921][ T7] pci 0000:03:00.1: [1002:ab38] type 00 class 0x040300
[ 2.305508][ T7] pci 0000:03:00.1: reg 0x10: [mem 0x00000000-0x00003fff]
[ 2.312647][ T7] pci 0000:03:00.1: PME# supported from D1 D2 D3hot D3cold
[ 2.321621][ T7] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03
[ 2.326616][ T7] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 03
[ 2.333885][ T7] pci 0000:00:00.0: BAR 9: no space for [mem size 0x18000000 pref]
[ 2.341525][ T7] pci 0000:00:00.0: BAR 9: failed to assign [mem size 0x18000000 pref]
[ 2.349547][ T7] pci 0000:00:00.0: BAR 8: assigned [mem 0xfc700000-0xfc8fffff]
[ 2.356964][ T7] pci 0000:00:00.0: BAR 6: assigned [mem 0xfc900000-0xfc90ffff pref]
[ 2.364814][ T7] pci 0000:00:00.0: BAR 7: assigned [io 0x1000-0x1fff]
[ 2.371562][ T7] pci 0000:01:00.0: BAR 9: no space for [mem size 0x18000000 64bit pref]
[ 2.379745][ T7] pci 0000:01:00.0: BAR 9: failed to assign [mem size 0x18000000 64bit pref]
[ 2.388274][ T7] pci 0000:01:00.0: BAR 8: assigned [mem 0xfc700000-0xfc7fffff]
[ 2.395691][ T7] pci 0000:01:00.0: BAR 0: assigned [mem 0xfc800000-0xfc803fff]
[ 2.403115][ T7] pci 0000:01:00.0: BAR 7: assigned [io 0x1000-0x1fff]
[ 2.409842][ T7] pci 0000:02:00.0: BAR 9: no space for [mem size 0x18000000 64bit pref]
[ 2.418030][ T7] pci 0000:02:00.0: BAR 9: failed to assign [mem size 0x18000000 64bit pref]
[ 2.426578][ T7] pci 0000:02:00.0: BAR 8: assigned [mem 0xfc700000-0xfc7fffff]
[ 2.433988][ T7] pci 0000:02:00.0: BAR 7: assigned [io 0x1000-0x1fff]
[ 2.440717][ T7] pci 0000:03:00.0: BAR 0: no space for [mem size 0x10000000 64bit pref]
[ 2.448906][ T7] pci 0000:03:00.0: BAR 0: failed to assign [mem size 0x10000000 64bit pref]
[ 2.457446][ T7] pci 0000:03:00.0: BAR 2: no space for [mem size 0x00200000 64bit pref]
[ 2.465637][ T7] pci 0000:03:00.0: BAR 2: failed to assign [mem size 0x00200000 64bit pref]
[ 2.474178][ T7] pci 0000:03:00.0: BAR 5: assigned [mem 0xfc700000-0xfc77ffff]
[ 2.481601][ T7] pci 0000:03:00.0: BAR 6: assigned [mem 0xfc780000-0xfc79ffff pref]
[ 2.489443][ T7] pci 0000:03:00.1: BAR 0: assigned [mem 0xfc7a0000-0xfc7a3fff]
[ 2.496867][ T7] pci 0000:03:00.0: BAR 4: assigned [io 0x1000-0x10ff]
[ 2.503595][ T7] pci 0000:02:00.0: PCI bridge to [bus 03]
[ 2.509204][ T7] pci 0000:02:00.0: bridge window [io 0x1000-0x1fff]
[ 2.515937][ T7] pci 0000:02:00.0: bridge window [mem 0xfc700000-0xfc7fffff]
[ 2.523358][ T7] pci 0000:01:00.0: PCI bridge to [bus 02-03]
[ 2.529208][ T7] pci 0000:01:00.0: bridge window [io 0x1000-0x1fff]
[ 2.535941][ T7] pci 0000:01:00.0: bridge window [mem 0xfc700000-0xfc7fffff]
[ 2.543367][ T7] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 2.549220][ T7] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff]
[ 2.555946][ T7] pci 0000:00:00.0: bridge window [mem 0xfc700000-0xfc8fffff]

[ 575.252782][ T2110] [drm] amdgpu kernel modesetting enabled.
[ 575.252934][ T2110] checking generic (7f807000 7e9000) vs hw (fc700000 80000)
[ 575.259966][ T2110] amdgpu 0000:03:00.0: enabling device (0000 -> 0003)
[ 575.266453][ T2110] [drm] initializing kernel modesetting (NAVI14 0x1002:0x7340 0x1DA2:0xE421 0xC5).
[ 575.275495][ T2110] amdgpu 0000:03:00.0: amdgpu: Trusted Memory Zone (TMZ) feature disabled as experimental (default)
[ 575.286027][ T2110] [drm] register mmio base: 0xFC700000
[ 575.291276][ T2110] [drm] register mmio size: 524288
[ 575.296204][ T2110] [drm] PCIE atomic ops is not supported
[ 575.302799][ T2110] [drm] add ip block number 0 <nv_common>
[ 575.307144][ T2110] [drm] add ip block number 1 <gmc_v10_0>
[ 575.312666][ T2110] [drm] add ip block number 2 <navi10_ih>
[ 575.318186][ T2110] [drm] add ip block number 3
[ 575.323189][ T2110] [drm] add ip block number 4
[ 575.328191][ T2110] [drm] add ip block number 5 <gfx_v10_0>
[ 575.333712][ T2110] [drm] add ip block number 6 <sdma_v5_0>
[ 575.339232][ T2110] [drm] add ip block number 7 <vcn_v2_0>
[ 575.344665][ T2110] [drm] add ip block number 8 <jpeg_v2_0>
[ 575.383540][ T2110] amdgpu 0000:03:00.0: amdgpu: Fetched VBIOS from ROM BAR
[ 575.384847][ T2110] amdgpu: ATOM BIOS: 113-E4210SM-U03
[ 575.389956][ T2110] [drm] VCN decode is enabled in VM mode
[ 575.395373][ T2110] [drm] VCN encode is enabled in VM mode
[ 575.400806][ T2110] [drm] JPEG decode is enabled in VM mode
[ 575.406335][ T2110] [drm] GPU posting now…
[ 575.410607][ T2110] [drm] vm size is 1 GB, 2 levels, block size is 9-bit, fragment size is 9-bit
[ 575.419272][ T2110] amdgpu 0000:03:00.0: amdgpu: VRAM: 8176M 0x0000008000000000 - 0x00000081FEFFFFFF (128M used)
[ 575.429357][ T2110] amdgpu 0000:03:00.0: amdgpu: GART: 32M 0x0000000000000000 - 0x0000000001FFFFFF
[ 575.438238][ T2110] amdgpu 0000:03:00.0: amdgpu: AGP: 267894784M 0x0000008400000000 - 0x0000FFFFFFFFFFFF
[ 575.447641][ T2110] [drm] Detected VRAM RAM=8176M, BAR=0M
[ 575.452987][ T2110] [drm] RAM width 128bits GDDR6
[ 575.457711][ T2110] [drm] amdgpu: 128M of VRAM memory ready
[ 575.463170][ T2110] [drm] amdgpu: 1482M of GTT memory ready.
[ 575.468774][ T2110] [drm] GART: num cpu pages 8192, num gpu pages 8192
[ 575.475252][ T2110] amdgpu 0000:03:00.0: amdgpu: (-22) kernel bo map failed
[ 575.482149][ T2110] [drm:amdgpu_device_init [amdgpu]] ERROR amdgpu_vram_scratch_init failed -22
[ 575.490942][ T2110] amdgpu 0000:03:00.0: amdgpu: amdgpu_device_ip_init failed
[ 575.498019][ T2110] amdgpu 0000:03:00.0: amdgpu: Fatal error during GPU init
[ 575.505062][ T2110] amdgpu: probe of 0000:03:00.0 failed with error -22

2

pci have only 1line -

vailable PCIe bandwidth, limited by 5.0 GT/s PCIe x1

plz check card spec!

but may be its not problem

3

num-lanes = <1>;
so it is ok

4

pcie

pcie: pcie@fc000000 {
compatible = “amlogic,g12a-pcie”, “snps,dw-pcie”;
reg = <0x0 0xfc000000 0x0 0x400000>,
<0x0 0xff648000 0x0 0x2000>,
<0x0 0xfc400000 0x0 0x200000>;
reg-names = “elbi”, “cfg”, “config”;
interrupts = <GIC_SPI 221 IRQ_TYPE_LEVEL_HIGH>;
#interrupt-cells = <1>;
interrupt-map-mask = <0 0 0 0>;
interrupt-map = <0 0 0 0 &gic GIC_SPI 223 IRQ_TYPE_LEVEL_HIGH>;
bus-range = <0x0 0xff>;
#address-cells = <3>;
#size-cells = <2>;
device_type = “pci”;
ranges = <0x81000000 0 0 0x0 0xfc600000 0 0x00100000>,
<0x82000000 0 0xfc700000 0x0 0xfc700000 0 0x1900000>;

  	clocks = <&clkc CLKID_PCIE_PHY
  		  &clkc CLKID_PCIE_COMB
  		  &clkc CLKID_PCIE_PLL>;
  	clock-names = "general",
  		      "pclk",
  		      "port";
  	resets = <&reset RESET_PCIE_CTRL_A>,
  		 <&reset RESET_PCIE_APB>;
  	reset-names = "port",
  		      "apb";
  	num-lanes = <1>;
  	phys = <&usb3_pcie_phy PHY_TYPE_PCIE>;
  	phy-names = "pcie";
  	status = "disabled";
  };