Booting with grub

Hi All,

During my experiments with u-boot i noticed that it is able to boot an EFI binary, which made me wonder if i could actually use u-boot to start grub. I usually like grub because it allows me to choose between different kernel versions and configurations.

I started my experiment with plain grub 2.06. It compiles fine for RISC-V. Actual installation in /boot with grub-install requires the use for the --force parameter because it can’t find the efibootmgr, which is ok because it’s not an actual UEFI platform.

I interrupted u-boot in it’s boot sequence and was able to start grub with these commands (nvme 0:3 is my /boot):

pci enum
nvme scan
load nvme 0:3 $kernel_addr_r /EFI/gentoo/grubriscv64.efi
bootefi $kernel_addr_r

After that grub showed it’s usual prompt. It is even able to access my SD card and my SSD. Now here’s the catch: This version doesn’t support the “linux” command (required to load the linux kernel) on RISC-V :frowning:

Then i found this thread on the grub-devel maillinglist:
https://lists.gnu.org/archive/html/grub-devel/2021-03/msg00343.html
These patches implement just the linux command for RISC-V. I added the patches to my grub and it’s partially succesful.

If i just type the regular linux command including all the parameters listed in extlinux.conf the kernel actually boots. Unfortunately a lot of hardware doesn’t work (including the SSD, the SD card does work), i think because of a missing devicetree. Grub has support for devicetrees, but if i load a devicetree i only get this message:

EFI stub: ERROR: /chosen/boot-hartid missing or invalid!

It seems that u-boot adds this to the devicetree and grub doesn’t. As far as i know the boot hart is selected at random during boot, is it possible to set it to a fixed value so i can add it as a fixed entry in the devicetree?

1 Like

Patching grub to simply add the entry to the devicetree was actually simpler than i expected :slight_smile:

This is the patch:

diff -Naur grub-2.06/grub-core/loader/riscv64/linux.c grub-2.06-mod/grub-core/loader/riscv64/linux.c
--- grub-2.06/grub-core/loader/riscv64/linux.c	2021-07-03 19:45:08.797409969 +0200
+++ grub-2.06-mod/grub-core/loader/riscv64/linux.c	2021-07-03 19:43:54.673239976 +0200
@@ -48,6 +48,13 @@
 static grub_addr_t initrd_start;
 static grub_addr_t initrd_end;
 
+#define get_hartid(__v)                        \
+{                                      \
+  __asm__ __volatile__ ("mv %0, tp"    \
+                       : "=r" (__v) :  \
+                       : "memory");    \
+}
+
 grub_err_t
 grub_arch_efi_linux_check_image (struct linux_arch_kernel_header * lh)
 {
@@ -68,6 +75,7 @@
 finalize_params_linux (void)
 {
   int node, retval;
+  int hartid;
 
   void *fdt;
 
@@ -83,6 +91,9 @@
   if (node < 1)
     goto failure;
 
+  get_hartid(hartid);
+  grub_fdt_set_prop32 (fdt, node, "boot-hartid", hartid);
+
   /* Set initrd info */
   if (initrd_start && initrd_end > initrd_start)
     {

After that the kernel booted … and immediately oops’ed when the initscripts tried to mount eftvarsfs. After i removed that from the initscrips it worked :smiley:. My system is now running after being booted via grub.

Now the next question is how to instruct u-boot to execute the required commands to start grub automatically, does anyone have a clue? I tried to add it to extlinux.conf with it’s own label but that doesn’t work.

1 Like

It seems that whatever u-boot does at boot time is hardcoded in include/config_distro_bootcmd.h

If first tries to boot from all partitions with extlinux.conf, and only if that fails it tries a EFI boot. So i’d have to remove extlinux.conf from both the SD card and the SSD before it would try that. I made a small patch to change that order to: first try EFI boot, then try with extlinux.conf. And it works :slight_smile:. My unmatched now automatically boots to grub which presents me with the usual menu over the serial console.

In case anyone is interested, this is the patch:

diff -Naur u-boot-2021.01_rc5/include/config_distro_bootcmd.h u-boot-2021.01_rc5-mod/include/config_distro_bootcmd.h                                            
--- u-boot-2021.01_rc5/include/config_distro_bootcmd.h  2021-07-04 11:32:47.9783
21442 +0200                                                                     
+++ u-boot-2021.01_rc5-mod/include/config_distro_bootcmd.h      2021-07-04 11:34
:05.930534226 +0200                                                             
@@ -462,11 +462,11 @@                                                           
        "scan_dev_for_boot="                                              \     
                "echo Scanning ${devtype} "                               \     
                                "${devnum}:${distro_bootpart}...; "       \     
+               SCAN_DEV_FOR_EFI                                          \     
                "for prefix in ${boot_prefixes}; do "                     \     
                        "run scan_dev_for_extlinux; "                     \     
                        "run scan_dev_for_scripts; "                      \     
                "done;"                                                   \     
-               SCAN_DEV_FOR_EFI                                          \     
                "\0"                                                      \     
        \                                                                       
        "scan_dev_for_boot_part="                                         \   
2 Likes

Hi,

I am the maintainer of the UEFI sub-system in U-Boot.

When OpenSBI calls U-Boot it transfers the boot hart ID in register a0 to U-Boot. This value is stored in gd->arch.boot_hart.

When the bootefi command is executed it fixes up the device-tree and sets the /chose/boot-hartid property in the device-tree.

The current patch series for GRUB to enable booting RISC-V booting into Linux is available here: [PATCH v2 0/7] Add LoadFile2 and riscv Linux loader. There is a pending Linux patch needed: [PATCH 1/1] RISC-V: load initrd wherever it fits into memory - Heinrich Schuchardt

Best regards

Heinrich

1 Like

I found that U-Boot UEFI provide invalid memory map that reports PMP-protected machine-mode memory as EfiBootServicesData that is available to general use after exiting boot services according to UEFI specification. Attempt to use PMP-protected memory in supervisor and user mode will cause CPU exception. I am not sure what exact memory type should be used, probably EfiReservedMemoryType, maybe that address range should be excluded from UEFI memory map.

FDT reserved memory entry:

node('reserved-memory')
    prop('#address-cells'): 2 (len 4)
    prop('#size-cells'): 2 (len 4)
    prop('ranges'):
       (len 0)
    node('mmode_resv0@80000000')
      prop('reg'): (0x80000000, 0x00040000) (len 16)

UEFI memory map:

  phys: 0x80000000, virt: 0x80000000, size: 0x40000, bootServicesData, attrs: 0x8
  phys: 0x80040000, virt: 0x80040000, size: 0x7eb6000, conventionalMemory, attrs: 0x8
  phys: 0x87ef6000, virt: 0x87ef6000, size: 0x14000, ACPIReclaimMemory, attrs: 0x8
  phys: 0x87f0a000, virt: 0x87f0a000, size: 0x741cb000, conventionalMemory, attrs: 0x8
  phys: 0xfc0d5000, virt: 0xfc0d5000, size: 0x25e9000, loaderData, attrs: 0x8
  phys: 0xfe6be000, virt: 0xfe6be000, size: 0x59000, loaderCode, attrs: 0x8
  phys: 0xfe717000, virt: 0xfe717000, size: 0x7000, reservedMemoryType, attrs: 0x8
  phys: 0xfe71e000, virt: 0xfe71e000, size: 0x1000, bootServicesData, attrs: 0x8
  phys: 0xfe71f000, virt: 0xfe71f000, size: 0x1000, runtimeServicesData, attrs: 0x8000000000000008
  phys: 0xfe720000, virt: 0xfe720000, size: 0x2000, bootServicesData, attrs: 0x8
  phys: 0xfe722000, virt: 0xfe722000, size: 0x1000, reservedMemoryType, attrs: 0x8
  phys: 0xfe723000, virt: 0xfe723000, size: 0x3000, runtimeServicesData, attrs: 0x8000000000000008
  phys: 0xfe726000, virt: 0xfe726000, size: 0x1000, bootServicesData, attrs: 0x8
  phys: 0xfe727000, virt: 0xfe727000, size: 0x4000, runtimeServicesData, attrs: 0x8000000000000008
  phys: 0xfe72b000, virt: 0xfe72b000, size: 0x1000, reservedMemoryType, attrs: 0x8
  phys: 0xfe72c000, virt: 0xfe72c000, size: 0x1000, bootServicesData, attrs: 0x8
  phys: 0xfe72d000, virt: 0xfe72d000, size: 0x1000, reservedMemoryType, attrs: 0x8
  phys: 0xfe72e000, virt: 0xfe72e000, size: 0x2000, bootServicesData, attrs: 0x8
  phys: 0xfe730000, virt: 0xfe730000, size: 0x2000, reservedMemoryType, attrs: 0x8
  phys: 0xfe732000, virt: 0xfe732000, size: 0x1000, bootServicesData, attrs: 0x8
  phys: 0xfe733000, virt: 0xfe733000, size: 0x1000, reservedMemoryType, attrs: 0x8
  phys: 0xfe734000, virt: 0xfe734000, size: 0x1000, bootServicesData, attrs: 0x8
  phys: 0xfe735000, virt: 0xfe735000, size: 0x1000, reservedMemoryType, attrs: 0x8
  phys: 0xfe736000, virt: 0xfe736000, size: 0x1000, bootServicesData, attrs: 0x8
  phys: 0xfe737000, virt: 0xfe737000, size: 0x1000, reservedMemoryType, attrs: 0x8
  phys: 0xfe738000, virt: 0xfe738000, size: 0x2000, bootServicesData, attrs: 0x8
  phys: 0xfe73a000, virt: 0xfe73a000, size: 0x1000, reservedMemoryType, attrs: 0x8
  phys: 0xfe73b000, virt: 0xfe73b000, size: 0x2000, bootServicesData, attrs: 0x8
  phys: 0xfe73d000, virt: 0xfe73d000, size: 0x1826000, loaderData, attrs: 0x8
  phys: 0xfff63000, virt: 0xfff63000, size: 0x1000, runtimeServicesCode, attrs: 0x8000000000000008
  phys: 0xfff64000, virt: 0xfff64000, size: 0x9c000, loaderData, attrs: 0x8
  phys: 0x100000000, virt: 0x100000000, size: 0x380000000, bootServicesData, attrs: 0x8

Where I can report bug?

@x512:

Please, have a look at devicetree-specification/chapter3-devicenodes.rst at master · devicetree-org/devicetree-specification · GitHub :

" Reserved regions with the no-map property must be listed in the memory map with type EfiReservedMemoryType . All other reserved regions must be listed with type EfiBootServicesData ."

U-Boot is doing exactly this.

It is Linux (or Haiku OS) that will know that it cannot use reserved-memory unrestrictedly. If you want to avoid speculative access, you should use the no-map property in the device-tree.

Best regards

Heinrich

But it is listed as EfiBootServicesData:

phys: 0x80000000, virt: 0x80000000, size: 0x40000, bootServicesData, attrs: 0x8

Currently there are no way to identify what memory blocks with EfiBootServicesData can be used by OS without reading FDT. FDT is not mentioned in UEFI specification. Provided memory map is taken from physical HiFive Unmatched board.

UEFI spec says:

Table 30. Memory Type Usage after ExitBootServices()
...
EfiBootServicesData - Memory available for general use.

Most of free RAM is also listed as EfiBootServicesData so ignoring it is a bad idea:

phys: 0x100000000, virt: 0x100000000, size: 0x380000000, bootServicesData, attrs: 0x8

There are no no-map property in FDT DTB table provided by UEFI (GUID: {B1B621D5-F19C-41A5-830B-D9152C69AAE0}). I use standard SD card provided with board without any modifications. UEFI boot loader is located at USB flash drive.

  node('reserved-memory')
    prop('#address-cells'): 2 (len 4)
    prop('#size-cells'): 2 (len 4)
    prop('ranges'):
       (len 0)
    node('mmode_resv0@80000000')
      prop('reg'): (0x80000000, 0x00040000) (len 16)

Thanks for joining @xypron.

I am aware there is a more recent patchset to add RISC-V support for the linux command to grub. From the replies to that patchset i got the impression it doesn’t work yet, so that why i’m still using the old patches.
If i’m not mistaken the new patchset also doesn’t set the /chosen/boot-hartid property in the devicetree.

/chosen/boot-hartid is set by U-Boot not by GRUB.

arch/riscv/lib/fdt_fixup.c:161: err = fdt_setprop_u32(blob, chosen_offset, “boot-hartid”,
arch/riscv/lib/fdt_fixup.c-162- gd->arch.boot_hart);

I know it’s set by u-boot, i’ve seen the code :slight_smile:
My point is that grub also has the ability to load a devicetree, and i think that means it should also add the boot-hartid in case it’s not present.

Updating the boot hart is not the only dtb fixup that is needed. I have implemented in U-Boot a protocol that could be called by GRUB and have further created GRUB patches for it. See

Please, mail Daniel Kieper and the GRUB-devel mailing list to endorse the suggestion.

Best regards

Heinrich

1 Like