Using harts 1 to 4 booting from bare metal SD card image



I’m working bare metal and booting from the SD card into my own code. So my code replaces the BBL and this runs hart 0 fine.

I’d like to find a way to use harts 1 to 4 but I can’t see a way. They are sitting in a loop around 0x00010086 in Mask ROM.

Sometimes when I reset, harts 1 to 4 may have their interrupts enabled after reset. If this is the case I can software interrupt from hart 0 and reset hart 1 to 4’s mepc to my code and use whichever hart 1 to 4 whose interrupts were enabled on reset, but it seems arbitrary for any given reset which harts 1 to 4 have interrupts enabled or not. Sometimes they all have interrupts enabled and sometimes none of them.

So, I can’t see any way to reliably get harts 1 to 4 out of Mask ROM and actually use them.

Is there a mechanism which would allow me to do this?


(mara) #2

It’s kind of curious that the harts stay stuck in the mask ROM (especially with interrupts disabled), if so, I wonder how Linux works around this.

Looking at the riscv-pk bbl implementation, it appears that they expect all the harts enter the entry point:

(Bruce Hoult) #3

I didn’t think the mask ROM code was complicated or long enough to have a loop, but I could be wrong.

As a general RISC-V note, there is no guarantee that harts have consecutive IDs (though they are on SiFive hardware, so far). It is required that hart 0 exists, and there is a mechanism to enumerate the others, but the other IDs can be random.

(Jim Sughrue) #4

Hi Andy,

I want to make sure you’ve seen the info in the FU540-C000 Manual:

To use terminology from that manual. I believe you’re seeing that harts 1-4 are stuck in the ZSBL or Zeroth Stage Boot Loader based on the address 0x00010086.

Would you be able to use the boot option MSEL=0001 from table 7 and table 9? This jumps directly to QSPI0, skips ZSBL, and is the method used by freedom-e-sdk for hifive-unleashed.

If you’d prefer to use the ZSBL, the issue you’re having may be the GUID Partition Table. See section 6.2 of the FU540-C000 manual which describes requirements for partition GUID of FSBL.


Thanks all for getting back with some useful info.

I’m hoping to achieve a bare-metal environment where the PLLs and DDR are set up (by the FSBL the docs suggest) and I can access all the harts. I.e. have them running my code.

I don’t care about the ZSBL as such, but since it loads and runs the FSBL which I think I do need, I’d don’t think I’d gain anything by using MSEL=0001.

I think I’m setting the GUID ok as I’m running my code instead of the BBL and this requires its GUID set up.

Currently I’m trying MSEL=1011 and loading the FSBL I got the source for from:

I think I’ve got all the SD card partitions and GUIDs set up ok, but I’m having problems with the FSBL code that’s being executed.

First it had an unrecognised instruction “fssr zero”, so I commented that line out.
Then it hung at the call “smp_resume(s1, s2)” so I commented that line out.
Now it’s returning from “main()” as a PLL test fails:

if ((UX00PRCI_REG(UX00PRCI_COREPLLCFG) ^ pll_default) & lockmask) return (LINE);

So I’m stuck again.

Is there any source available for the actual FSBL in flash? Since it works fine for a normal boot, I’d have hoped it would work for me here.

As I said earlier, I’m hoping to achieve a bare-metal environment where the PLLs and DDR are set up and I can have all the harts running my code.

Any suggestions would be most welcome.


(mara) #6

Strange that the FSBL from github doesn’t work for you, from what I’ve heard some people managed to get it working. It might be a toolchain issue, for example gcc version.

Curious—fssr is an instruction in the -F extension (documented as frcsr). Is it compiling for rv64gc?


Hi Mara

Thanks for getting back.

Please tell me if I’m not understanding something fundamental here, but I thought hart 0 was running the FSBL, and hart 0 is an E51 core with rv64imac architecture, so I can’t see why it would be expected to run fssr, a floating point instruction?

Please help me understand what’s going on here.


(mara) #8

All the harts run the FSBL, at least, the first part until smp_pause(…). But I agree that for the non-F hart it’s not necessary to run a fssr instruction.