Linux does not boot after a massive code addition to the bbl

HiFive Unleashed
1

Hi,

I’ve added a security monitor to bbl, but the Linux does not boot even if I comment out every function call that belongs to the security monitor.

At the very beginning of the boot process, the kernel always gets stuck after getting this message:

... (truncated) ...
[    0.000000] software IO TLB [mem 0xfbfff000-0xfffff000] (64MB) mapped at [        (ptrval)-        (ptrval)]
[    0.000000] CPU with hartid=0 has a non-okay status of "masked"
/home/ubuntu/freedom-u-sdk/riscv-pk/machine/mtrap.c:22: machine mode: unhandlable trap 4 @ 0x0000000080002950

trap # 4 is for misaligned load, and 0x80002950 contains “illegal_insn_trap” as follows

3668 0000000080002926 <illegal_insn_trap>:
3669     80002926: 8832                  mv  a6,a2
3670     80002928: 300028f3            csrr  a7,mstatus
3671     8000292c: 34302773            csrr  a4,mbadaddr
3672     80002930: 478d                  li  a5,3
3673     80002932: 00377313            andi  t1,a4,3
3674     80002936: 02f31263            bne t1,a5,8000295a <illegal_insn_trap+0x34>
3675     8000293a: 00480793            addi  a5,a6,4
3676     8000293e: 34179073            csrw  mepc,a5
3677     80002942: 00004697            auipc a3,0x4
3678     80002946: 50868693            addi  a3,a3,1288 # 80006e4a <illegal_insn_trap_table>
3679     8000294a: 07c77793            andi  a5,a4,124
3680     8000294e: 97b6                  add a5,a5,a3
3681     80002950: 0007e303            lwu t1,0(a5)
3682     80002954: 86c6                  mv  a3,a7

But I have no idea why this trap is raised because the security monitor is never executed at all.
Moreover, the trap occurs during booting the kernel, which is not modified.

If I completely remove the security monitor, Linux boots okay.

Do you have any clues for the cause, or how to debug the problem?

Thank you for your help!

2

I think I found the cause myself.

illegal_insn_trap contains following code:

 69 void illegal_insn_trap(uintptr_t* regs, uintptr_t mcause, uintptr_t mepc)
 70 {
 71   asm (".pushsection .rodata\n"
 72        "illegal_insn_trap_table:\n"
...

and later, it dereferences the pointer to call the second handler

145   uint32_t* pf = (void*)illegal_insn_trap_table + (insn & 0x7c);
146   emulation_func f = (emulation_func)(uintptr_t)*pf;

but since I added some stuffs into .rodata section of bbl,
illegal_insn_trap_table got misaligned, causing trap while handling an illegal instruction.

I fixed it by adding “.align 6\n” before the table label.

Let me know if this patch should be applied to the upstream.

Thanks!