@HEX-Five, I’m currently trying to port FreeRTOS for running in a Zone on top of MultiZone, but I’m facing one major challenge.
As you should probably know, FreeRTOS is a tick-driven RTOS which needs a tick timer to trigger a synchronous interrupt; however, to the best of my knowledge, the platform I’m currently targeting (SiFive E31) only provides one single timer, i.e. the machine timer.
I took a glance on the MultiZone Manual (v1.01) available on the Hex Five github repository, but I couldn’t find any API which might be able to provide any mechanism for that.
According to the MultiZone demo you have available on github, I’m sure the MultiZone nanokernel is using the machine timer, but I can’t figure out how to use it from the user mode.
Do you have an idea how I can handle it?
The MultiZone Security trusted execution environment does provide secure user-mode timers for each zone. The mechanism for setting the timer and serving the interrupt is absolutely identical to the standard RISC-V mtime / mtimecmp framework. The only difference in that it triggers a synchronous trap 0x3 rather than an asynchronous interrupt 0x7. That’s because SiFive’s implementation of the E300/E31/E51 cores provides only one hardware timer that can only be shared via trap & emulate. The MultiZone implementation for other IP vendors actually supports a number of hardware timers per core. In this case it generates PLIC / CLINT interrupts mapped to the zone via the usual multizone.cfg policy file.
Here is a brief extract of the MultiZone free and open API manual:
void ECALL_CSRR_MTIMECMP(uint64_t)
Secure user-mode emulation of the machine-mode timer compare register (mtimecmp). Causes a trap 0x3 exception when the mtime register contains a value greater than or equal to the value assigned. Each zone has its own secure instance of timer and trap handler. Per RISC-V specs this is a one-shot timer: once set it will execute its callback function only once. Note that mtime and mtimecmp size is 64-bit even on rv32 architecture. Registering the trap 0x3 handler sets the value of mtimecmp to zero to prevent spurious interrupts. If the timer is set but no handler is registered the exception is ignored.
Example: 10ms period free-running timer - see zone1/main.c
Thanks for the note. It’s really nice to see a quick answer and nice support from you.
Just to keep you posted, in the meantime, I have updated the MultiZone version with the one available on the “soft-timer” branch and, after that, I was able to get the timer interrupt on FreeRTOS.
However, as the port continued to progress - I was able to reach a FreeRTOS task and get the ticks - I have identified another three challenges, which I would like to point and get some comments from you:
The first point is related to the lack of information regarding the preempted point of execution when an interrupt or exception is triggered. Let me explain:
When a timer interrupt arrives while FreeRTOS is executing a task, the task is interrupted and the execution flow is redirected to machine mode where the nanokernel will handle the machine timer exception and in some way return to the registered user mode timer handler. At this point, the RTOS will need to have information about the originally preempted point so that it can save it in the running task’s context. I have no mechanism to get this information.
I have noticed that the nanokernel sends a message to the preempted zone own’s mailbox when most exceptions are triggered containing the value of the mcause, mtval and mepc registers. However, in the case of the new timer exception, as in the case of interrupts, I was not able to get such information. I need at least the information regarding the mepc. This is important as the RTOS might perform a context-switch during the exception/interrupt and try to return to a different instruction (of a different task) than the originally preempted (please refer to all APIs available on FreeRTOS ending in xxFromISR - xTaskResumeFromISR() or xQueueSendFromISR()). This leads me to the second issue. I need to have some mechanism that allows the exception handler to return to a different point than the originally presented. A kind of “virtual” epc would be needed and accompanying ecalls to read and write it.
Please let me know if you got my point or if I am missing some features in your implementation that could let me address these issues.
Another point is related to the portYIELD() API available on FreeRTOS. Several FreeRTOS services, such as vTaskDelay() or vTaskSuspend(), makes use of such API to explicitly create a scheduling point. In the current port of FreeRTOS for RISC-V, the portYIELD() is implemented by issuing an ecall - ECALL(PORT_YIELD). This is similar to what happens in Arm architectures using the SWI instruction. As you can imagine, if I issue an ecall I will end trapping into the nanokernel, but as it is currently implemented, it does not provide any support for redirecting the execution to one of the Zone’s synchronous trap handler - ulSynchTrap() in FreeRTOS RISC-V port. While I thought about emulating such feature by issuing an illegal instruction (for example, “csrr t0, uepc” since the E31 does not support the N extension) and then managing it in the illegal instruction handler (by reading the mcause and mtval values by checking the zone own’s mailbox), it seems to me it is more an alternative solution than an appropriate solution. In my opinion, one possible solution might come from some mechanism offered by the nanokernel which might explicitly generate an exception from software, and be able to register a handler for it. Please let me know If you have thought in any alternative solution, or again if I am missing some features in your implementation that could let me address this problem.
The last point is related to critical sections, i.e. the need of ensuring atomic operations while executing some RTOS services. FreeRTOS has a specific macro - taskENTER_CRITICAL(), which calls vPortEnterCritical(), and which ends calling portDISABLE_INTERRUPTS() - for ensuring that specific operations runs without being preempted. Atomic sections ensure for example, that an asynchronous interrupt will not arrive while the OS is context-switching between tasks. This can be critical because in the meantime a new scheduling point can be triggered while another one is currently being performed. Furthermore, in the current port of FreeRTOS for RISC-V, this is accomplished by disabling and enabling interrupts through the mstatus register. Wondering about a possible solution to work around this problem, I realized a possible approach might go through a new MultiZone service (API) which might enable mechanisms to disable only the interrupts of a specific zone (but not of the other zones). Anyway, once again, please let me know If you have thought in an alternative solution, or again if I am missing some features in your implementation that could let me address this problem.
I have the feeling that after overcoming the three aforementioned challenges I will not need much effort to complete the port
Thanks!
PS: Sorry for the extension of the post but I decided to put all my doubts in a row
preempted point of execution when an interrupt or exception is triggered …
MultiZone pushes mepc and mtval into the stack of the zone interrupted. You could use these values to redirect the execution flow within the context of the zone.
Another point is related to the portYIELD() API …
MultiZone provides the ECALL_YIELD() API: Indicates to the nanoKernel scheduler that the zone has nothing pressing to do and causes the nanoKernel to switch context to the next zone.
ensuring atomic operations …
MultiZone provides ECALL_CSRS_MIE() / ECALL_CSRC_MIE() : Secure user-mode emulation of the Machine Status Register (mstatus) MIE bit. Enables / disables all interrupts (PLIC + CLINT) mapped to the zone including the soft timer (trap 0x3). The operation is atomic with respect to the context of the zone.
- I have identified another three challenges, which I would like to point and get some comments from you: