the bandwidth of DDR5 is sufficient and can reach an overall utilization of 80% under multiple master accesses, providing a bandwidth of 51GB * 0.8 = 40GB.
1、The CPU has an insufficient number of outstanding requests (only 16), which cannot hide the overall memory access latency.
2、Due to the lack of vector instructions, the CPU can only process 64 bits (8 bytes) per load/store operation, so memory copy must be implemented using scalar instructions.
3、Currently, the Ubuntu system is not running at the highest frequency (currently at 1.4GHz), but the maximum frequency can reach 1.8GHz.
4、If cacheable copying is used, the proportion of CPU cache prefetching can be increased, but this may affect the performance of other applications during normal operation.
There is currently a Debian-based image that supports drivers for all modules (including video encoding/decoding, NPU, DSP, etc.). It can run at the maximum frequency of 1.8GHz and allows adjustment of cache prefetching based on demand. However, even with these optimizations, the CPU alone cannot fully utilize the DDR bandwidth. The utilization can be improved by running other programs that consume DDR bandwidth, such as playing 4K videos, running a 4K desktop system, gaming, or running AI applications. In these scenarios, the DDR bandwidth utilization can reach approximately 80%.
The current Ubuntu system is not as feature-complete as the Debian system. We will continue to improve the Ubuntu system in the future.
Also you can download the AI-enabled Debian image and user manual from this link:Release Debian-v1.0.0-p550-20241230 · eswincomputing/eic7x-images · GitHub
By the way, I have seen the inference results of LLM. I would like to provide some additional information on the large model inference results that ESWIN supports.Currently, these models are running on the AI-enabled Debian system,
