How to add a general AXI device to the freedom project?

Archive Freedom U500
1

Hi,
I want to add a general AXI device to the freedom project. That is to say, the AXI device is a black box, which provides standard slave interface externally. I only need to connect it to the AXI bus and allocate an address space.Here, it is not necessary to assign interruption signals to the AXI equipment, and it is not necessary to generate the equipment tree in the DTS. The related codes are as follows:
1)fpga-shells\src\main\scala\devices\xilinx\xilinxadcvaramserdes\XilinxAdcvaRamSerDes.scala
case class AdcvaRamSerDesParams(baseAddress: BigInt)

class AdcvaRamSerDesPads extends AdcvaRamSerDesIO{
val I_tile_0_ref_clk_p = Input(Clock())
val I_tile_0_ref_clk_n = Input(Clock())
val I_tile_1_ref_clk_p = Input(Clock())
val I_tile_1_ref_clk_n = Input(Clock())
}

class AdcvaRamSerDes(c: AdcvaRamSerDesParams)(implicit p: Parameters, val crossing: ClockCrossingType = AsynchronousCrossing(8)) extends LazyModule with CrossesToOnlyOneClockDomain {
val adapter = LazyModule(new TLWidthWidget(8))
val frag = LazyModule(new TLFragmenter(4, p(CacheBlockBytes), holdFirstDeny = true))
val toaxi4 = LazyModule(new TLToAXI4(adapterName = Some(“ramserdes”)))
// val deint = LazyModule(new AXI4Deinterleaver(p(CacheBlockBytes)))
val index = LazyModule(new AXI4IdIndexer(idBits = 0))
val yank = LazyModule(new AXI4UserYanker(capMaxFlight = Some(1)))
val buffer = LazyModule(new AXI4Buffer)

val axislavenode = AXI4SlaveNode(Seq(AXI4SlavePortParameters(
slaves = Seq(AXI4SlaveParameters(
address = AddressSet.misaligned(c.baseAddress, 0x200000000L),
//resources = device.reg,
regionType = RegionType.UNCACHED,
executable = false,
supportsWrite = TransferSizes(1, 4),
supportsRead = TransferSizes(1, 4),
interleavedId = Some(0))),
beatBytes = 4)
))

this.crossAXI4In(axislavenode) := buffer.node := yank.node := toaxi4.node := frag.node := adapter.node
val node: TLInwardNode = adapter.node

lazy val module = new LazyModuleImp(this) {
// The IO definition is used in Shell.
val io = IO(new AdcvaRamSerDesIO {
val I_tile_0_ref_clk_p = Input(Clock())
val I_tile_0_ref_clk_n = Input(Clock())
val I_tile_1_ref_clk_p = Input(Clock())
val I_tile_1_ref_clk_n = Input(Clock())
val I_sys_rst = Input(Bool())
val I_sys_clk = Input(Clock())
})

val blackbox  = Module(new transceiver_8)

val (axi,_) = axislavenode.in(0)

// connect up IOs
// since we don't have a bajillion, we'll do it by hand
io.O_tile_txp_out              := blackbox.io.O_tile_txp_out
io.O_tile_txn_out              := blackbox.io.O_tile_txn_out
blackbox.io.I_tile_rxp_in := io.I_tile_rxp_in
blackbox.io.I_tile_rxn_in := io.I_tile_rxn_in

2)fpga-shells\src\main\scala\shell\RamSerDesOverlay.scala
abstract class RamSerDesOverlay(val params: RamSerDesOverlayParams)
extends IOOverlay[RamSerDesPads, ModuleValue[RamSerDesPCS]]
{
implicit val p = params.p

def ioFactory = new RamSerDesPads
val designOutput = InModuleBody { Wire(new RamSerDesPCS) }
}

3)fpga-shells\src\main\scala\shell\xilinx\AdcvaRamSerDesOverlay.scala
case class AxiRamSerDesOverlayParams(
address: BigInt,
wrangler: ClockAdapterNode,
corePLL: PLLNode)(
implicit val p: Parameters)

case object AxiRamSerDesOverlayKey extends FieldSeq[DesignOverlay[AxiRamSerDesOverlayParams, TLInwardNode]]

abstract class AdcvaXilinxRamSerDesOverlay[IO <: Data](val params: AxiRamSerDesOverlayParams)
extends IOOverlay[IO, TLInwardNode]
{
implicit val p = params.p
}

4)Codes added in the class DevKitFPGADesign:

p(AxiRamSerDesOverlayKey).headOption.map{
case e => { val ramSerDes = e(AxiRamSerDesOverlayParams(0x480000000L, wranglerNode(0), corePLL))
sbus.coupleFrom(“ram_bus”) { ramSerDes := TLWidthWidget(sbus.beatBytes) := _ }
}
}

Error occurs when compiling the verilog. The information is as follows:
[error] /freedom/src/main/scala/unleashed/DevKitFPGADesign.scala:173:90: overloaded method value := with alternatives:
[error] [EY](h: freechips.rocketchip.diplomacy.OutwardNodeHandle[freechips.rocketchip.tilelink.TLClientPortParameters,freechips.rocketchip.tilelink.TLManagerPortParameters,EY,freechips.rocketchip.tilelink.TLBundle])(implicit p: freechips.rocketchip.config.Parameters, implicit sourceInfo: chisel3.internal.sourceinfo.SourceInfo)freechips.rocketchip.diplomacy.NoHandle
[error] [DX, UX, EX, BX <: Chisel.Data, EY](h: freechips.rocketchip.diplomacy.NodeHandle[DX,UX,EX,BX,freechips.rocketchip.tilelink.TLClientPortParameters,freechips.rocketchip.tilelink.TLManagerPortParameters,EY,freechips.rocketchip.tilelink.TLBundle])(implicit p: freechips.rocketchip.config.Parameters, implicit sourceInfo: chisel3.internal.sourceinfo.SourceInfo)freechips.rocketchip.diplomacy.InwardNodeHandle[DX,UX,EX,BX]
[error] cannot be applied to (freechips.rocketchip.tilelink.TLInwardNode)
[error] sbus.coupleFrom(“ram_bus”) { ramSerDes := TLWidthWidget(sbus.beatBytes) :*= _ }
[error] ^
[error] one error found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 6 s, completed 2020-3-26 15:20:25
common.mk:48: recipe for target ‘/freedom/builds/adcva-u500devkit/sifive.freedom.unleashed.DevKitU500FPGADesign_WithDevKit100MHz.fir’ failed

I modify the code to the following:
val SerDesRamOverlay = p(AxiRamSerDesOverlayKey)
if(SerDesRamOverlay.nonEmpty){
val ramSerDes = SerDesRamOverlay.headOption.map((AxiRamSerDesOverlayParams(0x480000000L, wranglerNode(0), corePLL)))
ramSerDes.zipWithIndex.map { case(ram, i) => {
val name = Some(s"ram$i")
//sbus.fromMaster(name) { ram } // Master Read rocket bus
sbus.toFixedWidthSlave(name) { ram }
}
}}

There is still an error in compilation:
[error] /freedom/src/main/scala/unleashed/DevKitFPGADesign.scala:167:38: type mismatch;
[error] found : freechips.rocketchip.tilelink.TLInwardNode
[error] (which expands to) freechips.rocketchip.diplomacy.InwardNodeHandle[freechips.rocketchip.tilelink.TLClientPortParameters,freechips.rocketchip.tilelink.TLManagerPortParameters,freechips.rocketchip.tilelink.TLEdgeIn,freechips.rocketchip.tilelink.TLBundle]
[error] required: freechips.rocketchip.diplomacy.NodeHandle[freechips.rocketchip.tilelink.TLClientPortParameters,freechips.rocketchip.tilelink.TLManagerPortParameters,freechips.rocketchip.tilelink.TLEdgeIn,freechips.rocketchip.tilelink.TLBundle,?,?,?,?]
[error] sbus.toFixedWidthSlave(name) { ram }
[error] ^
[error] one error found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 6 s, completed 2020-3-26 16:44:39

In this case, there is no need for interruption. How to connect to the AXI bus as a general AXI equipment?
Thank you very much!

2

Could you finally do it? I am trying to attach a simple IP blackbox as an AXI slave but I am not really sure how to proceed. I am using the NVDLA.scala file as a template as it has a AXI master connection assuming it should not be very different.

3

Hi,

May I ask if u have done this ? I’m also trying to add a blackbox module to the project with the gcd example from the chipyard branch, but it still not work…

4

I have finished but I directly wrote my module in Chisel. Anyway, can you tell me what is the error? Maybe I can help you…

5

Thanks for your quick reply! I didn’t use the nvdla template under the freedom project but the gcd example from the shipyard. It also supports chisel module. According to the chipyard tutorial I add the gcd.scala file into an subfolder of the rocket-chip folder, and also modified the ExampleRocketSystem.scala file and the Config.scala file under the system subfolder in order to add the GCD config to the BaseConfig of the rocket-chip. However, after I generated the verilog file, I just cant find my GCD module. So I guess I need to do more to make it run in freedom project. It would be great if you could give me some clues what I have missed?

Thanks a lot!

6

I finally implemented a variation of the GCD instead of the NVDLA. I tried to implement my solution importing my module as a Blackbox but the FIRRTL compiler could not locate it so I was having an error when generating the bitstream, not the Verilog (I do not know why the Verilog generation was not rising an error flag). In the end, I just implemented my Chisel module and that is just fine. Anyway, I remember someone was suggesting to see what is the parent folder of the FIRRTL compiler, maybe that is the error. Try to locate the GCD.scala file under the firrtl folder inside rocket-chip (freedom/rocket-chip/firrtl/GCD.scala). I suggest you to identify what is the parent folder for the FIRRTL compiler and locate the GCD over there. Also, try to post an issue on the https://github.com/sifive/freedom repository, they are responsive and helpful. Just ask that you want to import a blackbox to attach a peripheral on the TileLink bus and that the BlackBox is not locating your Verilog module.

Good luck and let me know if you solve it :slight_smile:

PD: I’ll try to solve it myself when I find some time because I might be interested in using Blackbox-es in my future designs.

7

Actually I didn’t met any errors during the verilog generation but the GCD module is not included in the generated verilog file. If I can solve this problem I will see if the blackbox module works for me. And i didn’t put GCD.scala file into the firrtl folder but just create a new subfolder named “example” under rocket-chip for it. And in chipyard, the GCD.scala file doesn’t exist in firrtl folder…so I guess this should not be the problem. May I ask where did u put the gcd.scala? So just want to confirm, to intergrate the GCD module, I think you didn’t touch/modify the DevKitConfigs.scala or DevKitFGPADesign.scala of the freedom project, right? you just added the GCD module into the Rocket-Chip like me, right?

Thanks for your advice and I will work on it …

8

You have to modify the “DevKitConfigs.scala” and “DevKitFPGADesign.scala” files to instantiate the GCD, otherwise you will not even compile it, thus, you will not have any error (the compiler does not even check your code).

I have included my GCD under the package “package sifive.blocks.devices.gpio”, so I make sure that I am importing this package from “DevKitConfigs.scala” and “DevKitFPGADesign.scala”.

import sifive.blocks.devices.gpio._ //GCD is here

Now you have to modify “DevKitFPGADesign.scala”:

class DevKitFPGADesign(wranglerNode: ClockAdapterNode)(implicit p: Parameters) extends RocketSubsystem
with HasPeripheryMaskROMSlave
with HasPeripheryDebug
with sifive.blocks.devices.gpio.CanHavePeripheryGCD //Write this line

And:

class U500VC707DevKitSystemModule[+L <: DevKitFPGADesign](_outer: L) extends RocketSubsystemModuleImp(_outer)
with HasRTCModuleImp
with HasPeripheryDebugModuleImp
with sifive.blocks.devices.gpio.CanHavePeripheryGCDModuleImp //Write this line

“DevKitConfigs.scala”:

    class FreedomU500Config extends Config(
      new WithJtagDTM            ++
      new WithNMemoryChannels(1) ++
      new sifive.blocks.devices.gpio.WithGCD(useAXI4=true, useBlackBox=true) ++  //Write this line
      new WithNBigCores(4)       ++
      new BaseConfig	     
    )

On the other hand, I was telling you that you can try placing “GCD.scala” under the FIRRTL parent folder. It might not be visible if you place it in the rocket-chip folder. Anyway, I am not 100% sure about this but I remember seeing something related to that. My problem was related to the Blackbox file not being visible by the FIRRTL compiler, and I guess you will have this problem as well if you do not place the Verilog file in the correct folder.

Again, good luck and let me know if you solve it! :slight_smile:

1 Like
9

Hi jalezeta,

Thanks you for your kinkd help I can generate the verilog file with GCD included correctly! And as you mentioned, I also have the error with “can’t find GCD blacbox” when I tried to generate bitstream. And this error can be avoided if I add GCDMMIOBlackBox.v to the makefile of the design. Now I can generate the bitstream successfully!

Thanks again!

10

Sounds great! What do you mean by adding “GCDMMIOBlackBox.v” to the design?
you mean adding a line in the “VSRCS” variable of the “Makefile.vc707-u500devkit”?
Also, did you change anything in the “GCDMMIOBlackBox” function?

class GCDMMIOBlackBox(val w: Int) extends BlackBox(Map("WIDTH" -> IntParam(w))) with HasBlackBoxResource
  with HasGCDIO
{
  addResource("/vsrc/GCDMMIOBlackBox.v")
}
11

yes, I added GCDMMIOBlackBox.c to the VSRCS resources. As I used VCU118, I added the following code to Makefile.vcu118-u500devkit.
VSRCS :=
$(rocketchip_dir)/src/main/resources/vsrc/GCDMMIOBlackBox.v \

I replaced ‘addResource’ in source code with ‘setResource’ because the freedom version I used doesn’t support it.

class GCDMMIOBlackBox(val w: Int) extends BlackBox(Map(“WIDTH” -> IntParam(w))) with HasBlackBoxResource
with HasGCDIO
{
setResource("/vsrc/GCDMMIOBlackBox.v")
}

And I also set an initial value for the parameter ‘WIDTH’ of the verilog module GCDMMIOBlackBox because VIVADO asked for it. like this:

#(parameter WIDTH = 32)

Otherwise, I didn’t touch anything about it.

Hope this also works for u.

12

Yes! thanks @teutonia! Now everything looks just fine!

13

Thanks for detailing your steps! May I ask which branch of the freedom repo you are using? I replicate these steps using the freedom master branch and GCD.scala from the Chipyard master branch as of today. I get the following error on running make -f Makefile.vc707-u500devkit verilog

    [error] /home/dunn/hydra-discrete-freedom/sifive-blocks/src/main/scala/devices/gpio/GCD.scala:101:14: not found: type Reset
    [error] val reset: Reset                                                                                                 
    [error]              ^                                                                                                     
    [error] one error found                                                                                                    
    [error] (sifiveBlocks / Compile / compileIncremental) Compilation failed

I usually associate not found: type errors with mismatched generator versions.

14

I was able to get GCD.scala to build with the Chisel/Rocket associated with the master branch of freedom by importing chisel3.core.{Reset} and removing asBool from impl.io.reset := reset.asBool.