authors: Jason Lowe-Power
last edited: 2024-03-16 04:10:48 +0000

Interconnection Network

The various components of the interconnection network model inside gem5’s ruby memory system are described here.

How to invoke the network

Simple Network:

./build/<ISA>/gem5.debug \
                      configs/example/ruby_random_test.py \
                      --num-cpus=16  \
                      --num-dirs=16  \
                      --network=simple
                      --topology=Mesh_XY  \
                      --mesh-rows=4

The default network is simple, and the default topology is crossbar.

Garnet network:

./build/<ISA>/gem5.debug \
                      configs/example/ruby_random_test.py  \
                      --num-cpus=16 \
                      --num-dirs=16  \
                      --network=garnet2.0 \
                      --topology=Mesh_XY \
                      --mesh-rows=4

Topology

The connection between the various controllers are specified via python files. All external links (between the controllers and routers) are bi-directional. All internal links (between routers) are uni-directional – this allows a per-direction weight on each link to bias routing decisions.

In each topology, each link and each router can independently be passed a parameter that overrides the defaults (in BasicLink.py and BasicRouter.py):

These two parameters can be used by routers to implement custom routing algorithms in garnet2.0

Routing

Table-based Routing (Default): Based on the topology, shortest path graph traversals are used to populate routing tables at each router/switch. This is done in src/mem/ruby/network/Topology.cc The default routing algorithm is table-based and tries to choose the route with minimum number of link traversals. Links can be given weights in the topology files to model different routing algorithms. For example, in Mesh_XY.py and MeshDirCorners_XY.py Y-direction links are given weights of 2, while X-direction links are given weights of 1, resulting in XY traversals. In Mesh_westfirst.py, the west-links are given weights of 1, and all other links are given weights of 2. In garnet2.0, the routing algorithm randomly chooses between links with equal weights. In simple network, it statically chooses between links with equal weights.

Custom Routing algorithms: In garnet2.0, we provide additional support to implement custom (including adaptive) routing algorithms (See outportComputeXY() in src/mem/ruby/network/garnet2.0/RoutingUnit.cc). The src_outport and dst_inport fields of the links can be used to give custom names to each link (e.g., directions if a mesh), and these can be used inside garnet to implement any routing algorithm. A custom routing algorithm can be selected from the command line by setting –routing-algorithm=2. See configs/network/Network.py and src/mem/ruby/network/garnet2.0/GarnetNetwork.py

Flow-Control and Router Microarchitecture

Ruby supports two network models, Simple and Garnet, which trade-off detailed modeling versus simulation speed respectively.

Simple Network

The default network model in Ruby is the simple network.

Configuration

Simple network uses the generic network parameters in Network.py:

Additional parameters are specified in simple/SimpleNetwork.py:

Switch Model

The simple network models hop-by-hop network traversal, but abstracts out detailed modeling within the switches. The switches are modeled in simple/PerfectSwitch.cc while the links are modeled in simple/Throttle.cc. The flow-control is implemented by monitoring the available buffers and available bandwidth in output links before sending.

Simple_network.jpg

Garnet2.0

Details of the new (2016) Garnet2.0 network are here.

Running the Network with Synthetic Traffic

The interconnection networks can be run in a standalone manner and fed with synthetic traffic. We recommend doing this with garnet2.0.

Running Garnet Standalone with Synthetic Traffic