Building Building EXTRAS
Develop Branch v19.0.0.0 v20.0.0.0 v20.0.0.2 v20.0.0.3
Creating Disk Images Devices m5term Building Linux ARM Kernel Building Android Marshmallow Guest binaries
Memory System gem5 Memory System Replacement Policies Indexing Policies Classic memory system coherence Classic caches
Ruby Cache Coherence Protocols Garnet 2.0 MOESI CMP directory Garnet Synthetic Traffic SLICC MI example Garnet standalone Interconnection network MOESI hammer MOESI CMP token MESI two level Replacement Policies
SimpleCPU O3CPU TraceCPU Minor CPU Model Execution Basics Visualization
Garnet Synthetic Traffic Ruby Random Tester
Trace-based Debugging Debugger-based Debugging Debugging Simulated Code Reporting Problems
Architecture Support ARM Implementation ISA Parser X86 microop ISA
Edit this page

GCN3

Table of Contents

  1. Using the model
  2. ROCm
  3. Documentation and Tutorials

The GCN3 GPU is a model that simulates a GPU at the ISA level, as opposed to the intermediate language level. This page will give you a general overview of how to use this model, the software stack the model uses, and provide resources that detail the model and how it is implemented.

Using the model

The gem5 repository comes with a dockerfile located in util/dockerfiles/gcn-gpu/. This dockerfile contains the drivers and libraries needed to run the GPU model

The gem5-resources repository also comes with a sample application (square) that can be used to verify that the model runs correctly.

Building the image

docker build -t <image_name> .

Building gem5 using the image

The following command assumes the gem5 directory is a subdirectory of your current directory

docker run --rm -v $PWD/gem5:/gem5 -w /gem5 <image_name> scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt

Building a GPU application using the image

The following command assumes the gem5-resources directory is a subdirectory of your current directory

docker run --rm -v $PWD/gem5-resources:/gem5-resources -w /gem5-resources <image_name> make square

Running the sample application

The following command assumes that gem5 and gem5-resources are subdirectories of your current directory

docker run --rm -v $PWD/gem5:/gem5 -v $PWD/gem5-resources:/gem5-resources \
                -w /gem5 <image_name> \
                build/GCN3_X86/gem5.opt configs/example/apu_se.py -n2 \
                --benchmark-root=/gem5-resources/output/test-progs/square \
                -c square

ROCm

The GCN3 model was designed with enough fidelity to not require an emulated runtime. Instead, the GCN3 model uses the Radeon Open Compute platform (ROCm). ROCm is an open platform from AMD that implements Heterogeneous Systems Architecture (HSA) principles. More information about the HSA standard can be found on the HSA Foundation’s website. More information about ROCm can be found on the ROCm website

Simulation support for ROCm

The model currently only works with system-call emulation (SE) mode, therefore all kernel level driver functionality is modeled entirely within the SE mode layer of gem5. In particular, the emulated GPU driver supports the necessary ioctl() commands it receives from the userspace code. The source for the emulated GPU driver can be found in:

The HSA driver code models the basic functionality for an HSA agent, which is any device that can be targeted by the HSA runtime and accepts Architected Query Language (AQL) packets. AQL packets are a standard format for all HSA agents, and are used primarily to initiate kernel launches on the GPU. The base HSADriver class holds a pointer to the HSA packet processor for the device, and defines the interface for any HSA device. An HSA agent does not have to be a GPU, it could be a generic accelerator, CPU, NIC, etc.

The GPUComputeDriver derives from HSADriver and is a device-specific implementation of an HSADriver. It provides the implementation for GPU-specific ioctl() calls.

The src/dev/hsa/kfd_ioctl.h header must match the kfd_ioctl.h header that comes with ROCt. The emulated driver relies on that file to interpret the ioctl() codes the thunk uses.

ROCm toolchain and software stack

The GCN3 model supports ROCm version 1.6

The following ROCm components are required:

The following additional components are used to build and run machine learning programs:

For information about installing these components locally, the commands in the GCN3 dockerfile (util/dockerfiles/gcn-gpu/) can be followed on an ubuntu 16 machine.

Documentation and Tutorials

GCN3 Model

Describes the GCN3 model

gem5 GCN3 ISCA tutorial

Covers information about the GPU architecture, GCN3 ISA and HW-SW interfaces in gem5. Also provides an introduction to ROCm.

GCN3 ISA

ROCm Documentation

Contains further documentation about the ROCm stack, as well as programming guides for using ROCm.

AMDGPU LLVM Information