This package contains two parallel task libraries for running gem5 experiments.
The actual gem5 experiment can be executed with the help of Python multiprocessing support, Celery or even without using any job manager (a job can be directly launched by calling
run() function of gem5Run object).
This package implicitly depends on the gem5art run package.
Use of Python Multiprocessing
This is a simple way to run gem5 jobs using Python multiprocessing library. You can use the following function in your job launch script to execute gem5art run objects:
run_job_pool([a list containing all run objects to execute], num_parallel_jobs = [Number of parallel jobs])
Use of Celery
Celery server can run many gem5 tasks asynchronously.
Once a user creates a gem5Run object (discussed previously) while using gem5art, this object needs to be passed to a method
run_gem5_instance() registered with Celery app, which is responsible for starting a Celery task to run gem5. The other argument needed by the
run_gem5_instance() is the current working directory.
Celery server can be started with the following command:
celery -E -A gem5art.tasks.celery worker --autoscale=[number of workers],0
This will start a server with events enabled that will accept gem5 tasks as defined in gem5art. It will autoscale from 0 to desired number of workers.
Celery relies on a message broker
RabbitMQ for communication between the client and workers.
If not already installed, you need to install
RabbitMQ on your system (before running celery) using:
apt-get install rabbitmq-server
Celery does not explicitly show the status of the runs by default. flower, a Python package, is a web-based tool for monitoring and administrating Celery.
To install the flower package,
pip install flower
You can monitor the celery cluster doing the following:
flower -A gem5art.tasks.celery --port=5555
This will start a webserver on port 5555.
Removing all tasks
celery -A gem5art.tasks.celery purge
Viewing state of all jobs in celery
celery -A gem5art.tasks.celery events