An Intro to the Linux Memory Access Workload Simulator (masim)

The memory subsystem is one of the most critical areas of the Linux kernel, and consequently, it must be performant, stable and reliable. Over the years, many tools and mechanisms have been developed to help improve the memory subsystem. These include HugePages, NUMA nodes/NUMA emulation, memory compaction, OOM killer, etc.
Another useful tool developed for the memory subsystem is masim, a userspace tool used to simulate intensive memory access workloads to test the behavior and performance of the memory system. It was introduced in 2018 by SeongJae Park.
As of early 2023, it is actively used in generating artificial workloads for testing damo, a userspace tool for DAMON, among other use cases. An important feature of masim is the ability to customize the workloads being generated. As a result, we can use masim to test multiple aspects of the memory.
Installing and Using Masim with a Predefined Config
Now, let us look at how we can use masim. To do that, we have first to download and install it, which can be done as follows:
1 2 3 |
$ git clone https://github.com/sjp38/masim $ cd masim $ make |
This will generate the masim executable. Now we can start generating workloads by either using a predefined config in the configs folder or defining one ourselves. Let’s use one of the predefined configs –100mb.cfg
.
1 2 |
$ ./masim configs/100mb.cfg huge phase 1 : 74868 accesses/msec, 5000 msecs run |
Let’s look at what 100mb.cfg
contains:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
$ cat configs/100mb.cfg #regions # name, length a, 100000000 # phase 1 # name of phase huge phase 1 # time in ms 5000 # access patterns # name of region, randomness, stride, probability a, 1, 64, 80 |
A config file contains multiple sections — regions, phases and access patterns. Let’s dissect each of them:
Regions
Regions define all the memory regions we wish to use for our workload. Each region has a name and size in bytes. The above example defines a single memory region named “a” and size 100 MB.
Phases
In the phases section, we can define the nature of memory accesses to be made by masim. The subsections of each phase are explained as follows:
- Name of phase:
- A user-supplied name of each phase. In the above example, the name is
huge phase 1
.
- A user-supplied name of each phase. In the above example, the name is
- Time:
- The time duration (in milliseconds) for which masim should run this phase.
- Access patterns:
- Here, we specify how a particular memory region should be accessed. In the above example, it is mentioned that masim should access region “a” with randomness set to 1 (0 refers to sequential access), stride of 64 means that there should be a gap of 64 bytes when reading or writing memory (relevant for sequential read/write) and probability refers to the share of total accesses that this particular pattern should get. If another access pattern was defined for this phase with a probability of 40, it would get a smaller share of memory accesses than the pattern defined above.
Defining a Custom Config
Let’s see how we can define our own config. For this example, we’ll be using another feature of masim — HugePages. Masim has the ability to use hugepages, alongside normal pages. Let’s first define a custom config to access 4MB of memory as follows:
1 2 3 4 5 6 7 8 9 10 11 12 |
#regions # name, length a, 4194304 # phase 1 # name of phase huge phase 1 # time in ms 5000 # access patterns # name of region, randomness, stride, probability a, 1, 64, 90 |
Now let’s run this workload with HugePages:
1 2 3 4 5 6 7 8 |
$ ./masim -p -h configs/custom_config.cfg memory regions a: 4194304 bytes Phase (huge phase 1) for 5000 ms Pattern 0 randomly access region a with stride 64 huge phase 1: 86428 accesses/msec, 5000 msecs run |
In the above example, the -h
flag instructs masim to use HugePages and the -p
flag is just used to give detailed output.
Other Important Flags
We can see all the flags that can be passed to masim as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
$ ./masim --help Usage: masim [OPTION...] [config file] Simulate given memory access pattern -d, --dry_run flag for dry run; If this flag is set, the program ends without real access -h, --use_hugetlb use hugepages -p, --pr_config flag for configuration content print -q, --quiet, --silent suppress all normal output -r, --default_rw_mode=<ro|wo|rw> set default read/write mode as this -t, --hint=<hint method> gives access pattern hint to system with given method -?, --help Give this help list --usage Give a short usage message |
We have already seen -h
and -p
, and the flags -d
, -q
and -?
are self-explanatory. Let’s see the other flags.
-r
: Specify the default mode to tune the memory access behavior of masim. We can specify read-only, write-only or read-write.-t
: Here, we can specifymlock
ormadvise
.mlock
prevents memory from being swapped to the swap area, andmadvise
directs the kernel about how the memory will be accessed to optimize the system performance.
Example Test Runs
Let us see some example test runs for the --hint
and --default_rw_mode
flags.
- Using
mlock
with read/write mode
In this example, we are generating a workload to read and write to memory and also use mlock
to make sure that the memory used by the process does not get swapped out.
1 2 3 4 5 6 7 8 9 10 11 12 |
$ ./masim -r rw -t mlock configs/stairs.cfg initial phase: 42961 accesses/msec, 10001 msecs run phase 0: 43262 accesses/msec, 5002 msecs run phase 1: 44188 accesses/msec, 5001 msecs run phase 2: 41742 accesses/msec, 5002 msecs run phase 3: 43122 accesses/msec, 5000 msecs run phase 4: 43594 accesses/msec, 5000 msecs run phase 5: 49676 accesses/msec, 5000 msecs run phase 6: 46549 accesses/msec, 5012 msecs run phase 7: 43498 accesses/msec, 5002 msecs run phase 8: 42170 accesses/msec, 5001 msecs run phase 9: 41418 accesses/msec, 5000 msecs run |
- Using
madvise
with write-only mode
In this example, we are generating a workload to only write to the memory and also use madvise
to direct the kernel about memory access patterns and optimize performance. Masim ensures that the specified memory region in the config file is aligned as per the page size.
1 2 3 4 5 6 7 8 9 10 11 12 |
$ ./masim -r wo -t madvise configs/stairs.cfg initial phase: 67672 accesses/msec, 10000 msecs run phase 0: 69480 accesses/msec, 5001 msecs run phase 1: 43018 accesses/msec, 5003 msecs run phase 2: 32479 accesses/msec, 5004 msecs run phase 3: 32198 accesses/msec, 5003 msecs run phase 4: 32184 accesses/msec, 5001 msecs run phase 5: 32735 accesses/msec, 5001 msecs run phase 6: 37524 accesses/msec, 5002 msecs run phase 7: 43079 accesses/msec, 5002 msecs run phase 8: 49813 accesses/msec, 5002 msecs run phase 9: 55050 accesses/msec, 5000 msecs run |
Summary
To summarize, we learned about what masim is, how it is used, and how it can be customized to generate custom workloads. We also learned about its internal working with the various flags and the config files.