Linux Performance: Almost Always Add Swap. Part 2: ZRAM

Previously, we looked at how swap space, particularly swapping, can severely slow down Linux performance. We then tweaked Linux kernel parameters to make better use of server memory and to avoid heavy swapping. That article created some debate and good arguments for and against swap space. For example, if you have more than enough memory available, wouldn’t it better to remove or disable swap completely? Is swap needed on VMs or containers with other set limits? etc. You can read the previous article if you have not already: Linux Performance: Why You Should Almost Always Add Swap Space . 

Today, I wanted to take a quick look at ZRAM. Why? Well, because it was something I failed to discuss in the previous article and also I was recently reminded of just how useful ZRAM can be when configured correctly on low-memory systems or in those few scenarios where you intentionally remove swap.

What is ZRAM?

ZRAM creates a block device in RAM where pages that would otherwise be written to swap (disk/ssd) are instead first compressed, then stored. This allows for much faster I/O of swap and also the data compression provides a great amount of memory savings. A downside of ZRAM is that it does use some CPU for compression but this is usually negated by the gains achieved from avoiding disk swap and also by the overall memory savings of compression. So keep those things in mind per your usage.

Why do I need SWAP anyway?

Swap can actually be useful on Linux systems, even if there is still available RAM. This is because the Linux Kernel will move memory pages which are hardly ever used into swap space to ensure that even more space is made available in-memory for more frequently used memory pages. Please note, in recent kernels the swapping algorithm is overhauled to consider active and inactive memory for both file-backed and anonymous (non-file-backed) memory.

Swap only really becomes a performance problem when the Kernel is pressured to continuously move memory pages in and out of memory and swap. Disk/ssd i/o = much slower. Read more about swap and the OOM killer .

Ok, so I’ve enabled SWAP, now what?

Next, consider adjusting your server’s cache pressure and tendency to swap by following the guide below, which is from the previous article: Linux server needs a RAM upgrade? Check with top, free, vmstat and sar :

vfs_cache_pressure – Controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects. (default = 100, recommend values 50 to 500)

swappiness– represents the kernel’s preference (or avoidance) of swap space. Swappiness can have a value between 0 and 100, the default value is 60. A low value causes the kernel to avoid swapping, a higher value causes the kernel to try to use swap space. Using a low value on sufficient memory is known to improve responsiveness on many systems. ( Source: Arch Wiki )

To edit, you can add or replace these lines in /etc/sysctl.conf file. For example:

If you have low memory, you can try something such as:

vm.swappiness=10
vm.vfs_cache_pressure=500

This will increase the cache pressure and that may seem somewhat counter productive since caching is good for performance. However, too frequent swapping reduces your server’s overall performance significantly more. So not keeping as much cache in memory will help make space and thus reduce swap/swapcache activity. Also, with vm.swappiness set to 10 or as low as 1, will reduce disk swapping.

On a healthy server with lots of available memory use the following:

vm.swappiness=10
vm.vfs_cache_pressure=50

This will decrease the cache pressure. Since caching is good for performance, we want to store cached date in-memory longer. Since the cache will grow larger we still want to reduce swapping so that it does not cause increased swap I/O.

Check current values using these commands:

sudo cat /proc/sys/vm/swappiness
sudo cat /proc/sys/vm/vfs_cache_pressure

To enable these settings temporarily without rebooting use the following commands:

sudo sysctl -w vm.swappiness=10
sudo sysctl -w vm.vfs_cache_pressure=50

Why didn’t you suggest ZRAM in the first place?

The previous article focused specifically on swap because I saw a trend of swap being removed unnecessarily. The article served mainly to address why you should almost always add swap space .

Last month, I switched laptops to the Pinebook Pro. It’s taken some optimization and tweaks to be able to go from Intel i7 + 16GB RAM + RAID NVMe’s and fit into a Rockchip RK3399 chip + 4GB RAM and much slower eMMC storage. Right off the bat, memory was the biggest bottleneck. To address this, I had to override a couple default kernel parameter values and setup ZRAM. Also, had to install The Great Suspender , disable composition and animations among other tweaks. I will update that article shortly .

But lets keep this article Linux server related! Basically, using ZRAM ( or ZSWAP ) reminded me of how useful it can be in stabilizing Linux server performance in low-memory situations.

Adjusting Kernel parameters for ZRAM instead of SWAP

On a server with low-memory, after adding ZRAM, you can add the following lines to /etc/sysctl.conf then enable in real-time with the appropriate commands.)

vm.vfs_cache_pressure=500
vm.swappiness=100
vm.dirty_background_ratio=1
vm.dirty_ratio=50

With the settings above you will… delay the inevitable (running out of memory), but start preparing for that sooner , rather than leaving it untill the inevitable happens all at once… heavy swapping.

vm.vfs_cache_pressure=500 – increase cache pressure, which increases the tendency of the kernel to reclaim memory used for caching of directory and inode objects. You will use less memory over a longer period of time. The performance hit is negated by the downside of swapping sooner.

vm.swappiness=100 – Increasing how aggressively the kernel will swap memory pages since we are using ZRAM first.

vm.dirty_background_ratio=1 – background processes will start writing right away when it hits the 1% limit

vm.dirty_ratio=50 –  the system won’t force synchronous I/O until it gets to 50% dirty_ratio.

These four lines – when used with ZRAM – will help performance when know that you will eventually end up requiring more than the installed memory. Thus, knowing that you will swap and that ZRAM makes swapping less expensive and can also store the same data compressed around two to three times smaller, you should begin this swap exchange sooner than later.

The cache pressure helps because we are in effect telling the kernel… “hey, look man, I don’t have any extra memory lying around to use for cache so please get rid of it sooner and only store the most frequently used important data”. Even with reduced caching, if over time we use most of installed memory, the kernel will start opportunistic swapping to ZRAM much sooner so that CPU (used for compression) and swap I/O aren’t delayed until too late/all at once.

Then, as a last resort, you can optionally keep a little swap space. Which looks something like this:

$ sudo cat /proc/swaps
Filename   Type      Size    Used   Priority
/swapfile  file      511996  0      -1
/dev/zram0 partition 2097144 <strong>399320</strong>  5

Or, for servers with less cache pressure, you can encourage even less than the default cache pressure (keep cached data longer) while using ZRAM:

vm.vfs_cache_pressure=50 
vm.swappiness=10 
vm.dirty_background_ratio=1 #increase slowly, up to 10%, if CPU gets too busy with compression. 
vm.dirty_ratio=20 #around 20 to 50 percent is fine.

Can I just completely remove SWAP/ZRAM?

As discussed, swap also provides some benefits to overall system stability and performance. The kernel was designed to work with swap. That said, of course you can, your specific setup and/or requirements in some cases may work best without swap or ZRAM. Red Hat puts nicely :

“Systems without swap can make sense and are supported by Red Hat – just be sure the behavior of such a system under memory pressure is what you want. In most environments, a bit of swap makes sense. Without swap, the system will call the OOM when the memory is exhausted. [Make sure to] …prioritize which processes get killed first in configuring oom_adj_score.”

Conclusion

Using ZRAM, especially in low-memory setup is very useful in allowing your Linux server access to using more of your installed RAM. For example, 1GB of ZRAM amounts to around 2 to 3 GB after compression. Also, in our tweaks we encouraged the kernel to swap to ZRAM more frequently, thus freeing up more usable RAM than possible with swap or with no swap. The possible improvements to stability and performance makes it definitely worth a try . Let me know how it goes!