The pieces of Fedora Silverblue

Fedora Silverblue provides a useful workstation build on an immutable operating system. In “What is Silverblue?“, you learned about the benefits that an immutable OS provides. But what pieces go into making it? This article examines some of the technology that powers Silverblue.

The filesystem

Fedora Workstation users may find the idea of an immutable OS to be the most brain-melting part of Silverblue. What does that mean? Find some answers by taking a look at the filesystem.

At first glance, the layout looks pretty much the same as a regular Fedora file system. It has some differences, like making /home a symbolic link to /var/home . And you can get more answers by looking at how libostree works. libostree treats the whole tree like it’s an object, checks it into a code repository, and checks out a copy for your machine to use.

libostree

The libostree project supplies the goods for managing Silverblue’s file system. It is an upgrade system that the user can control using rpm-ostree commands .

libostree knows nothing about packages—an upgrade means replacing one complete file system with another complete file system. libostree treats the file system tree as one atomic object (an unbreakable unit). In fact, the forerunner to Silverblue was named Project Atomic .

The libostree project provides a library and set of tools. It’s an upgrade system that carries out these tasks.

1. Pull in a new file system
2. Store the new file system
3. Deploy the new file system

Pull in a new file system

Pulling in a new file system means copying an object (the entire file system) from a remote source to its own store. If you’ve worked with virtual machine image files, you already understand the concept of a file system object that you can copy.

Store the new file system

The libostree store has some source code control qualities—it stores many file system objects, and checks one out to be used as the root file system. libostree’s store has two parts:

• a repository database at /sysroot/ostree/repo/
• file systems in /sysroot/ostree/deploy/fedora/deploy/

libostree keeps track of what’s been checked in using commit IDs. Each commit ID can be found in a directory name, nested deep inside /sysroot .A libostree commit ID is a long checksum, and looks similar to a git commit ID.

$ <strong>ls -d /sysroot/ostree/deploy/fedora/deploy/*/</strong>
/sysroot/ostree/deploy/fedora/deploy/c4bf7a6339e6be97d0ca48a117a1a35c9c5e3256ae2db9e706b0147c5845fac4.0/

rpm-ostree status gives a little more information about that commit ID. The output is a little confusing; it can take a while to see this file system is Fedora 31.

$ <strong>rpm-ostree status</strong>
State: idle
AutomaticUpdates: disabled
Deployments:
● ostree://fedora:fedora/31/x86_64/silverblue
                   Version: 31.1.9 (2019-10-23T21:44:48Z)
                    Commit: c4bf7a6339e6be97d0ca48a117a1a35c9c5e3256ae2db9e706b0147c5845fac4
              GPGSignature: Valid signature by 7D22D5867F2A4236474BF7B850CB390B3C3359C4

Deploy the new filesystem

libostree deploys a new file system by checking out the new object from its store. libostree doesn’t check out a file system by copying all the files—it uses hard links instead. If you look inside the commit ID directory, you see something that looks suspiciously like the root directory. That’s because it is the root directory. You can see these two directories are pointing to the same place by checking their inodes.

$ <strong>ls -di1 / /sysroot/ostree/deploy/fedora/deploy/*/</strong>
260102 /
260102 /sysroot/ostree/deploy/fedora/deploy/c4bf7a6339e6be97d0ca48a117a1a35c9c5e3256ae2db9e706b0147c5845fac4.0/

This is a fresh install, so there’s only one commit ID. After a system update, there will be two. If more copies of the file system are checked into libostree’s repo, more commit IDs appear here.

Upgrade process

Putting the pieces together, the update process looks like this:

1. libostree checks out a copy of the file system object from the repository
2. DNF installs packages into the copy
3. libostree checks in the copy as a new object
4. libostree checks out the copy to become the new file system
5. You reboot to pick up the new system files

In addition to more safety, there is more flexibility. You can do new things with libostree’s repo, like store a few different file systems and check out whichever one you feel like using.

Silverblue’s root file system

Fedora keeps its system files in all the usual Linux places, such as /boot for boot files, /etc for configuration files, and /home for user home directories. The root directory in Silverblue looks much like the root directory in traditional Fedora, but there are some differences.

• The filesystem has been checked out by libostree
• Some directories are now symbolic links to new locations. For example, /home is a symbolic link to /var/home
• /usr is a read-only directory
• There’s a new directory named /sysroot . This is libostree’s new home

Juggling file systems

You can store many file systems and switch between them. This is called rebasing , and it’s similar to git rebasing. In fact, upgrading Silverblue to the next Fedora version is not a big package install—it’s a pull from a remote repository and a rebase.

You could store three copies with three different desktops: one KDE, one GNOME, and one XFCE. Or three different OS versions: how about keeping the current version, the nightly build, and an old classic? Switching between them is a matter of rebasing to the appropriate file system object.

Rebasing is also how you upgrade from one Fedora release to the next. See “ How to rebase to Fedora 32 on Silverblue ” for more information.

Flatpak

The Flatpak project provides a way of installing applications like LibreOffice. Applications are pulled from remote repositories like Flathub . It’s a kind of package manager, although you won’t find the word package in the docs . Traditional Fedora variants like Fedora Workstation can also use Flatpak, but the sandboxed nature of flatpaks make it particularly good for Silverblue. This way you do not have to do the entire ostree update process every time you wish to install an application.

Flatpak is well-suited to desktop applications, but also works for command line applications. You can install the vim editor with the command flatpak install flathub org.vim.Vim and run it with flatpak run org.vim.Vim .

toolbox

The toolbox project provides a traditional operating system inside a container. The idea is that you can mess with the mutable OS inside your toolbox (the Fedora container) as much as you like, and leave the immutable OS outside your toolbox untouched. You pack as many toolboxes as you want on your system, so you can keep work separated. Behind the scenes, the executable /usr/bin/toolbox is a shell script that uses podman .

A fresh install does not include a default toolbox. The toolbox create command checks the OS version (by reading /usr/lib/os-release ), looks for a matching version at the Fedora container registry, and downloads the container.

$ <strong>toolbox create</strong>
Image required to create toolbox container.
Download registry.fedoraproject.org/f31/fedora-toolbox:31 (500MB)? [y/N]: y
Created container: fedora-toolbox-31
Enter with: toolbox enter

Hundreds of packages are installed inside the toolbox. The dnf command and the usual Fedora repos are set up, ready to install more. The ostree and rpm-ostree commands are not included – no immutable OS here.

Each user’s home directory is mounted on their toolbox, for storing content files outside the container.

Put the pieces together

Spend some time exploring Fedora Silverblue and it will become clear how these components fit together. Like other Fedora variants, all these of tools come from open source projects. You can get as up close and personal as you want, from reading their docs to contributing code. Or you can contribute to Silverblue itself.

Join the Fedora Silverblue conversations on discussion.fedoraproject.org or in #silverblue on Freenode IRC .