PostmarketOS Project Direction 2020

This is somewhat of a follow-up tolast year's big project direction post, but this time I did not come up with it all on my own. Quite the contrary, the core team has a monthly phone call now, in which we talk one or two hours about moving postmarketOS forward and this is basically what we came up with. In addition, and probably because of that fact, the ideas are more technically mature and ready to be implemented. Let me just write it all together in a big picture, and fill in the blanks as necessary. Further below are detailed TODOs, which will link to separate issues, as we approach them. Get comfortable in your chair and grab a drink, here goes a long-read!

:tropical_drink: :seat: :wink:

Where we are now

Since the last direction post, we had a huge amount of commits in pmaports.git and pmbootstrap.git obviously, but I'm not going to recount those here as this would blow up this issue. But to focus on what is important for the overall project direction, the biggest change has been, that I am not building the binary packages at home anymore - they are built on sourcehut with bpo . Previously all merge requests would only be merged by yours truly (as only I could see whether builds fail or not after merging), but now the build logs are transparently online and we have more people merging the MRs - this allows the whole project to scale (see Alpine stable goal below) and me to work less on MRs and more on tasks like this one.

We have dropped several UIs that were not well maintained and all remaining UIs have maintainers.

On the hardware side, both the Pinephone and the Librem 5 are delivering the first prototypes to people who ordered them these days (afaik Librem 5 has already, and Pinephone is either in December or January). Both phones are made for Linux distributions, running a (close to) mainline kernel, free software where possible, and not just Android with a downstream kernel fork and tons of blobs like your regular smartphone. This is great for postmarketOS, because it makes a daily driver goal more realistic. More on that below.

Notably the Pinephone expects to be ready for daily usage around March 2020 (that's what I've gathered from a recent interview in the Linux Unplugged Podcast), which is a reasonable timeframe for getting postmarketOS in shape for daily usage - at least for basic tasks!

Road to daily driver in 2020

Base on Alpine's stable releases

With the binary repo blocker out of the way, it's time to have a stable postmarketOS channel that follow Alpine's stable releases. Alpine releases a new version roughly every six months. Instead of just the edge-based channel like it is now, we will always have two postmarketOS channels, edge (based on aports master branch) and latest (e.g. based on aports branch 3.10-stable ). Here's an overview, the latest-next and legacy channels are explained right afterwards:

When a new Alpine version comes out, there is a transition phase during which we have up to three channels:

• New Alpine release (3.11) arrives
• Temporary postmarketOS channel "latest-next" is introduced, points to a new pmaports.git branch 3.11 that gets branched from master . No new features are added after branching, only bug fixes (with exceptions, we'll have to see how it works out, but this is how projects would typically do it).
• We test and fix bugs in this new version for one month, especially with the supported devices (that definition will be revisited, further below)
• Then we let a new "legacy" channel point to the previous latest branch (3.10), and change "latest-next" to "latest" (3.11; in total we have 3 channels and branches again)
• During the next month, users must upgrade to the current "latest". We still provide fixes for the "legacy" channel for that one month (keeping timeframes short for now, so we don't have an overwhelming amount of overhead)
• After that month, we drop the "legacy" channel. (Ideally upgrade scripts would still work though, we'll see how that works out.)
• Two months have passed since the Alpine channel, so in four months the cycle begins again.

At all times, we recommend end-users to install the "latest" channel.

You may be wondering why I'm writing this out in such detail (instead of a dedicated issue), but these details are important to see what we require in order to accomplish this.

channels.cfg

We need a good location to store the table above - which channel points to which branch at any given time?

So I came up with a channels.cfg file:

[channels]
edge=master
latest=3.10
latest-next=3.11
# legacy=3.9

I would like to store it in a dedicated branch in pmaports.git, which is also called channels.cfg . The reason is, that if we store it in any other pmaports.git branch, that we are using for building aports (e.g. master ), then we could get an outdated channels.cfg on staled branches. By having it in a dedicated branch, it always counts for the whole repository. Another possible location would be pmbootstrap, but this would tie the pmbootstrap version unnecessarily to these versions.

No matter which branch is checked out, a script (pmbootstrap) or user would be able to see the file with ( more ):

$ git show channels.cfg:channels.cfg

Devices in separate repository

Devices should go into a separate repository. This allows using any device with both the latest channels, or with edge (as long as they don't depend on device-independent features in edge). Without the downside of keeping all devices duplicated in two or even three branches at the same time.

Each device should have the minimum compatible postmarketOS channels, that it was tested with, in its deviceinfo , so let's introduce a new variable:

deviceinfo_min_postmarketos_channel="3.10"

In case a device does depend on a feature that is only available in edge, it would simply put the next version there (latest + 1), so if latest is at 3.10, this would be 3.11. For consistency with pmbootstrap.git and pmaports.git, let's call the new repository pmdevices.git.

Get serious about supported devices

Right now we don't have any devices that we claim are actually "supported" on the homepage. We do have two devices, that are highlighted however, and these are the Nokia N900 and the Nexus 5. Both have great mainline support and what is possible with postmarketOS on them is impressive considering the downstream kernel and nonfree userspace drivers they come from. However, in the current state, these ports are not usable as daily drivers. So let's take a look at the up-and-coming devices from Pine64 and Purism:

• PinePhone
• PineTab
• Librem 5

Since they are made for running Linux, and we can collaborate and build on work from related Linux smartphone projects, it should be reasonable to get the following features working:

• Calls (not for tablet)
• SMS (not for tablet)
• Usable phone UI (Plasma Mobile, possibly Phosh)
• Battery charging
• Wifi

"Nice to have" (points above must definitively be prioritized!):

• Bluetooth
• Camera

So we're going to approach this from a realistic angle. Start with two or three supported devices, by name the two Pine64 devices (team members should have enough of those and be able to maintain them), and if we find somebody who has one and is able to maintain it, the Librem 5 (possibly@craftyguy?). If in doubt, just start with two or even just the PinePhone, but have that one working reliably with a small feature set and expand from there.

Let's add a new deviceinfo variable, where we can see if a device is officially supported or not. Think of all other devices like the AUR - still useful, but sometimes broken and you need to know what you are doing when using these. (Unsupported devices should also have this variable in the future, so it's explicit.)

deviceinfo_officially_supported="true"

I hope that in the future, we can expand the supported devices to some of our favorite old-but-gold ones like the Nexus 5. But this one specifically doesn't have audio working, so it's out of question for now. Having postmarketOS working as daily driver for at least one phone would be a game changer, which is likely to lead to new hackers showing up that eventually lead to making old devices supported. So while the device selection is tiny, consider this a catalyst for the whole project that will let it grow a lot more, for all devices - after all, the whole architecture of postmarketOS is designed to have as few device specific code as possible!

More sophisticated approach, that allows more than two values ("supported, "unsupported") is in#16.

pmbootstrap changes

pmbootstrap init

Behind the scenes, it should clone pmdevices.git if it is missing (just like it clones pmaports.git) and add a symlink in the local clone of pmaports.git, so that pmaports/device and pmaports/firmware point to the new repository. This is consistent with how we handled the split of pmaports.git from pmbootstrap/aports to its own repository.

If pmaports.git is on the master branch, or on a known channel branch, it would ask the user which channel they would like to use, along with the available choices parsed from channels.cfg. On the first run, we recommend "latest" (that's the stable one regular users should be using).

During device selection, this should show a warning if the selected device is unsupported.

pmbootstrap pull

So we have yet-another-repository that developers, but also regular users, need to keep updated. Let's make it a little bit easier for both and provide a shorthand command for "git pull" in both pmaports.git and pmdevices.git.

This should check first, if the repository in question is on "master" or a channel branch, and only then run "git pull". Regular users will benefit most from this change, because then they don't need to familiarize with where the repositories are and how to update them - just run one command.

pmbootstrap status

Basically a health check of your pmbootstrap setup, which would most importantly check if you need to update your pmaports.git and pmdevices.git repositories, and if so, directly point the user at "pmbootstrap pull". We had the idea for this, before the concept of pmdevices.git, but with the big picture at hand, this seems like the perfect addition to make the transition to the new repository as painfree as possible.

This should also run at the end of "pmbootstrap init" and also show if the device is supported or not.

Full issue:pmbootstrap#1829.

TODOs

Here is what needs to be done in chronological order (unlike the above, which is thematically sorted to show what it builds up to). We'll add related issues in the future, for now I'll just add checkboxes unless we have an existing issue, so I can finish this post today.

pmdevices.git related

• pmbootstrap: add pmbootstrap pull as shorthand for git pull in pmaports.git clone, if it is on master branch ( pmbootstrap#1858 (closed) )
• pmbootstrap: add pmbootstrap status to display whether pmaports.git must be updated, run it at the end of pmbootstrap init (pmbootstrap#1829)
• pmbootstrap: support pmdevices.git (merge together with split, see below)
  • pmbootstrap init: clone pmdevices.git and create "device" and "firmware" symlinks in pmaports.git clone
  • pmbootstrap init: check pmdevices.git version from something like pmaports.cfg, just like we do for pmaports (so it can have a min pmbootstrap version, and we can require a min pmdevices.git version in pmbootstrap)
  • pmbootstrap pull: also pull pmdevices.git
  • pmbootstrap status: also display status of pmdevices.git
  • pmbootstrap: create release
  • require new pmaports version
• pmaports: split repository into pmdevices.git
  • adjust CI in both repos
  • bump pmaports version in pmaports.cfg
  • require new pmbootstrap version in pmaports.cfg
• bpo: start building new packages when commits get pushed to pmdevices.git
• publish a blog post about this change (I'd like to get smaller, more frequent ones out)

Stable branch in pmaports.git (depends on pmdevices.git)

• pmaports: possibly get rid of all branches except for master (put into other repository and create wiki page documenting where they are now) - let's discuss this first
• pmaports: branch master to new branch named like current Alpine stable (3.11 probably at this point?)
• pmaports: create "channels.cfg" branch and "channels.cfg" file, with latest=3.11
• pmaports: set up permissions, so that nobody can force-push to channels.cfg and 3.11 branches, just like master
• pmbootstrap: allow branch selection (default to "master" for now) (pmbootstrap#1855)
• pmbootstrap: check for apk version: this check exists already, but from now on it should check for different versions depending on the channel branch
• bpo: properly support building from multiple branches ( build.postmarketos.org#63 )
• bpo: build devices and firmware only from edge channel, use symlinks in all other channels
• bpo: configure to build latest channel branch (this will take some time...)
• pmbootstrap pull: also pull pmaports.git, when on channel branch from channels.cfg
• pmbootstrap: write postmarketOS channel (3.11) into /etc/os-release, instead of pmbootstrap version (1.x)
• pmbootstrap: aportgen: add deviceinfo_min_postmarketos_channel
• pmdevices: add deviceinfo_min_postmarketos_channel to each device, with CI test
• pmbootstrap: check deviceinfo_min_postmarketos_channel when selecting device and channel
• pmbootstrap: init: default to "latest" channel (!)
• document branch switching workflow from above in wiki
• blog post

Supported device selection (could be worked on in parallel to both above)

(see#16)

• pmbootstrap: add support for main / community / testing folders below device/
• pmbootstrap: generate new devices into testing
• consider adding a new deviceinfo_status property or somehow generate it automatically
• pmbootstrap init: show warning if device is in community / testing (preferably a different one)
• pmbootstrap status: show device status
• wiki: document requirements for main / community
• update wiki: separate devices by status ( main / community / testing )
  • bonus (could be done later): display devices differently, e.g.

    main
    community
    testing
    

• prepare homepage update (merge as one MR)
  • pmdevices.git and stable branch changes from above must be completed first
  • update homepage, so it doesn't feature the N900 and Nexus 5 anymore. Replace with devices that are supported.
  • get rid of "phone calls don't work" (or reword it... should at least work on the supported devices)
  • new blog post

Eventually later:

• pmbootstrap: add support for unmaintained
  • device should be selectable with an immediate error in pmbootstrap
  • packages should not get built
• pmbootstrap: add some way to set a reason for unmaintained status, and display it when selecting the device

Streamlining

• make sure that upgrading always works properly (post-install scripts should not fail, provide upgrade paths if subpackages changed like with mesa in !896 etc.)

Next up for me is writing a short blog post related to build.postmarketos.org being in production. I wanted to get this issue written down first, so I could link to it from there, in case some readers are curious about what we have planned. After that, I'll switch between making necessary bpo improvements, and starting to implement the first points from this list (besides doing other necessary admin work).

The list above reflects how I would implement it. If somebody is interested in helping out, it would be great and certainly speeds up the process. Pick one of the easy starting points, like "pmbootstrap: add pmbootstrap pull as shorthand"... above, or "pmbootstrap: aportgen: add deviceinfo_officially_supported ", then create an issue in which you state that you will work on it, and link to it from here (I'll try to do the same, so we don't have duplicate work). But please make sure that you don't take too long with it if it's blocking the next task.

Thanks for reading, and thanks to everybody for all the awesome contributions that lead us here! :rocket:

Looking forward to comments.

EDIT 2019-12-13:

• changed wording from release to channel where appropriate
• "firmware" dir also needs to move to pmdevices.git
• mention that we need something like pmaports.cfg in pmdevices.git (so we can check min pmbootstrap and pmdevices.git versions in pmbootstrap, just like we do with pmaports)
• bpo: add TODO for building devices/firmware only for one release and using symlinks

EDIT 2020-01-26:

• add "Streamlining" section

EDIT 2020-02-07:

• link to#16

EDIT 2020-02-25: (by@Minecrell)

• rewrite steps for "supported device selection"