Collapse OS – Why Forth?

Collapse OS — Why Forth?

Collapse OS' first incarnation was written in Z80 assembler. One of the first feedbacks I had after it went viral was "why not Forth?". I briefly looked at it and it didn't seem such a great choice at first, so I first dismissed it. Then, I had what alcoholics refer to as a "Moment of clarity" .

Forth is a stellar fit to Collapse OS design goals. If you're not familiar with it, it might be hard to understand why. Let me try to explain.

What it's not

Some Forth enthusiasts like the language for itself. They think that its easier to express much of the logic of a program in this language. Many books and tutorials are touting this.

That's not my opinion. Forth is a fine and capable language, but I tend to agree with those who say that it tends to produce write-only programs. Most of the books on Forth are very old and compare Forth's elegance to Fortran's or COBOL's. I'm of the opinion that C is more expressive and generally produce more elegant and maintainable code.

When you look for Forth's advantages, don't look there.

Forth isn't particularly fast either. As an interpreted language, it is blazing fast, but the speed tax of the indirect threaded model still has to be paid, so it will generally be slower than code produced by a C compiler.

However, as a Python developer, I know this tradeoff well. It's not 100% of your program that needs to be fast, only bottlenecks. Like with Python, it's easy in Forth to write bottlenecks in native code. Therefore, speed is not such a big issue.

What it is

Forth is, to my knowledge, the most compact language allowing high level constructs. It is so compact that Collapse OS' implementation achieves self-hosting with about the same amount of resources than its assembler counterpart!

What is this sorcery? At face value, nothing can be more compact than native code. It's self-evident. If a piece of Forth code is doing the exact same piece of logic in a way that is as compact as native code, you've been lazy on your assembly writing!

To understand this compactness, we have to look deeper.

The Z80 asm version of Collapse OS self-hosts on a RC2014 with a 5K shell on ROM, a 5K assembler binary loaded in RAM from SD card (but that could be in ROM, that's why I count it as ROM in my project's feature highlights) and 8K of RAM. That is, it can assemble itself from source within those resources.

The biggest piece of software to assemble is the assembler itself. It's a reasonably well-featured assembler that supports forward labels, includes, many useful directives. The code implementing those features requires those aforementioned resources to assemble.

...But also, the features included in the assembler are required to assemble its source code! I could cut a bit on its features to reduce resource usage, but I'd need to adapt the assembler's code for this lack of features and possibly make the code much more complex. I believe that with this assembler, I stroke a good balance and achieved something close to optimal.

If I wanted to re-implement that assembler feature-for-feature in Forth, it would probably require much more resources to build. Even though higher level words are more compact, the base of the pyramid to get there couldn't compete with the straight assembler version. This was under this reasoning that I first dismissed Forth.

So, again, what makes Forth more compact than assembler? Simplicity. The particularity of Forth is that it begins "walking by itself", that is, implementing its own words from its base set, very, very early. This means that only a tiny part of it needs to be assembled into native code. This tiny part of native code requires much less tooling, and thus an assembler with much less features. This assembler requires less RAM.

What is more compact than something that doesn't exist? Even Z80 assembler can't beat the void.

That's how although Forth is not more compact that native code (duh!), a Forth Collapse OS achieves self-hosting with as much resources than its Z80 counterpart .

So for the very low price of manageable speed constraints, we get a high level language that is much more powerful and extensible than the BASIC shell that was previously there. I call that a definite win.

Cherry on top

Because compactness has always been a primary design goal of Collapse OS, I initially chose Z80 assembler over high level languages so that I could continue to meet that design goal.

However, now that I can have my cake and eat it too, thanks to Forth, new opportunities open. First and foremost, multiple architectures! Although I was ready to give that up when I tied myself to the Z80, I'm now very glad to have it and will certainly take advantage of it.

Culture of simplicity

Forth's culture of simplicity is worth writing about. At face value, a traditional Forth environment is severly constrained according to modern standards.

Who would want to work within the constraint of 64x16 blocks filesystem? Someone who knows the complexity associated with implementing a filesystem and everything that underpins it. When simplicity is valued, such constraints are gladly accepted.

We can't say that asceticism is a dominant trait of our relation to technology. We aren't used at all to willingly accept constraints with regards to what we could call our convenience. Why would anyone not want a piece of software or hardware to be easier to use, more accessible to the layman, more featureful? If someone is willing to build that complexity, Moore's law allow us to adopt it, and we do. This leads to cancerous complexity, layers upon layers upon layers with each new generation of developers blissfully accepting the previous layers.

(Something analogous can probably be said about our collapsing civilization, and probably even eloquently quote Joseph Tainter)

The base design of Forth is entirely built around a primal simplicity . Even better, the path of least resistance generally guides you towards simplicity. When a word definition becomes too big and complicated, it generally prompts you to reorganize your code toward a simpler solution that might not have occurred to you in another language.

To be clear: Forth doesn't elegantly describe complex algorithms, at least not when compared to languages like C. Those generally make the head hurt when reading. However, this mental pain does makes you question your need for complexity and encourages "sideways" solutions.

This is something that I think few developers are familiar with and is hard to describe with words. It needs to be experienced. One of the good ways to experience it I think is to implement your own Forth.