Perl 7: A Risk-Benefit Analysis

At the recent Conference in the Cloud for Perl and Raku, Sawyer X (the pumpking of Perl) announced an ambitious plan for Perl 7 . Since Perl 6 was renamed to Raku to better communicate its fundamental differences from the well known identity of Perl, major versions are now available again for Perl to leverage, and it is a very important step to show that the language is still developed and used. I completely agree with the motivation and ideals presented, and have thought a lot about the benefits and risks involved in such ideas long before I was aware of this project.

I do not generally work with ancient code that uses ancient practices. I work with CPAN modules that maintain compatibility with wider or narrower ranges of Perl versions for various reasons. I work with modern code for my own and business use that already will not function on older Perls. I work with newcomers that have written code based on modern Perl tutorials, and newcomers that have written code based on ancient Perl tutorials. It's from this perspective that I evaluate the proposed direction, the stated goal of which is to optimize for new users and active maintainers over abandoned code.

The announced direction

I am sure there are plenty of well studied mechanisms for evaluating proposals of such a nature, but I am a simple software engineer, so I will analyze the risks and benefits of the approach. This is an opinionated process, but I have tried to be fair and consider the honest potential of the currently stated plans.

Unequivocally, I will not consider ideas here that effectively result in forking Perl. These include ideas to intentionally be incompatible with CPAN, or use file extensions like .pm7 to indicate the incompatibility. We have the most useful possible evidence in where this leads thanks to Raku. This also does not take into account various milder assumptions people have made about implementation, because no such details have so far been announced.

The benefits of the announced direction:

• The use of major versions allows for a clear message to those outside the Perl bubble about continued development of Perl and easy to remember milestones for when features are introduced or removed.
• New defaults will signal to the user what style is considered modern.
• New code, written by either newcomers or experienced Perl coders, will not need to consider what set of esoteric pragmas to enable to get a modern style, as they will be enabled by default.
• Code can be written to take advantage of new features without lots of boilerplate or restricting the code from running on older versions; this benefit only really applies to strict and warnings, since any other features create a de facto minimum requirement of the Perl version they were introduced.

The risks of the announced direction:

• CPAN modules which largely support more than only the latest version of Perl will not be able to make use of Perl 7 changes.
• Perl users who miss the Perl 7 announcement or use old tutorials will continue to miss the existence of new features.
• Various tools and modules assuming a major version of 5 will need their assumptions fixed.
• Newcomers using tutorials written for Perl 5, Perl 5 books, code on Stack Overflow, etc may not understand why the code does not work on Perl 7 or how to fix it.
• Code written for Perl 7 will contain no indication of the assumptions it has made which will be broken (often silently, due to the nature of the proposed new defaults) if it is run in Perl 5; and will have no indication of the minimum version required to support the features it is using.
• The relationship between the ":7.0" feature bundle and the features enabled by default is currently not specified. If they are to be the same, important features for writing modern code that are problematic as defaults such as 'unicode_strings' and 'signatures' cannot ever be enabled in feature bundles, thus continuing the current problem of having to use esoteric boilerplate for modern code. If they are to be different, the implementation and presentation of this distinction will be necessarily very complex.
• The impact on oneliners is yet to be explored. The -E option is commonly used to enable the most current feature bundle, but not strict, because while the feature bundle is quite unlikely to break oneliners in comparison to full scripts and modules, strict is generally only annoying in a oneliner context. Either -e or -E enabling strict could significantly affect Perl's use as a sysadmin and ops tool; and -e enabling unexpected features when being used specifically to avoid that would be another symptom of the proposed incompatibility.
• Large amount of CPAN modules will not work in Perl 7; plans for working around this would either involve every affected CPAN author, which is a virtual impossibility for the stated 1 year time frame; or the toolchain group, a loose group of people who each maintain various modules and systems that are necessary for CPAN to function, who either have not been consulted as of yet or have not revealed their plans related to the tools they maintain. Going into this potential problem sufficiently would be longer than this blog post, but suffice to say that a Perl where highly used CPAN modules don't seamlessly work is not Perl.
• Downstream vendors will need to evaluate how to package Perl going forward. Since Perl 7 would be incompatible, the most likely option is maintaining a separate Perl 7 binary and leaving Perl 5 in place for the foreseeable future, similar to the Python 2 to 3 transition. They will need to be aware of the strategy chosen to make CPAN modules work with Perl 7 and employ a solution of their own for the CPAN modules they package as well as the CPAN installers they provide.
• Active core developers, toolchain developers, CPAN authors, and downstream packagers are currently relatively small groups of people, thus the burden on each of these groups to solve the respective problems is that much greater, risking further burnout from these essential workers of the Perl ecosystem.
• Perl development will be frozen for the period of time until Perl 7 is released, which has currently been specified as from 6 months to a year; we are very familiar with the dangers of pausing development of Perl in anticipation of a new version.
• The culture and assumptions of the Perl stability policy will be fundamentally challenged through each aspect of the ecosystem. This may be the intent, but it is still a risk.
• Critics will point out that Perl 7 breaks code and does not contain anything new.

You may notice that I did not include "Perl 7 allows cruft to be removed, simplifying the language implementation" as a benefit, because this is already done in every yearly release and not changed in any way by the announced direction, except for the improved ability to advertise such changes by associating them with major version releases.

I also did not include the benefit of "New defaults encourage users to stop using old features and syntax in existing code", because in practice nothing encourages this except deprecating or removing the ability to use such syntax, which again does not significantly change under this direction.

A slightly different proposal

Let's examine the risks and benefits of a slightly different proposal, as follows.

Perl development continues in the normal fashion in the Perl 5.33 development track, and the next major release is 7.0.0 instead of 5.34.0. This version's feature bundle is enhanced to removed the 'indirect' feature, and possibly with new features that have been added in this year of development; there are already rough plans for try/catch and multidimensional features, and the signatures feature could be moved towards completion . In addition, "use v7" or higher will enable the warnings pragma, and possibly the utf8 pragma for source encoding , in addition to strict that is already enabled for "use v5.12" or higher. Old features and syntax can be removed through the normal policies if deprecated, or the deprecation process can be started for others. Crucially, the default state of the interpreter does not set any features or pragmas differently from Perl 5, barring removal of deprecated syntax as in previous releases.

The benefits of this proposed direction:

• The use of major versions allows for a clear message to those outside the Perl bubble about continued development of Perl and easy to remember milestones for when features are introduced or removed. (same as announced)
• The "use v7" pseudo-pragma will signal to the user what style is considered modern.
• New code, written by either newcomers or experienced Perl coders, will not need to consider what set of esoteric pragmas to enable to get a modern style, as it will be represented by "use v7" in a way that "use VERSION" previously was inadequate for. (same as announced)
• Code written for Perl 7 will contain a clear indication of its assumptions and be restricted to running on that version or higher.
• The standard development process for the next release of Perl will have the added benefit of a major version for improved messaging of any significant features.

The risks of this proposed direction:

• CPAN modules which largely support more than only the latest version of Perl will not be able to make use of Perl 7 changes. (same as announced)
• Perl users who miss the Perl 7 announcement or use old tutorials will continue to miss the existence of new features. (same as announced)
• Various tools and modules assuming a major version of 5 will need their assumptions fixed. (same as announced)
• New code will need to include "use v7" to get a modern featureset.
• Critics will point out that Perl 7 does not take any chances.

Conclusions

It is easy to frame this discussion as being for or against progress, but please keep in mind I am not motivated to keep Perl 4 code running forever on new Perls, or to stand in the way of progress, but instead to focus efforts in order to improve the chances of successful progress. I completely support the consideration of incompatible changes (as, I will note again, is already part of the Perl development policy) provided I believe they will have a proportional potential for benefit to the ecosystem. I absolutely want Perl 7 to be a positive message for the continued development and usage of Perl, and to encourage changes to how we think about using the language as a whole.

While I recognize that there are potential benefits that only changing the default features can provide, I encourage the reader to consider the practical fallout of changing these assumptions in each component of the Perl ecosystem, and what it will look like if we were to consider doing this again for each major version following. The proposal suggests it would be the first of many such breakages, and even if we only do this once, it is not just a one time change to adapt to, but rather has lasting implications for the language and its users for the foreseeable future.

I can only conclude in this case, as I have for the past few years watching Raku grow into its own identity, that the significant risks and unknowns unique to the decision to incompatibly change feature defaults are a cost not worth the benefits for Perl. The announced direction does not solve many of the stated problems, and most of the same benefits can be achieved with nearly the standard development process, by leveraging the newly available major versions in messaging, and requiring "use v7" to activate the new normal.

Reddit comments