Adding Mainline Arm Frame Buffer Compression Support for Rockchip

栏目: IT技术 · 发布时间: 4年前

内容简介:Rockchip SoCs, notably the RK3399, are popular in devices such as Chromebooks and single-board computers. Indeed, they bring some interesting features, one of them being theTo understand that, let's have a look at a typical display pipeline. There usually

Rockchip SoCs, notably the RK3399, are popular in devices such as Chromebooks and single-board computers. Indeed, they bring some interesting features, one of them being the Arm Frame Buffer Compression (AFBC).

Why AFBC?

To understand that, let's have a look at a typical display pipeline. There usually is a GPU for 3D rendering and ultimately its output data must be sent to the actual display, and to do so it must be formatted suitably by a Display Processor. Nowadays each frame weighs at least a few megabytes and sometimes you may access multiple frames at once. Obviously, this amount of memory cannot be provided internally by the SoC, so we need to be accessing memory which is external to it, and we need to do this a lot! This, in turn, directly translates into memory bandwidth (of which there is a limit: you can only transfer so much in a unit of time), battery power usage and heat generation. These reasons alone justify looking for a way to reduce memory bandwidth usage in a display pipeline.

Arm created AFBC exactly to mitigate the problems mentioned in the previous paragraph. The trick is that the image is split into blocks (of e.g. 16 x 16 pixels) and each block's data is compressed. Each block also has a fixed length header associated with it to store compression metadata. All the headers are sent first, followed by blocks.

Adding Mainline Arm Frame Buffer Compression Support for Rockchip

So, theoretically more memory is used by a compressed frame, but read on to understand how the reduction of memory bandwidth becomes possible.

AFBC will allocate slightly more memory to hold each block's data (headers plus usual amount for the frame) than a normal uncompressed frame would . However, if the block's data compresses well (and you can compress by up to 50%), much of that allocation won't actually be used. So, depending on the dynamic compression ratio, you can save up to 50% memory bandwidth, improving both efficiency and performance.

The compression is lossless, which wouldn't be that important if frames were to be only displayed to human users, but compression being lossless is crucial if decompressed frames serve as reference frames, because there are no inaccuracies which would otherwise keep accumulating. And despite the compression, the scheme even allows randomly accessing image data down to 4px x 4px! Great? Wait...

A spoonful of tar in a barrel of honey

The said compression scheme is proprietary, so unfortunately we don't know how it works. The AFBC-related parts of hardware in the GPU and Display Processor are total black boxes, thus limiting the potential of Open Source community to write great software utilizing AFBC. Nevertheless, AFBC-aware IP blocks should be able to exchange AFBC buffers even if we don't fully understand the buffer contents, so it still is an interesting option for the moment. It does prove useful because of memory bandwidth reduction - but it has trade-offs, including a complete lack of knowledge of its internals. You should know that other vendors have competing implementations of similar schemes, too -- so here's to a proper, open standard for frame buffer compression in the future.

So, what can be done?

We can still use AFBC in our display pipelines without knowing "what's inside": if the components of our pipeline understand AFBC, we can make them use it and enjoy the benefits. To do so we need to control the involved components and that can be done even without understanding what they do under the hood.

Some insight

That being said, we can try understanding what's inside an AFBC-compressed frame. You can have a look here, branch afbc-test to see how an all-red frame can be generated. It doesn't take a rocket scientist to notice that for each 16x16 block there is a fixed-length header of 16 bytes. All the headers go first and then all the blocks follow. In the header the first 4 bytes specify where in the buffer its corresponding block data is, and the fifth byte is 0x07:

Adding Mainline Arm Frame Buffer Compression Support for Rockchip

Inside the block the first 6 bytes contain specific values and the rest is filled with zeros. So, for an all-red frame, we use only 6 bytes out of 1024! (16 * 16 * 4 bpp):

Adding Mainline Arm Frame Buffer Compression Support for Rockchip

Of course the all-red frame is kind of an extreme case, but you get the idea of how we are limiting memory accesses with AFBC.

Upstreaming AFBC for RK3399

Since late 2018, the Mali-DP display drivers, malidp and komeda, have had the ability to use AFBC, and now support for Rockchip (RK3399) is also on its way. While the initial work was done by Rockchip in 2014 , it unfortunately wasn't upstreamed. Efforts to provide AFBC support for Rockchip in mainline have recently concluded and the feature is available in drm-misc-next tree. You can find the actual patches here [ 1 ][ 2 ][ 3 ][ 4 ] [5 ].

Some design considerations in Linux kernel DRM.

The DRM subsystem follows the "library instead of midlayer" approach, which is nicely described in this lwn article from 2009 . So, consequently, there is DRM core and DRM drivers. The latter are free to use so called DRM helpers, but there is nothing preventing them from opting out. During the work on AFBC, an idea has crystalized in the mailing list in November 2019 to put AFBC handling in helpers. This indeed proves a good design decision, because thanks to it the core does not deal with a very specific (and proprietary!) extension and, in fact, the use of helpers is purely optional. The patch series first lays the foundation for drivers to allocate struct drm_afbc_framebuffer explicitly in order to be able to do special AFBC-related checks and now the drivers can opt-in to use the new helpers.

The future

We should be expecting that AFBC support for Rockchip will be landing in the next Linux kernel release. It is worth mentioning that existing AFBC users can also benefit from the newly added helper functions (patch [ 2 ]). f you have any questions on how to these new functions, or if you would like to try to bring AFBC support to another AFBC-enabled SoC, please contact us! While we obviously can't create AFBC per se, we can help control the hardware so that it starts using it.


以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

数据结构教程

数据结构教程

彭波 / 第1版 (2004年3月1日) / 2004-3-1 / 34.00元

精心策划,准确定位 概念清晰,例题丰富 深入浅出,内容翔实 体系合理,重点突出一起来看看 《数据结构教程》 这本书的介绍吧!

Base64 编码/解码
Base64 编码/解码

Base64 编码/解码

正则表达式在线测试
正则表达式在线测试

正则表达式在线测试

HEX CMYK 转换工具
HEX CMYK 转换工具

HEX CMYK 互转工具