Ernie - binary encoding format based on External Term Format (Erlang)

栏目: IT技术 · 发布时间: 6年前

内容简介:Simple binary data exchange format that is based onErnie is fun over the original BERT name, from the Sesame Street Bert & Ernie couple.No, this document will not define RPC syntax because of 2 reasons:

Ernie

Simple binary data exchange format that is based on External Term Format in Erlang. This is update on BERT which is a little bit outdated right now.

Name

Ernie is fun over the original BERT name, from the Sesame Street Bert & Ernie couple.

RPC

No, this document will not define RPC syntax because of 2 reasons:

  1. This is out of scope of this document. It is meant to define only data encoding format, not services one will use it for.
  2. Authors of this document believes that RPC are bad, and you should not use them. Instead use message passing between your services.

Schema/structured data

This is meant to encode free-form data, if you want to encode structured data with schema then we highly suggest you to take look into other formats like:

  • ASN.1 and any of it's encodings
  • ProtoBuffers
  • Apache Thrift
  • FlatBuffers
  • Cap'n'Proto

Specification

Introduction

  1. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 .

Types

  1. Integer

    Binary representation of integers.

    Implementation MUST support at least 8-bit unsigned and 32-bit signed integers with full precision.

  2. 64-bit Float

    IEEE 754 double precision floats.

    Implementation MUST support all non-NaN, non-infinite, non-subnormal values described in IEEE 754 binary64 spec. Implementations SHOULD NOT encode these values.

  3. Tuple

    Fixed length sequence of values.

  4. List

    Dynamic length sequence of values.

  5. Binary

    Representation of bytes with arbitrary length. If used to store human-readable strings it MUST be encoded as correct UTF-8 string.

  6. Map

    List of key-value pairs in arbitrary order. Keys MUST NOT appear twice in the same map.

Encoding

  1. Top level

    Binary encoded data MUST start with magic byte of decimal value of 131 followed by single byte Tag value followed by arbitrary length data:

    0        1       2
     +--------+-------+-------+-------
     | 131    | Tag   | Data ...
     +--------+-------+-------+-------
  2. Integer

    Integer can be encoded in 3 different forms:

    • Values in range 0..255 (aka byte)

      0       1        2
        +-------+--------+
        | 97    | Data   |
        +-------+--------+

      Where Data is unsigned byte representation of value.

    • Values in range -4 294 967 295..4 294 967 294 (signed 32-bit value)

      0       1       2      3      4      5
        +-------+-------+------+------+------+
        | 98    | Data                       |
        +-------+-------+------+------+------+

      Where Data is signed 32-bit representation of integer in U2 big-endian encoding.

    • "Short" arbitrary long signed integers (up to 2040-bit)

      0       1       2       3     ...     N+3
        +-------+-------+-------+------+------+
        | 110   | N     | Sign  | Data        |
        +-------+-------+-------+------+------+

      N is big-endian 16-bit unsigned integer. Data represents little-endian encoded N byte long integer. Sign is 0 for positive integer and 1 for negative one.

    • "Long" arbitrary long signed integers (from 2041 to 524 288-bit long)

      0       1       2       3       4     ...     N+4
        +-------+-------+-------+-------+------+------+
        | 111   | N             | Sign  | Data        |
        +-------+-------+-------+-------+------+------+

      N is big-endian 16-bit unsigned integer. Data represents little-endian encoded N byte long integer. Sign is 0 for positive integer and 1 for negative one.

  3. Float

    0       1       2        3       4       5       6       7       8       9
     +-------+-------+--------+-------+-------+-------+-------+-------+-------+
     | 70    | Data                                                           |
     +-------+-------+--------+-------+-------+-------+-------+-------+-------+

    Where Data is IEEE 754 double precision floating-point number encoded as big-endian.

  4. Tuple

    • Tuples up to 255 elements

      0       1       2       ...       N+2
        +-------+-------+--------+--------+
        | 104   | Arity | Data ...        |
        +-------+-------+--------+--------+
    • Tuples up to 4 294 967 295 elements

      0       1       2        3        4        5       ...       N+5
        +-------+-------+--------+--------+--------+--------+--------+
        | 105   | Arity                            | Data ...        |
        +-------+-------+--------+--------+--------+--------+--------+

    Where Arity is big-endian unsigned integer declaring amount of elements in Data .

  5. List

    • Byte list

      0       1       2       3      ...      N+3
        +-------+-------+-------+-------+-------+
        | 107   | N             | Data ...      |
        +-------+-------+-------+-------+-------+

      This is optimisation for lists that contains only values in range 0..255, inclusive.

    • General list

      0       1       2       3       4       5      ...      N+5     N+6
        +-------+-------+-------+-------+-------+-------+-------+-------+
        | 108   | Arity                         | Data ...      | 106   |
        +-------+-------+-------+-------+-------+-------+-------+-------+

      Where Arity is big-endian unsigned 32-bit integer declaring amount of elements in Data .

    • Empty list

      0       1
        +-------+
        | 106   |
        +-------+
  6. Binary

    0       1       2       3       4       5      ...      N+5
     +-------+-------+-------+-------+-------+-------+-------+
     | 109   | N                             | Data ...      |
     +-------+-------+-------+-------+-------+-------+-------+
  7. Map

    0       1       2       3       4       5      ...      N+5
     +-------+-------+-------+-------+-------+-------+-------+
     | 116   | Arity                         | Data ...      |
     +-------+-------+-------+-------+-------+-------+-------+

    The Arity field is an unsigned 32-bit integer in big-endian format that determines the number of key-value pairs in the map. Key and value pairs ( Ki => Vi ) are encoded in section Data in the following order: K1, V1, K2, V2,..., Kn, Vn .

Open questions

  1. Should we support atoms?
  2. Should we allow defining improper lists?

License

This document is released on Creative Commons Attribution-ShareAlike 4.0 International License .


以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

高性能Python

高性能Python

(美)戈雷利克、(英)欧日沃尔德 / 东南大学出版社 / 2015-2

你的Python代码也许运行正确,但是你需要运行得更快速。通过探讨隐藏在设计备选方案中的基础理论,戈雷利克和欧日沃尔德编著的《高性能Python》将帮助你更深入地理解Python的实现。你将了解如何定位性能瓶颈,从而显著提升高数据流量程序中的代码执行效率。 你该如何利用多核架构和集群?或者你该如何搭建一个可以自由伸缩而不会影响可靠性的系统?有经验的Python程序员将会学习到这类问题的具体解......一起来看看 《高性能Python》 这本书的介绍吧!

JSON 在线解析
JSON 在线解析

在线 JSON 格式化工具

RGB转16进制工具
RGB转16进制工具

RGB HEX 互转工具

XML 在线格式化
XML 在线格式化

在线 XML 格式化压缩工具