Nationwide Polamorous Matrimony
    Have ideas to improve npm?Join in the discussion! »

    msgpack-lite
    DefinitelyTyped icon, indicating that this package has TypeScript declarations provided by the separate @types/msgpack-lite package

    0.1.26 • Public • Published

    msgpack-lite npm version Build Status

    Fast Pure JavaScript MessagePack Encoder and Decoder

    Sauce Test Status

    Online demo: http://kawanet.github.io/msgpack-lite/

    Features

    • Pure JavaScript only (No node-gyp nor gcc required)
    • Faster than any other pure JavaScript libraries on node.js v4
    • Even faster than node-gyp C++ based msgpack library (90% faster on encoding)
    • Streaming encoding and decoding interface is also available. It's more faster.
    • Ready for Web browsers including Chrome, Firefox, Safari and even IE8
    • Tested on Node.js v0.10, v0.12, v4, v5 and v6 as well as Web browsers

    Encoding and Decoding MessagePack

    var msgpack = require("msgpack-lite");
     
    // encode from JS Object to MessagePack (Buffer)
    var buffer = msgpack.encode({"foo": "bar"});
     
    // decode from MessagePack (Buffer) to JS Object
    var data = msgpack.decode(buffer); // => {"foo": "bar"}
     
    // if encode/decode receives an invalid argument an error is thrown

    Writing to MessagePack Stream

    var fs = require("fs");
    var msgpack = require("msgpack-lite");
     
    var writeStream = fs.createWriteStream("test.msp");
    var encodeStream = msgpack.createEncodeStream();
    encodeStream.pipe(writeStream);
     
    // send multiple objects to stream
    encodeStream.write({foo: "bar"});
    encodeStream.write({baz: "qux"});
     
    // call this once you're done writing to the stream.
    encodeStream.end();

    Reading from MessagePack Stream

    var fs = require("fs");
    var msgpack = require("msgpack-lite");
     
    var readStream = fs.createReadStream("test.msp");
    var decodeStream = msgpack.createDecodeStream();
     
    // show multiple objects decoded from stream
    readStream.pipe(decodeStream).on("data", console.warn);

    Decoding MessagePack Bytes Array

    var msgpack = require("msgpack-lite");
     
    // decode() accepts Buffer instance per default
    msgpack.decode(Buffer([0x81, 0xA3, 0x66, 0x6F, 0x6F, 0xA3, 0x62, 0x61, 0x72]));
     
    // decode() also accepts Array instance
    msgpack.decode([0x81, 0xA3, 0x66, 0x6F, 0x6F, 0xA3, 0x62, 0x61, 0x72]);
     
    // decode() accepts raw Uint8Array instance as well
    msgpack.decode(new Uint8Array([0x81, 0xA3, 0x66, 0x6F, 0x6F, 0xA3, 0x62, 0x61, 0x72]));

    Command Line Interface

    A CLI tool bin/msgpack converts data stream from JSON to MessagePack and vice versa.

    echo '{"foo": "bar"}' | ./bin/msgpack -Jm | od -tx1
    0000000    81  a3  66  6f  6f  a3  62  61  72
     
    echo '{"foo": "bar"}' | ./bin/msgpack -Jm | ./bin/msgpack -Mj
    {"foo":"bar"}

    Installation

    $ npm install --save msgpack-lite

    Tests

    Run tests on node.js:

    $ make test

    Run tests on browsers:

    $ make test-browser-local
    open the following url in a browser:
    http://localhost:4000/__zuul

    Browser Build

    Browser version msgpack.min.js is also available. 50KB minified, 14KB gziped.

    <!--[if lte IE 9]>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/es5-shim/4.1.10/es5-shim.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/json3/3.3.2/json3.min.js"></script>
    <![endif]-->
    <script src="https://rawgit.com/kawanet/msgpack-lite/master/dist/msgpack.min.js"></script>
    <script>
    // encode from JS Object to MessagePack (Uint8Array)
    var buffer = msgpack.encode({foo: "bar"});
     
    // decode from MessagePack (Uint8Array) to JS Object
    var array = new Uint8Array([0x81, 0xA3, 0x66, 0x6F, 0x6F, 0xA3, 0x62, 0x61, 0x72]);
    var data = msgpack.decode(array);
    </script> 

    MessagePack With Browserify

    Step #1: write some code at first.

    var msgpack = require("msgpack-lite");
    var buffer = msgpack.encode({"foo": "bar"});
    var data = msgpack.decode(buffer);
    console.warn(data); // => {"foo": "bar"}

    Proceed to the next steps if you prefer faster browserify compilation time.

    Step #2: add browser property on package.json in your project. This refers the global msgpack object instead of including whole of msgpack-lite source code.

    {
      "dependencies": {
        "msgpack-lite": "*"
      },
      "browser": {
        "msgpack-lite": "msgpack-lite/global"
      }
    }

    Step #3: compile it with browserify and uglifyjs.

    browserify src/main.js -o tmp/main.browserify.js -s main
    uglifyjs tmp/main.browserify.js -m -c -o js/main.min.js
    cp node_modules/msgpack-lite/dist/msgpack.min.js js/msgpack.min.js

    Step #4: load msgpack.min.js before your code.

    <script src="js/msgpack.min.js"></script>
    <script src="js/main.min.js"></script>

    Interoperability

    It is tested to have basic compatibility with other Node.js MessagePack modules below:

    Benchmarks

    A benchmark tool lib/benchmark.js is available to compare encoding/decoding speed (operation per second) with other MessagePack modules. It counts operations of 1KB JSON document in 10 seconds.

    $ npm install msgpack msgpack-js msgpack-js-v5 msgpack-unpack msgpack5 notepack
    $ npm run benchmark 10
    operation op ms op/s
    buf = Buffer(JSON.stringify(obj)); 1055200 10000 105520
    obj = JSON.parse(buf); 863800 10000 86380
    buf = require("msgpack-lite").encode(obj); 969100 10000 96910
    obj = require("msgpack-lite").decode(buf); 600300 10000 60030
    buf = require("msgpack").pack(obj); 503500 10001 50344
    obj = require("msgpack").unpack(buf); 560200 10001 56014
    buf = Buffer(require("msgpack.codec").msgpack.pack(obj)); 653500 10000 65349
    obj = require("msgpack.codec").msgpack.unpack(buf); 367500 10001 36746
    buf = require("msgpack-js-v5").encode(obj); 189500 10002 18946
    obj = require("msgpack-js-v5").decode(buf); 408900 10000 40890
    buf = require("msgpack-js").encode(obj); 189200 10000 18920
    obj = require("msgpack-js").decode(buf); 375600 10002 37552
    buf = require("msgpack5")().encode(obj); 110500 10009 11040
    obj = require("msgpack5")().decode(buf); 165500 10000 16550
    buf = require("notepack")().encode(obj); 847800 10000 84780
    obj = require("notepack")().decode(buf); 599800 10000 59980
    obj = require("msgpack-unpack").decode(buf); 48100 10002 4809

    Streaming benchmark tool lib/benchmark-stream.js is also available. It counts milliseconds for 1,000,000 operations of 30 bytes fluentd msgpack fragment. This shows streaming encoding and decoding are super faster.

    $ npm run benchmark-stream 2
    operation (1000000 x 2) op ms op/s
    stream.write(msgpack.encode(obj)); 1000000 3027 330360
    stream.write(notepack.encode(obj)); 1000000 2012 497017
    msgpack.Encoder().on("data",ondata).encode(obj); 1000000 2956 338294
    msgpack.createEncodeStream().write(obj); 1000000 1888 529661
    stream.write(msgpack.decode(buf)); 1000000 2020 495049
    stream.write(notepack.decode(buf)); 1000000 1794 557413
    msgpack.Decoder().on("data",ondata).decode(buf); 1000000 2744 364431
    msgpack.createDecodeStream().write(buf); 1000000 1341 745712

    Test environment: msgpack-lite 0.1.14, Node v4.2.3, Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz

    MessagePack Mapping Table

    The following table shows how JavaScript objects (value) will be mapped to MessagePack formats and vice versa.

    Source Value MessagePack Format Value Decoded
    null, undefined nil format family null
    Boolean (true, false) bool format family Boolean (true, false)
    Number (32bit int) int format family Number (int or double)
    Number (64bit double) float format family Number (double)
    String str format family String
    Buffer bin format family Buffer
    Array array format family Array
    Map map format family Map (if usemap=true)
    Object (plain object) map format family Object (or Map if usemap=true)
    Object (see below) ext format family Object (see below)

    Note that both null and undefined are mapped to nil 0xC1 type. This means undefined value will be upgraded to null in other words.

    Extension Types

    The MessagePack specification allows 128 application-specific extension types. The library uses the following types to make round-trip conversion possible for JavaScript native objects.

    Type Object Type Object
    0x00 0x10
    0x01 EvalError 0x11 Int8Array
    0x02 RangeError 0x12 Uint8Array
    0x03 ReferenceError 0x13 Int16Array
    0x04 SyntaxError 0x14 Uint16Array
    0x05 TypeError 0x15 Int32Array
    0x06 URIError 0x16 Uint32Array
    0x07 0x17 Float32Array
    0x08 0x18 Float64Array
    0x09 0x19 Uint8ClampedArray
    0x0A RegExp 0x1A ArrayBuffer
    0x0B Boolean 0x1B Buffer
    0x0C String 0x1C
    0x0D Date 0x1D DataView
    0x0E Error 0x1E
    0x0F Number 0x1F

    Other extension types are mapped to built-in ExtBuffer object.

    Custom Extension Types (Codecs)

    Register a custom extension type number to serialize/deserialize your own class instances.

    var msgpack = require("msgpack-lite");
     
    var codec = msgpack.createCodec();
    codec.addExtPacker(0x3F, MyVector, myVectorPacker);
    codec.addExtUnpacker(0x3F, myVectorUnpacker);
     
    var data = new MyVector(1, 2);
    var encoded = msgpack.encode(data, {codec: codec});
    var decoded = msgpack.decode(encoded, {codec: codec});
     
    function MyVector(x, y) {
      this.x = x;
      this.y = y;
    }
     
    function myVectorPacker(vector) {
      var array = [vector.x, vector.y];
      return msgpack.encode(array); // return Buffer serialized
    }
     
    function myVectorUnpacker(buffer) {
      var array = msgpack.decode(buffer);
      return new MyVector(array[0], array[1]); // return Object deserialized
    }

    The first argument of addExtPacker and addExtUnpacker should be an integer within the range of 0 and 127 (0x0 and 0x7F). myClassPacker is a function that accepts an instance of MyClass, and should return a buffer representing that instance. myClassUnpacker is the opposite: it accepts a buffer and should return an instance of MyClass.

    If you pass an array of functions to addExtPacker or addExtUnpacker, the value to be encoded/decoded will pass through each one in order. This allows you to do things like this:

    codec.addExtPacker(0x00, Date, [Number, msgpack.encode]);

    You can also pass the codec option to msgpack.Decoder(options), msgpack.Encoder(options), msgpack.createEncodeStream(options), and msgpack.createDecodeStream(options).

    If you wish to modify the default built-in codec, you can access it at msgpack.codec.preset.

    Custom Codec Options

    msgpack.createCodec() function accepts some options.

    It does NOT have the preset extension types defined when no options given.

    var codec = msgpack.createCodec();

    preset: It has the preset extension types described above.

    var codec = msgpack.createCodec({preset: true});

    safe: It runs a validation of the value before writing it into buffer. This is the default behavior for some old browsers which do not support ArrayBuffer object.

    var codec = msgpack.createCodec({safe: true});

    useraw: It uses raw formats instead of bin and str.

    var codec = msgpack.createCodec({useraw: true});

    int64: It decodes msgpack's int64/uint64 formats with int64-buffer object.

    var codec = msgpack.createCodec({int64: true});

    binarraybuffer: It ties msgpack's bin format with ArrayBuffer object, instead of Buffer object.

    var codec = msgpack.createCodec({binarraybuffer: true, preset: true});

    uint8array: It returns Uint8Array object when encoding, instead of Buffer object.

    var codec = msgpack.createCodec({uint8array: true});

    usemap: Uses the global JavaScript Map type, if available, to unpack MessagePack map elements.

    var codec = msgpack.createCodec({usemap: true});

    Compatibility Mode

    The compatibility mode respects for msgpack's old spec. Set true to useraw.

    // default mode handles both str and bin formats individually
    msgpack.encode("Aa"); // => <Buffer a2 41 61> (str format)
    msgpack.encode(new Buffer([0x41, 0x61])); // => <Buffer c4 02 41 61> (bin format)
     
    msgpack.decode(new Buffer([0xa2, 0x41, 0x61])); // => 'Aa' (String)
    msgpack.decode(new Buffer([0xc4, 0x02, 0x41, 0x61])); // => <Buffer 41 61> (Buffer)
     
    // compatibility mode handles only raw format both for String and Buffer
    var options = {codec: msgpack.createCodec({useraw: true})};
    msgpack.encode("Aa", options); // => <Buffer a2 41 61> (raw format)
    msgpack.encode(new Buffer([0x41, 0x61]), options); // => <Buffer a2 41 61> (raw format)
     
    msgpack.decode(new Buffer([0xa2, 0x41, 0x61]), options); // => <Buffer 41 61> (Buffer)
    msgpack.decode(new Buffer([0xa2, 0x41, 0x61]), options).toString(); // => 'Aa' (String)

    Repository

    See Also

    License

    The MIT License (MIT)

    Copyright (c) 2015-2016 Yusuke Kawasaki

    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

    Install

    npm i msgpack-lite

    DownloadsWeekly Downloads

    385,038

    Version

    0.1.26

    License

    MIT

    Last publish

    Collaborators

    • avatar