# serialize.js: Extended JSON serilization JSON-like extended serialization/deserialization, and serialization-based object isolated deep and semi-deep copying library. This extends the default JSON specification adding the following: - Recursive data structure serialization - Sparse array serialization - `undefined`/`NaN` serialization - Serialization of `Infinity`, `BigInt`'s, `Set`'s, `Map`'s - Function serialization - Deep and partial-deep cleen object copy Data not stored: - Attributes on arrays, maps, sets, and functions, - Function closures. Relaxed JSON input format: - Comments are treated as whitespace and are stripped from input - Trailing commas are ignored - All JavaScript string quotes are supported - Object keys are optionally quoted if they confirm to JavaScript identifier spec Note that this a strict superset of JSON, this if only JSON-supported data is serialized the output will be strict JSON. ## Motivation This was originally built as a companion to a testing module for a programming class, illustrating several concepts, including: guaranteed clean isolation of data structures via serialization, instrumenting code and tooling design, basic parsing, among others. ## Installation For basic use: ```shell $ npm install ig-serilaize ``` Or just download and drop [serialize.js](./serialize.js) into your code. ```javascript var serialize = require('ig-serialize') ``` ## Introduction `serialize.js` provides two toolsets: 1. A means to serialize and deserialize complex data structures into an extended JSON format. ```javascript var obj = { sparse_array: [,,,,1], bad_number: NaN, really_large_number: 99999999999999999999999n, } obj.recursive = obj obj.re_reference = obj.sparse_array var str = serialize(obj) // ... var copy = deserialize(str) ``` This is useful when requiering serialization of data structures more complex than pure JSON can handle. 2. A means to cleanly copy deep data structures with guaranteed isolation. ```javascript var obj = { // ... } var copy = deepCopy(obj) ``` ### Long strings and large `BigInt`'s Repeating strings and `BigInt`'s longer that `MIN_LENGTH_REF` are stored by reference by default. See: [`MIN_LENGTH_REF`](#min_length_ref--optionsmin_length_ref) ### Serializing functions Due to how JavaScript is designed it is not possible to trivially and fully clone a function with all of it's references, `.serilaize(..)` will not attempt to clone any state a function may have, this will lead to loosing: - Function closure - Attributes set on the function or any of it's prototypes, including the `.__proto__` value if it was changed. Thus, care must be taken when serializing structures containing function. ## API ### `eJSON` An `JSON`-api-compatible object providing [`.stringify(..)`](#serialize--ejsonstringify) and [`.parse(..)`](#deserialize--ejsonparse) static methods. ### `serialize(..)` / `eJSON.stringify(..)` Serialize a JavaScript value into a JSON/eJSON string. ``` serialize() eJSON.stringify() -> ``` More control: ``` serialize(obj, options){ serialize(obj, indent, depth=0, options){ -> ``` Options format: ``` { // pretty-printing indent... // (default: undefined) indent: undefined, // outout root indent... // (default: 0) depth: 0, // minimal referenced string/bigint length... // (default: MIN_LENGTH_REF) min_length_ref: MIN_LENGTH_REF, // functions list... // (default: undefined) functions: undefined, } ``` Supported options: - `indent` controls formatting and nested value indent, if set to a number that number of spaces will be used to indent nested values if given a string that string is used for indenting, note that only whitespace is supported currently. Default: `undefined` (disabled) - `depth` if given is a number of `indent`'s, used to set top level indent depth of the returned string, this can be useful when pretty-printing or nesting the output. Default: `0` - `min_length_ref` sets the minimal length of a string or big-int value for referencing when encountered repeatedly. If set to `0` or `Infinity` referencing of strings and big-ints will be is disabled. Default: 'MIN_LENGTH_REF' - `functions` if passed an array, encounterd functions will be pushed to it and stored in the output by index. Default: `undefined` ### `deserialize(..)` / `eJSON.parse(..)` Deserialize a JSON/eJSON into a value. ``` deserialize() eJSON.parse() -> ``` Deserializing function is disabled by default as it can be a security risk if the eJSON came from an untrusted source. Enable function deserialization: ``` deserialize(, true) eJSON.parse(, true) deserialize(, {functions: true}) eJSON.parse(, {functions: true}) -> ``` Passing a function list (generated by `serialize(, {functions: })`) for deserialization: ``` deserialize(, {functions: }) eJSON.parse(, {functions: }) -> ``` ### `toVanilla(..)` Convert eJSON formatted string to Vanilla JSON string. ``` toVanilla() -> ``` Note that this will fail on anny data not supported by the Vanilla `JOSN.stringify(..)`. ### `deepCopy(..)` Deep-copy an object. ``` deepCopy() -> ``` The returned object is a fully sanitized through serialization clean copy. ### `partialDeepCopy(..)` Partially deep-copy and object, retaining only references to functions. ``` partialDeepCopy() -> ``` The returned object is a partially sanitized through serialization clean copy with function references copied as-is from the input retaining and "transferring" all associated function state like attributes and closures. Note that this is by definition a _controlled state leak_ from input object to copy, so care must be taken to _control_ object-specific state in function closures and attributes -- keeping function state independent from object state is _in general_ a good practice. ### `MIN_LENGTH_REF` / `.min_length_ref` Defines the default minimum length of repeating string or `BigInt`'s to include as a reference in the output. If set to `0`, referencing will be disabled. Default: 96 ## Format The output of `.serialize(..)` is a strict superset of [standard JSON](https://www.json.org/json-en.html), while the input format is a bit more relaxed than in several details. ### Structural paths Paths are used for internal references in cases when objects are encountered multiple times, e.g. in recursion. A path is an array of keys, the semantics of each key depend on the data structure traversed: - `Array` expects a number - `Set` expects a number -- item order in set - `Map` expects pair of consecutive numbers -- the first indicates item order the second if `0` selects the key, if `1` selects the value. - `Object` expects a string An empty path indicates the root object. Notes: - String path items are unambiguous and are always treated as attributes. This enables referencing of attributes of any object like arrays, maps, ...etc. - Map/set paths are structured as if sets and maps are represented by arrays structured as their respective constructors expect as input. ### Referencing If an object is encountered for a second time it will be serialized as a reference by path to the first occurrence. Grammar: ```bnf ::= '' | ' ']> ::= | ',' ::= | ``` Example: ```javascript // a recursive array... var o = [] o.o = o // root object reference... serialize(o) // -> '[]' // array item... serialize([o]) // -> '[[]]' // set item... // NOTE: the path here is the same as in the above example -- since we // use ordered topology for paths sets do not differ from arrays. serialize(new Set([o])) // -> 'Set([[]])' // map key... serialize(new Map([[o, 'value']])) // -> 'Map([[[],"value"]])' // map value... serialize(new Map([['key', o]])) // -> 'Map([["key",[]]])' ``` ### Null types In addition to `null`, serialize.js adds support for `undefined` and `NaN` which are stored as-is. Example: ```javascript serialize([null, undefined, NaN]) // -> '[null,undefined,NaN]' ``` ### Sparse arrays Sparse arrays are represented in the same way JavaScript handles them syntactically -- with commas separating empty "positions". Example: ```javascript serialize([,]) // -> '[,]' serialize([,,]) // -> '[,,]' ``` Trailing commas are handled in the same way JavaScript handles them: ```javascript // trailing commas are ignored... serialize([1,]) // -> '[1]' // sparse element... serialize([1,,]) // -> '[1,,]' ``` ### BigInt Serialized as represented in JavaScript. ```javascript serialize(9999999999n) // -> '9999999999n' ``` ### Infinity Serialized as represented in JavaScript ```javascript serialize(Infinity) // -> 'Infinity' serialize(-Infinity) // -> '-Infinity' ``` ### Map / Set Maps and sets are stored in the same format as their respective constructor calls, dropping the `new` keyword: Grammar: ```bnf ::= 'Map([]) | 'Map([' ']) ::= | ',' ::= '[' ',' ']' ``` ```bnf ::= 'Set([]) | 'Set([' ']) ::= | ',' ``` Examples: ```javascript serialize(new Set([1,2,3])) // -> 'Set([1,2,3])' serialize(new Map([['a', 1], ['b', 2]])) // -> 'Map([["a",1],["b",2]])' ``` ### Functions A function can be stored in one of two ways: 1. As code (default) 2. As a function index, if an array to store function references is given. Grammar: ```bnf ::= ' ',(' ')]> ::= | ::= ::= ``` `` is the length of the next block in chars, including the braces. ```javascript serialize(function(){}) // -> '' var functions = [] serialize(function(){}, {functions}) // -> '' ``` Note that deserializing functions is disabled by default as it can pose a security risk if the input to `deserialize(..)` is not trusted. (see: [`deserialize(..)`](#deserialize--ejsonparse)) ## Running tests `serialize.js` uses ['test.js'](/flynx/test.js) for testing. Get the development dependencies: ```shell $ npm install -D ``` Run the tests: ```shell $ npm test ``` To run the tests directly: ```shell $ ./test.js ``` To run the tests with modifier chains of length 3: ```shell $ ./test.js -m 3 ``` ## Alternatives ### https://github.com/WebReflection/flatted A very similar library that takes a different approach to the output format (not JSON-compatible), lacks copying utils, and does not support function serialization, but is smaller, and adds other language support. ### https://github.com/ungap/structured-clone/ As above but slightly different approach to things, more types, cloning. ## License [BSD 3-Clause License](./LICENSE) Copyright (c) 2014-2026, Alex A. Naanou, All rights reserved.