"Styled string" representation doesn't handle binary data properly

[wolframroesler]

Since version 1.8.4 it seems that converting Json::Values to JSON strings and back messes up binary data. The following fails:

  std::string s = /* a string with random bytes, not properly UTF-8 encoded */;
  Json::Value a = s;
  std::istringstream iss(a.toStyledString());
  Json::Value b;
  iss >> b;
  JSONTEST_ASSERT(b.asString()==s);

Submitted pull request #725 with a test case that reproduces this behavior.

[wolframroesler]

Any feedback from anyone?

Is binary data in JSON strings within jsoncpp's specification in the first place, or did it just happen to work in previous versions? I'm aware that the JSON specification says that strings may contain "any-Unicode-character-except-"-or--or-control-character", so I'm a bit on shaky grounds by putting random binary data into strings and expecting them to be preserved, but don't we all love to tailor JSON to our needs ... so if anyone decided whether the current behavior is a bug or a feature I'd be very grateful.

[cdunn2001]

(See discussion at #687.)

Yes, it just happened to work. Code was recently added for proper Unicode escapes, but it seems that it should be by configuration only. Could you provide such configuration in a PR?

[wolframroesler]

I'll try. What kind of configuration? Compile time (#ifdef) or runtime (something like void Value::UseUnicodeEscapesWhenConvertingStrings(bool))? A function parameter like Value::toStyledString(bool UseUnicodeEscapes) would be nice but doesn't seem feasible since the escaping happens deep down the call stack (in valueToQuotedStringN as it seems at first glance).

[cdunn2001]

Have you seen the "settings" JSON object? It's used to build a CharReader or StreamWriter.

[wolframroesler]

Seen it, settings object as described here:

• http://open-source-parsers.github.io/jsoncpp-docs/doxygen/class_json_1_1_char_reader_builder.html#a03ff031e06aabff989ab4addc87294ab
• http://open-source-parsers.github.io/jsoncpp-docs/doxygen/class_json_1_1_char_reader_builder.html#ac69b7911ad64c171c51ebaf2ea26d958

and

• http://open-source-parsers.github.io/jsoncpp-docs/doxygen/class_json_1_1_stream_writer_builder.html#a53bf106b141e28637b01ad0ecd2acbf6
• http://open-source-parsers.github.io/jsoncpp-docs/doxygen/class_json_1_1_stream_writer_builder.html#a79bdf2e639a52f4e758c0b95bd1d3423

[wolframroesler]

Thought about it and don't think it's a good idea anymore. JSON strings are defined to contain UTF-8, not arbitrary binary data. Applications shouldn't rely on a library's ability to go beyond the JSON standard.

Issue closed.