tools/utils: Fix issue with loading json files as ascii on python3 linux

[andrewleech]

Description

This resolves #7026, a failure to run mbed-cli tool on linux python 3.5 due to the error:

Traceback (most recent call last):
  File "/project/mbed-os/tools/make.py", line 41, in <module>
    from tools.tests import TESTS, Test, TEST_MAP
  File "/project/mbed-os/tools/tests.py", line 18, in <module>
    from tools.data.support import DEFAULT_SUPPORT, CORTEX_ARM_SUPPORT
  File "/project/mbed-os/tools/data/support.py", line 17, in <module>
    from tools.targets import TARGETS
  File "/project/mbed-os/tools/targets/__init__.py", line 569, in <module>
    update_target_data()
  File "/project/mbed-os/tools/targets/__init__.py", line 558, in update_target_data
    in Target.get_json_target_data().items()
  File "/project/mbed-os/tools/targets/__init__.py", line 87, in wrapper
    CACHES[(func.__name__, args)] = func(*args, **kwargs)
  File "/project/mbed-os/tools/targets/__init__.py", line 159, in get_json_target_data
    Target.__targets_json_location_default)
  File "/project/mbed-os/tools/utils.py", line 372, in json_file_to_dict
    object_pairs_hook=OrderedDict)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

This problem was accidentally introduced in 7b49edc when the recursive function which called str.encode('ascii').decode() was essentially replaced with a one liner which calls str.encode('ascii', 'ignore') without any conversion back to string. It appears some versions / platforms of python handle bytes being passed into json.loads while others don't (it's supposed to take str)

This method is also somewhat inefficient however (though still better than the recursive function). Reading in the file and converting a couple of times before sending into json in unneccesary, the builtin codecs module gives us a much cleaner way of achieving the same goal

Pull request type

[X] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[ ] Breaking change

[cmonr]

Ooh, nice!

@andrewleech Isn't the use of the codec module deprecated?
https://bugs.python.org/issue8796

[andrewleech]

@cmonr good call, looks like I should be using the matching method from io

[andrewleech]

Well that was simple. Turns out io.open is an alias for builtin open in current python3, however it is different in python2 so worth explicitly declaring as such.

[cmonr]

Gonna test this against all other Python versions we support tomorrow, but assuming nothing bad is found, will start CI on it.

[andrewleech]

Any updates?

[cmonr]

@andrewleech My bad. Started looking into this, and found more issues that needed to be fixed, right before I left for a week.

Let's get this in to unblock you, and I'll continue working on those other issues.

[andrewleech]

Sounds good!

[cmonr]

/morph build

[mbed-ci]

Build : SUCCESS

Build number : 3285
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/8177/

Triggering tests

/morph test
/morph export-build
/morph mbed2-build

[mbed-ci]

Test : SUCCESS

Build number : 3092
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/8177/3092

[mbed-ci]

Exporter Build : SUCCESS

Build number : 2910
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/8177/

[cmonr]

/morph mbed2-build