UnicodeEncodeError when running keras.model.predict

[geniusonanotherlevel]

The following problem occurs in keras version 3.1.1 (tensorflow version 2.16.1)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 19-38: character maps to

prediction = trained_nn_model.predict(last_sequence)
A bit more details:
File "...\lib\site-packages\keras\src\utils\traceback_utils.py", line 122, in error_handler
raise e.with_traceback(filtered_tb) from None
File "....pyenv\pyenv-win\versions\3.10.5\lib\encodings\cp1252.py", line 20, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 19-38: character maps to

(The positions varies when tried with different test data).
The code works ok with keras version 2.15.0 (tensorflow 2.15.0)

[SuryanarayanaY]

Hi @geniusonanotherlevel ,

The issue seems to be with preprocessing steps. With the given info I am not sure to say whether this is Keras issue or not.
It seems the error is related to encoding and may be adding encoding="utf-8" might helps. If it not resolved with this
please provide more details along with a repro code snippet.

Thanks!

[ryand626]

I’m seeing this in Windows 11, but not in Windows 10, or Mac.

[ryand626]

It was working with Keras <3.0, but after updating to 3.1.1 it breaks for Windows 11 runs executed by Snakemake in a git bash conda environment.

If I copy/paste the same command Snakemake ran into the git bash conda environment and run it: it succeeds.

I have set the KERAS_BACKEND environment variable to tensorflow, but I suspect there are other environment shenanigans at play I am unaware of.

I get the same 19-38: character maps to <undefined> error when running model.fit as the OP (exact position match).

[geniusonanotherlevel]

Hi @SuryanarayanaY , thanks for the response. It can be noted that I did try to add encoding of utf-8 with the numpy array, but that wasn't an acceptable option as it only accepted ascii, latin & something else which I can't remember. But the fact that it works with an earlier version of tensorflow/keras while all the pre-processing steps and versions of other packages are exactly the same, is why I suspected keras. I've also noted your request for repro code, I will try and get that next week.

@ryand626 , I'm running on Windows 10.

[SuryanarayanaY]

Hi @geniusonanotherlevel ,

I will be waiting for your repro code. Thanks!

[SuryanarayanaY]

Hi @geniusonanotherlevel ,

Could you please provide repro for same. Thanks!

[geniusonanotherlevel]

Hi @geniusonanotherlevel ,

Could you please provide repro for same. Thanks!

Hi, thanks for the reminder. I don't have the code with me atm, so I will do when I go back to the office (I'm just not sure which day I'll be there yet... but soon) Thanks for your patience.

[github-actions]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.