Skip to content

Python3 JSON Adapter byte string is not serializable #13

Description

@Amorik

Use of the python3 branch and the JSON adapter results in the following error:

Process Process-3:
Traceback (most recent call last):
  File "/opt/eNcore/estreamer/baseproc.py", line 208, in receiveInput
    self.onReceive( item )
  File "/opt/eNcore/estreamer/baseproc.py", line 313, in onReceive
    self.onEvent( item )
  File "/opt/eNcore/estreamer/pipeline.py", line 397, in onEvent
    data = transform( item, self.settings )
  File "/opt/eNcore/estreamer/pipeline.py", line 205, in transform
    output = adapters[ index ].dumps( event['record'] )
  File "/opt/eNcore/estreamer/adapters/json.py", line 30, in dumps
    return json.dumps( data )
  File "/usr/local/lib/python3.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

Im not very familiar with all of the code yet but it appears that the dictionary data being dumped by json.dumps contains values of byte/bytestrings/bytesequences. This was fine in python2 since it handled the encoding/decoding but that not longer happens on its own in the python3 version of JSON.

To resolve i switched from import json to import simplejson as simplejson can handle the bytes conversion. This might not be the right approach for this project, but either way the bytes objects needs to be UTF8 encoded to be JSON compliant/serializable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions