Skip to content

fix(core): preserve bytes in MemoryContent across JSON round-trip - #8482

Open
AILICute (lkuko) wants to merge 1 commit into
microsoft:mainfrom
lkuko:fix/memory-content-bytes-serialization
Open

AILICute (lkuko) wants to merge 1 commit into
microsoft:mainfrom
lkuko:fix/memory-content-bytes-serialization

Conversation

@lkuko

Copy link
Copy Markdown

Why are these changes needed?

MemoryContent.content is typed as Union[str, bytes, Dict[str, Any], Image], and MemoryMimeType.BINARY exists for binary payloads. However, only mime_type had a custom serializer. In pydantic's JSON mode, bytes are UTF-8 decoded into str, which means:

  • Non-UTF-8 bytes (e.g. a PNG header b"\x89PNG\r\n\x1a\n") raise UnicodeDecodeError on model_dump(mode="json") / model_dump_json() / BaseMessage.dump() / team.save_state() — a hard crash.
  • UTF-8 bytes (e.g. b"hello") are silently converted to str, so the type is lost on load: model_validate(dumped).content is "hello", not b"hello".

The Image member already has its own base64 (de)serialization, so bytes being left out is an asymmetry rather than an intentional design. This matters for the JSON persistence path that MemoryContent is meant to support (see the earlier serialization work on mime_type).

Reproduction on main:

from autogen_core.memory import MemoryContent, MemoryMimeType

MemoryContent(content=b"\x89PNG\r\n\x1a\n", mime_type=MemoryMimeType.BINARY).model_dump(mode="json")
# UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0

d = MemoryContent(content=b"hello", mime_type=MemoryMimeType.BINARY).model_dump(mode="json")
MemoryContent.model_validate(d).content  # 'hello' (str), not b'hello'

What changed

  • Added a when_used="json" field_serializer for content that encodes bytes as a base64 data URI (data:application/octet-stream;base64,...). Other content types (str, dict, Image) are returned unchanged and serialized by pydantic as usual.
  • Added a mode="before" field_validator that restores those data URIs back to bytes on load, so JSON round-trips are lossless.
  • Python-mode model_dump() (non-JSON) still returns raw bytes, so existing in-memory behavior is unchanged.

Checks

  • I've added tests corresponding to the changes introduced in this PR.
  • I've made sure all auto checks have passed.

Tests: added test_memory_content_bytes_json_roundtrip in python/packages/autogen-core/tests/test_memory.py, covering non-UTF-8 bytes (no crash, round-trips as bytes), UTF-8 bytes (no silent coercion to str), unchanged str/dict content, and unchanged python-mode dump.

Local verification:

  • pytest python/packages/autogen-core/tests/test_memory.py -q → 11 passed (10 existing + 1 new); without the fix the new test fails with UnicodeDecodeError / AssertionError.
  • ruff check on both changed files → clean.
  • mypy (repository pyproject.toml strict config) on _base_memory.py → Success: no issues found.

Related issue number

N/A — found while auditing JSON round-trip behavior of MemoryContent.

MemoryContent.content is typed as Union[str, bytes, Dict[str, Any], Image],
but only mime_type had a custom serializer. In JSON mode pydantic UTF-8
decodes bytes into str (silently changing the type) or raises
UnicodeDecodeError for non-UTF-8 data (e.g. PNG headers).

Add a when_used="json" field serializer that encodes bytes as a base64 data
URI, and a mode="before" field validator that restores them, so bytes content
survives JSON serialization/deserialization losslessly. Other content types
(str, dict, Image) are unchanged.
@lkuko

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant