Skip to content

fix(serializer): preserve Decimal values and non-primitive dict keys#1707

Open
i-anubhav-anand wants to merge 2 commits into
langfuse:mainfrom
i-anubhav-anand:fix/serializer-decimal-and-dict-keys
Open

fix(serializer): preserve Decimal values and non-primitive dict keys#1707
i-anubhav-anand wants to merge 2 commits into
langfuse:mainfrom
i-anubhav-anand:fix/serializer-decimal-and-dict-keys

Conversation

@i-anubhav-anand

@i-anubhav-anand i-anubhav-anand commented Jun 14, 2026

Copy link
Copy Markdown

Two cases where EventSerializer silently dropped data in serialized trace input/output:

1. decimal.Decimal values are lost

There is no Decimal branch in _default_inner, so a Decimal falls through to the "<{type}>" fallback:

EventSerializer().encode(Decimal("1.5"))            # -> "\"<Decimal>\""
EventSerializer().encode({"price": Decimal("19.99")})  # -> {"price": "<Decimal>"}

Decimal is common in financial/tool outputs. Fix serializes it via str() so the exact value is preserved (float() would silently round high-precision values and can overflow on very large ones); Decimal("NaN")/Decimal("Infinity") still render as "NaN"/"Infinity".

2. A non-primitive dict key discards the whole dict

The dict branch did {self.default(k): self.default(v) ...}. When a key serializes to a container/object (tuple, set, custom object), the comprehension/JSON encoding raises and the except turns the entire dict into an error string:

EventSerializer().encode({(1, 2): "v"})  # -> "\"<not serializable object of type: dict>\""

Fix stringifies such keys via str(key) so a single non-primitive key no longer drops every value.

Tests

Adds 4 unit tests in tests/unit/test_serializer.py (verified fail-before / pass-after). Existing datetime/uuid/enum/int key behavior is unchanged (existing parametrized test still passes). ruff check/ruff format clean.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Greptile Summary

This PR fixes two silent data-loss bugs in EventSerializer: decimal.Decimal values were falling through to the "<Decimal>" opaque fallback, and a dict with any non-primitive key would discard the entire dict instead of just stringifying that key.

  • Decimal fix: Decimal is now routed through float() before the existing NaN/Infinity handlers, so it serializes as a JSON number. Special Decimal values (NaN, Infinity, -Infinity) are handled correctly. The precision trade-off (float has ~15–17 significant digits) is an intentional design choice but is undocumented.
  • Dict key fix: The dict branch now checks whether a serialized key is a JSON-scalar type, and falls back to str(k) for non-scalars (tuples, sets, custom objects), so a single exotic key no longer discards every value in the dict.
  • Four new unit tests verify both fixes, including special Decimal values, tuple keys, and custom-object keys.

Confidence Score: 4/5

Safe to merge — both fixes address real silent-data-loss regressions, the logic is correct, and the new tests exercise the changed paths including edge cases.

The Decimal-to-float conversion works correctly for normal and special values, but silently narrows high-precision Decimal values to float precision without any documentation or warning. For a tracing library this is unlikely to cause problems in practice, but users who pass high-precision financial Decimals would see quietly rounded values in their trace data.

The Decimal precision trade-off in langfuse/_utils/serializer.py (lines 79–80) is the only area worth a second look; tests/unit/test_serializer.py is straightforward.

Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
langfuse/_utils/serializer.py:79-80
**Decimal precision silently lost on float conversion**

`float()` can only represent ~15–17 significant digits, so any `Decimal` with higher precision is silently rounded. For example, `Decimal("1.0000000000000001")` becomes `1.0` and `Decimal("123456789012345678")` would gain rounding error. Since `Decimal` is often chosen specifically to avoid float imprecision (financial, scientific), users may find their high-precision values quietly corrupted in trace data. A comment documenting this trade-off would help, or alternatively the value could be converted to `str` instead, preserving the exact decimal representation while still avoiding the opaque `"<Decimal>"` fallback.

Reviews (1): Last reviewed commit: "fix(serializer): preserve Decimal values..." | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

EventSerializer silently dropped data in two cases:

- decimal.Decimal had no branch, so it fell through to the "<Decimal>"
  type fallback and the value was lost. Route it through float() so it is
  serialized as a number (and Decimal("NaN")/Decimal("Infinity") reuse the
  existing special-value handlers).
- A dict whose key serializes to a container/object (e.g. tuple, set,
  custom object) raised inside the dict comprehension / json encoding,
  which the except clause turned into "<not serializable ...>" for the
  WHOLE dict. Stringify such keys via str(key) so one non-primitive key
  no longer discards every value in the dict.

Adds unit tests covering both (fail before, pass after). Existing
datetime/uuid/enum/int key behavior is unchanged.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@CLAassistant

CLAassistant commented Jun 14, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Comment thread langfuse/_utils/serializer.py Outdated
Comment on lines +79 to +80
if isinstance(obj, decimal.Decimal):
return self.default(float(obj))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Decimal precision silently lost on float conversion

float() can only represent ~15–17 significant digits, so any Decimal with higher precision is silently rounded. For example, Decimal("1.0000000000000001") becomes 1.0 and Decimal("123456789012345678") would gain rounding error. Since Decimal is often chosen specifically to avoid float imprecision (financial, scientific), users may find their high-precision values quietly corrupted in trace data. A comment documenting this trade-off would help, or alternatively the value could be converted to str instead, preserving the exact decimal representation while still avoiding the opaque "<Decimal>" fallback.

Prompt To Fix With AI
This is a comment left during a code review.
Path: langfuse/_utils/serializer.py
Line: 79-80

Comment:
**Decimal precision silently lost on float conversion**

`float()` can only represent ~15–17 significant digits, so any `Decimal` with higher precision is silently rounded. For example, `Decimal("1.0000000000000001")` becomes `1.0` and `Decimal("123456789012345678")` would gain rounding error. Since `Decimal` is often chosen specifically to avoid float imprecision (financial, scientific), users may find their high-precision values quietly corrupted in trace data. A comment documenting this trade-off would help, or alternatively the value could be converted to `str` instead, preserving the exact decimal representation while still avoiding the opaque `"<Decimal>"` fallback.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in the latest commit — switched to str(obj) so the exact Decimal value is preserved (this also avoids the OverflowError that float() raises on very large Decimals). NaN/Infinity still render as "NaN"/"Infinity". Tests updated to assert exact string output incl. high-precision and >JS-safe-int cases.

Address review feedback: float(Decimal) silently rounds high-precision values
and can OverflowError on very large ones. Use str() to preserve the exact
value (NaN/Infinity still render as "NaN"/"Infinity"/"-Infinity", matching the
float handling). Update tests to assert exact string output incl. high-precision
and >JS-safe-int cases.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants