Feature/conversation segmenter

[jasperblues]

PR: Conversation segmentation for incremental extraction

Summary

Adds ConversationSegmenter, a pure utility that splits a timestamped item stream
(e.g. a chat channel's messages) into conversation segments, so each segment can be
handed to IncrementalSource-based extraction as one conversation.

Motivation

Feeding a raw firehose of messages into the incremental analyzer would window purely
by index, cutting mid-conversation. Segmenting first keeps each extraction unit
coherent. This separates two concerns:

• ConversationSegmenter decides conversation boundaries.
• The incremental analyzer (WindowConfig) chunks within a segment by
  size/overlap. Those windows, not whole segments, are what an LLM sees.

Why brief silence is not a boundary

The obvious heuristic — start a new conversation after N minutes of quiet — is wrong
for async chat. On Discord a thread routinely goes quiet for eight or twenty-four
hours while contributors sleep across timezones. Cutting on that silence would shred
exactly the long-running conversations most worth extracting.

So within a conversation, segments are bounded by size, not silence. When size
forces a cut, the cut lands on the largest interior gap, so the seam falls in a
lull rather than mid-exchange.

Why long silence is a boundary

settle is the point at which a trailing conversation is declared finished and handed
to extraction. Once a conversation has been extracted, a later reply must begin a new
segment rather than re-open the extracted one — otherwise the next call re-emits a
superset of what was already consumed. The same duration therefore has to govern both:
a silence longer than settle ends the conversation, and a conversation quiet for
longer than settle is extractable.

A single knob keeps the two consistent. (An earlier draft had a separate maxSilence
ceiling defaulting to null, which contradicted settle: a 24h gap would simultaneously
close a conversation and not be a boundary.)

Behavior

closedSegments(items, now, timestamp, size) returns the closed segments, each a
time-ordered sub-list:

• Items are sorted by timestamp; ties keep input order.
• A silence longer than settle starts a new conversation.
• Within a conversation, a run is split only once it exceeds maxSize, cutting at the
  largest interior gap within the longest prefix that fits. The cut is constrained to
  leave at least minItems behind, so a lull near the head can't strand items in a
  sub-minItems fragment. Ties favour the later index, keeping segments full.
• A segment is closed — safe to extract — once further content follows it, or, for
  the trailing segment, once it has been quiet longer than settle. The still-open
  trailing segment is withheld so an in-progress conversation isn't extracted and then
  re-extracted when the rest arrives.
• An undersized tail segment is folded back into the segment before it rather than
  dropped, so no message is silently lost. Only a whole conversation shorter than
  minItems is dropped.

Pure and deterministic — no LLM, no I/O. Generic over the item type; the caller
supplies a timestamp accessor and, optionally, a size accessor. Each accessor is
invoked exactly once per item, so a costly one (a token estimator) is affordable.

API

class ConversationSegmenter(
    maxSize: Int = DEFAULT_MAX_SIZE,      // 500
    settle: Duration = DEFAULT_SETTLE,    // 24h
    minItems: Int = 1,
) {
    fun <T> closedSegments(
        items: List<T>,
        now: Instant,
        timestamp: (T) -> Instant,
        size: (T) -> Int = { 1 },
    ): List<List<T>>
}

On the defaults

maxSize is not a context-window constraint — the analyzer chunks a segment into
WindowConfig.windowSize (10–20) message windows, and only those reach the model. It
is a coherence cap: past some length, a run of messages is no longer plausibly one
conversation.

The default of 500 is 25–50 analyzer windows wide, is a genuinely long thread, and
— should a whole segment ever be sent to a model — is roughly 20k tokens at typical
message length, comfortably inside any modern context window. A segment absorbing an
undersized tail may exceed it by up to minItems - 1 items; it is a cap on coherence,
not a hard limit.

maxSize is expressed in whatever units the size accessor returns. The default
accessor counts one unit per item, making it a message count; a caller supplying a
token estimator must scale maxSize accordingly. Note that minItems always counts
items — setting it above the number of items maxSize admits discards the stream.

settle defaults to 24h: long enough to respect timezone-spanning replies, short
enough that extraction isn't perpetually stalled.

Idempotency

Segmentation of a given prefix is stable as later items arrive, so repeated calls over
a growing stream emit each closed segment exactly once — provided items are ingested in
timestamp order. An item back-dated to within settle of an already-extracted segment
will re-open it.

Tests

ConversationSegmenterTest (20 tests) covers:

• An overnight (24h) lull does not split a conversation.
• A silence beyond settle splits a dormant channel, and each conversation is then
  independently capped by maxSize.
• Splitting only once over maxSize, cutting at the largest interior gap.
• A run at or under maxSize is never split, however lulled.
• Withholding the still-open trailing segment until it settles; a now preceding the
  last item leaves it open.
• Earlier segments are closed even while the trailing one is open.
• The cut leaves at least minItems behind.
• A settled undersized tail is folded into the preceding segment; an open one is
  withheld rather than folded.
• The size accessor weights items against maxSize, and each accessor is invoked
  once per item.
• A lone item heavier than maxSize becomes its own segment.
• Dropping a whole conversation below the min-item floor; minItems above what
  maxSize admits; empty input; out-of-order input; tied timestamps; zero gaps
  throughout.

Files

• dice/src/main/kotlin/com/embabel/dice/incremental/ConversationSegmenter.kt (new)
• dice/src/test/kotlin/com/embabel/dice/incremental/ConversationSegmenterTest.kt (new)