Threads: Investigate whether we can have pure state only threads, with stateless behaviors living elsewhere

[westey-m]

Background

Today threads can have an attached ChatMessageStore or AIContextProvider that both contribute state to the thread.

Having both the state and behaviors attached to the thread makes it difficult to deserialize a thread without needing an agent based factory, since both ChatMessageStores and AIContextProviders typically require some service client to be constructed.

While deserializing via the factory and serializing via a custom helper on AgentThread is not complex, it is a pattern that users don't expect. They often assume that they should just be able to use the serializer of their choice to serialize and deserialize the thread object.

It also means that deserializing any container object that also contains thread json, cannot just be deserialized in it's entirety.
You also cannot easily just serialize a List<AgentThread>. So none of this is a great user experience.

Option to investigate

We should investigate whether we can have AgentThreads that resemble the state tree instead of resembling the behaviors attached to it. The AgentThread would contain state properties for each type of behavior it supports, e.g. in the case of ChatClientAgentThread, one for ChatMessageStore and one for AIContextProvider. These objects would be directly serializable and deserializable with the AgentThread.

After a thread is deserialized, we would still need to re-create the ChatMessageStore and AIContextProvider that use these state objects and point them at the state objects. We could also set the re-created behavior classes on the AgentThread, but mark them as non-serializable. That way, they get recreated on first use, each time after deserialization, but doesn't stop anyone from serializing the thread as is.

A challenge here is polymorphic deserialization. Since users can attach their own ChatMessageStore/AIContextProvider, that stores its own state, we would need to potentially have a base class for each state type. E.g. a ChatMessageStore base type, that we can annotate with the list of inheriting types. Plus a way for developers to extend these with their own types.

Example

Something along the lines of this.

public class ChatClientAgentThread
{
    public ChatMessageStoreState StoreState { get; set; }
    public AIContextProviderState AIContextProviderState { get; set; }

    [JsonIgnore]
    internal ChatMessageStore? ChatMessageStore { get; set; }
    [JsonIgnore]
    internal AIContextProvider? AIContextProvider { get; set; }
}

[JsonPolymorphic(TypeDiscriminatorPropertyName = "$type")]
[JsonDerivedType(typeof(InMemoryChatMessageStoreState), typeDiscriminator: "inmemory")]
public abstract class ChatMessageStoreState {}

public InMemoryChatMessageStoreState : ChatMessageStoreState 
{
    public List<ChatMessage> Messages { get; set; } = [];
}

public InMemoryChatMessageStore : ChatMessageStore
{
    public InMemoryChatMessageStore(InMemoryChatMessageStoreState state) {}
    public InMemoryChatMessageStore() {};

    public ChatMessageStoreState State { get; }

    ...
}

public class ChatClientAgent
{
    public Task<AgentRunResponse> RunAsync(AgentThread thread, ...)
    {
        var typedThread = thread as ChatClientAgentThread!;
        if (typedThread.StoreState is null)
        {
            typedThread.ChatMessageStore = new InMemoryChatMessageStore();
            typedThread.StoreState = typedThread.ChatMessageStore.State;
        }
        thread.ChatMessageStore ??= typedThread.ChatMessageStore =
            new InMemoryChatMessageStore(typedThread.StoreState);

        ...
    }
}

[westey-m]

Addressed via #3806, with SessionStateBags on Session.