Skip to content

Conversation

@polina-c
Copy link
Collaborator

@polina-c polina-c commented Jan 17, 2026

Contributes to #607

gemini-code-assist[bot]

This comment was marked as outdated.

@polina-c polina-c changed the title - Add image and thinking parts. Jan 17, 2026
@polina-c polina-c marked this pull request as ready for review January 17, 2026 04:01
@polina-c polina-c requested a review from csells January 17, 2026 04:01
@polina-c
Copy link
Collaborator Author

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces "ImagePart" and "ThinkingPart" to handle image data and model reasoning steps. A high-severity Server-Side Request Forgery (SSRF) vulnerability has been identified in the "ImagePart.fromJson" factory due to unvalidated URL parsing, which could allow access to internal resources. Additionally, the review suggests improving the robustness of "fromJson" methods to handle malformed JSON, adding negative test cases, and a minor style improvement for better code maintainability.

polina-c and others added 3 commits January 16, 2026 20:09
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@polina-c polina-c changed the title Add image and thinking parts. Add image and thinking parts and bump version. Jan 17, 2026
@csells
Copy link
Collaborator

csells commented Jan 17, 2026

How is an image different from another mime type?

@polina-c
Copy link
Collaborator Author

polina-c commented Jan 17, 2026

How is an image different from another mime type?

Couple things:

  1. It can be referenced by URL, and thus visualized by widgets like Image.network, with lazy loading, and with caching on the level of HTTP protocol
  2. It is guaranteed to be some type of image

Clients of the package do not have to use every part, presented in the package.

@csells
Copy link
Collaborator

csells commented Jan 18, 2026

How is an image different from another mime type?

Couple things:

  1. It can be referenced by URL, and thus visualized by widgets like Image.network, with lazy loading, and with caching on the level of HTTP protocol

LinkPart is for data of all kinds, including images, to be referenced by URI.

And can't images be bytes, too? That's what DataParts are for.

I don't understand the value of an additional abstraction for images.

  1. It is guaranteed to be some type of image

So? Why not a PdfPart then? Or a MarkdownPart? Or an ExcelPart? What makes images special?

Clients of the package do not have to use every part, presented in the package.

Actually, many clients of the package DO have to use every part, if they're mapping messages to/from a lower level API. Otherwise, if someone creates a message with an ImagePart, if they're ignored, because they're redundant, then they get dropped on the floor.

And if I have a message come back with a link, does my mapping layer have to detect if it's an image and create an ImagePart instead? Then the consumer of message parts has to write twice the code, too.

@polina-c
Copy link
Collaborator Author

polina-c commented Jan 19, 2026

Yes, your questions are valid.

And, yes, there are ton of ways to model part that represents image.

This PR's code is copied as is from existing implementation of GenUI SDK: additional abstraction for images is a convenient thing for GenUI SDK and may be convenient for other systems.

However, we do not want this to be true: "many clients of the package DO have to use every part, if they're mapping messages to/from a lower level API".

List of part types is expected to grow (for example, ThinkingPart showed up very recently), so we want it to be flexible and extendable, and we do not want to enforce this list to clients.

And, at the same time, we want to enable clarity: what parts are expected in certain system from curtain role.

With this goals in mind (extensibility and clarity) in the current implementation the method fromJson can be parameterized by registry of deserializers exactly with extensibility in mind.

To summarize I see two points of disagreement between us:

  1. Should list of parts be extendable?
  2. Should ImagePart exist in the library?

Thoughts?

@csells
Copy link
Collaborator

csells commented Jan 19, 2026

  • Should list of parts be extendable?

No. I don't think so. Every time there's a new part, that breaks mappings.

  • Should ImagePart exist in the library?

Sure! Can you build it around the base parts, e.g.

class ImagePart // doesn't derive from Part
  ImagePart.data(DataPart data) {...}
  ImagePart.link(LinkPart link) {...}
}

That gives you the convenience and behavior you want without affecting mappings.

@polina-c
Copy link
Collaborator Author

polina-c commented Jan 20, 2026

New parts will definitely appear (as it happened with ThinkingPart), and we do not want to break dependents with every new part. Also we want dependents (including dartantic_ai and those who define their own Message) to be able to define their parts. So, the list of parts should be extendable.

Every time there's a new part, that breaks mappings.

It should not be this way.

Which method is breaking?

About Image: there are many options and what you suggested is a valid one.

@polina-c
Copy link
Collaborator Author

We may think about sealing list of parts for ChatMessage, but then genui will have to define its own Message, because it has its own parts.

@polina-c
Copy link
Collaborator Author

I had discussion with Chris and the suggestion is to have two sets of types:
(naming is up to discussion)
Parts - extendable
ChatParts - sealed, used in ChatMessage

Next step - Polina will create PR.

@polina-c
Copy link
Collaborator Author

Closed at sake of another one: csells/dartantic#91

@polina-c polina-c closed this Jan 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants