-
Notifications
You must be signed in to change notification settings - Fork 119
Add image and thinking parts and bump version. #670
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
/gemini review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces "ImagePart" and "ThinkingPart" to handle image data and model reasoning steps. A high-severity Server-Side Request Forgery (SSRF) vulnerability has been identified in the "ImagePart.fromJson" factory due to unvalidated URL parsing, which could allow access to internal resources. Additionally, the review suggests improving the robustness of "fromJson" methods to handle malformed JSON, adding negative test cases, and a minor style improvement for better code maintainability.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
How is an image different from another mime type? |
Couple things:
Clients of the package do not have to use every part, presented in the package. |
LinkPart is for data of all kinds, including images, to be referenced by URI. And can't images be bytes, too? That's what DataParts are for. I don't understand the value of an additional abstraction for images.
So? Why not a PdfPart then? Or a MarkdownPart? Or an ExcelPart? What makes images special?
Actually, many clients of the package DO have to use every part, if they're mapping messages to/from a lower level API. Otherwise, if someone creates a message with an ImagePart, if they're ignored, because they're redundant, then they get dropped on the floor. And if I have a message come back with a link, does my mapping layer have to detect if it's an image and create an ImagePart instead? Then the consumer of message parts has to write twice the code, too. |
|
Yes, your questions are valid. And, yes, there are ton of ways to model part that represents image. This PR's code is copied as is from existing implementation of GenUI SDK: additional abstraction for images is a convenient thing for GenUI SDK and may be convenient for other systems. However, we do not want this to be true: "many clients of the package DO have to use every part, if they're mapping messages to/from a lower level API". List of part types is expected to grow (for example, ThinkingPart showed up very recently), so we want it to be flexible and extendable, and we do not want to enforce this list to clients. And, at the same time, we want to enable clarity: what parts are expected in certain system from curtain role. With this goals in mind (extensibility and clarity) in the current implementation the method To summarize I see two points of disagreement between us:
Thoughts? |
No. I don't think so. Every time there's a new part, that breaks mappings.
Sure! Can you build it around the base parts, e.g. class ImagePart // doesn't derive from Part
ImagePart.data(DataPart data) {...}
ImagePart.link(LinkPart link) {...}
}That gives you the convenience and behavior you want without affecting mappings. |
|
New parts will definitely appear (as it happened with ThinkingPart), and we do not want to break dependents with every new part. Also we want dependents (including dartantic_ai and those who define their own Message) to be able to define their parts. So, the list of parts should be extendable.
It should not be this way. Which method is breaking? About Image: there are many options and what you suggested is a valid one. |
|
We may think about sealing list of parts for ChatMessage, but then genui will have to define its own Message, because it has its own parts. |
|
I had discussion with Chris and the suggestion is to have two sets of types: Next step - Polina will create PR. |
|
Closed at sake of another one: csells/dartantic#91 |
Contributes to #607