GH-48868: [Doc] Document security model for the Arrow formats #48870
+151
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
Component: Documentation
awaiting review
Member
Author
|
@github-actions crossbow submit preview-docs |
|
Revision: 593babb Submitted crossbow builds: ursacomputing/crossbow @ actions-4f7018459b
|
raboof
reviewed
Jan 15, 2026
Member
raboof
left a comment
raboof
left a comment
There was a problem hiding this comment.
Looks reasonable (without any particular Arrow expertise)
(noticed two typo's)
| ------------------ | ||
|
|
||
| A less obvious pitfall is when some parts of an Arrow array are left uninitialized. | ||
| For example, if a element of a primitive Arrow array is marked null through its |
Member
There was a problem hiding this comment.
Suggested change
| For example, if a element of a primitive Arrow array is marked null through its | |
| For example, if an element of a primitive Arrow array is marked null through its |
| purposes. It is therefore tempting, when creating an array with null values, to | ||
| not initialize the corresponding value slots. | ||
|
|
||
| However, this then introduces a serious security if the Arrow data is serialized |
Member
There was a problem hiding this comment.
Suggested change
| However, this then introduces a serious security if the Arrow data is serialized | |
| However, this then introduces a serious security risk if the Arrow data is serialized |
github-actions
bot
added
awaiting committer review
felipecrv
reviewed
Jan 15, 2026
| uninitialized in a buffer if the array might be sent to, or read by, a untrusted | ||
| third-party, even when the uninitialized data is logically irrelevant. The | ||
| easiest way to do this, though perhaps not the most efficient, is to zero-initialize | ||
| any buffer that will not be populated in full. |
Contributor
There was a problem hiding this comment.
Worth pointing out something about query engines and dataframe libraries deciding to not do so for internal/intermediate values in computations but applying a canonicalization pass when data leaves the system.
| from an untrusted source (for example because you are writing a proxy to | ||
| an arbitrary third-party service), it is **recommended** that you validate | ||
| the data first, as the consumer may assume that the data is valid already. | ||
|
|
Contributor
There was a problem hiding this comment.
Suggested change
| In addition to invalid pointers, some array types have offsets, sizes, and buffer indices that might be out-of-bounds. The library producing arrays through the | |
| C data interface might be performing only very light validation of these values. |
github-actions
bot
added
awaiting changes
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Accessing Arrow data or any of the formats can have non-trivial security implications, this is an attempt at documenting those.
What changes are included in this PR?
Add a Security Considerations page in the Format section.
Doc preview: https://s3.amazonaws.com/arrow-data/pr_docs/48870/format/Security.html
Are these changes tested?
N/A
Are there any user-facing changes?
No.