Skip to content

Arm backend: Add support for StaticCache in TOSA+INT#18124

Open
tom-arm wants to merge 1 commit intopytorch:mainfrom
tom-arm:int8_static_cache
Open

Arm backend: Add support for StaticCache in TOSA+INT#18124
tom-arm wants to merge 1 commit intopytorch:mainfrom
tom-arm:int8_static_cache

Conversation

@tom-arm
Copy link
Collaborator

@tom-arm tom-arm commented Mar 12, 2026

  • Create pass to decompose index_copy to index_put
  • Enable INT tests for all targets except U55
  • Adjust insert cast tests as the graph should delegate

Change-Id: I139728a78884ff1cbdc72568938f7b43f0b12279

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell

* Create pass to decompose index_copy to index_put
* Enable INT tests for all targets except U55
* Adjust insert cast tests as the graph should delegate

Signed-off-by: Tom Allsop <tom.allsop@arm.com>
Change-Id: I139728a78884ff1cbdc72568938f7b43f0b12279
@tom-arm tom-arm requested a review from digantdesai as a code owner March 12, 2026 15:50
Copilot AI review requested due to automatic review settings March 12, 2026 15:50
@pytorch-bot
Copy link

pytorch-bot bot commented Mar 12, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18124

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Awaiting Approval, 17 New Failures, 1 Cancelled Job, 5 Pending

As of commit f05ee4e with merge base 76dfb19 (image):

AWAITING APPROVAL - The following workflow needs approval before CI can run:

NEW FAILURES - The following jobs have failed:

CANCELLED JOB - The following job was cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 12, 2026
@tom-arm tom-arm added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk release notes: arm Changes to the ARM backend delegate labels Mar 12, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the Arm backend’s INT/TOSA lowering path to support transformers.cache_utils.StaticCache by adjusting the pre-quantization graph transformations and enabling previously-xfailed INT tests (with U55 remaining xfailed due to SCATTER limitations).

Changes:

  • Add a new transform-for-annotation decomposition pass to rewrite aten.index_copy{,_} into aten.index_put{,_} ahead of quantization/lowering.
  • Update INT test expectations/config to align with the new decomposition behavior (including disabling fold_quantize where needed for StaticCache).
  • Minor cleanup in RewriteIndexPutPass by removing an unused parameter from _expand_none_indices.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
backends/arm/test/passes/test_insert_int32_casts_after_int64_placeholders_pass.py Updates INT test expectations to match index_copyindex_put decomposition and associated cast op presence.
backends/arm/test/modules/test_static_cache.py Enables TOSA+INT/VGF quant StaticCache tests (removing prior xfails), sets fold_quantize=False, and refines U55 xfail reasoning.
backends/arm/_passes/rewrite_index_put_pass.py Removes an unused parameter from _expand_none_indices and updates its call site.
backends/arm/_passes/insert_int32_casts_after_int64_placeholders.py Drops index_copy{,_} from the “requires i64 input” callsite-cast map now that index_copy is decomposed earlier.
backends/arm/_passes/decompose_index_copy_pass.py Introduces DecomposeIndexCopyPass to decompose aten.index_copy{,_} during transform-for-annotation.
backends/arm/_passes/arm_pass_manager.py Wires DecomposeIndexCopyPass into the transform-for-annotation pipeline.
backends/arm/_passes/init.py Exports DecomposeIndexCopyPass from the passes package.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.



class DecomposeIndexCopyPass(GetDecompositionPass):
"""Decomposes aten.index_copy into aten.index_put, as well as it's
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docstring grammar: "it's" is the contraction for "it is"; here it should be the possessive "its" ("...as well as its surrounding operators").

Suggested change
"""Decomposes aten.index_copy into aten.index_put, as well as it's
"""Decomposes aten.index_copy into aten.index_put, as well as its

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: arm Changes to the ARM backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants