Fix BitArray to clear dangling high bits in LeftShift and deserialization #123296

Copilot · 2026-01-16T23:57:15Z

Description

BitArray.LeftShift was not clearing unused bits in the most significant storage element after shifting. This caused PopCount to return incorrect values when bits were shifted out of range.

BitArray ba = new(33);
ba[^1] = true;

ba.LeftShift(1);
Assert.False(ba.HasAnySet());   // passes
Assert.Equal(0, ba.PopCount()); // failed: returned 1

Additionally, the deserialization constructor was not clearing unused high bits after copying data from the serialized int[] array, which could leave garbage bits if the serialized data came from older versions or was malformed.

Changes

Call ClearHighExtraBits() after shift operation completes in LeftShift
Call ClearHighExtraBits() in the deserialization constructor to ensure the invariant is maintained when deserializing data
Add comprehensive Theory-based tests for both LeftShift and RightShift operations validating PopCount and HasAnySet with various bit patterns and shift counts
Test coverage includes edge cases around int32 boundaries (1, 8, 31, 32, 33, 64, 65 bits), various shift amounts, and bit positions at start, middle, and end

Original prompt

This section details on the original issue you should resolve

<issue_title>BitArray ShiftLeft and PopCount produce unexpected behavior</issue_title>
<issue_description>Consider:

[Fact]
public static void PopCount_LeftShiftLeavesDanglingBits()
{
    BitArray ba = new(33);
    ba[^1] = true;

    Assert.True(ba.HasAnySet());
    Assert.Equal(1, ba.PopCount());

    ba = ba.LeftShift(1);
    Assert.False(ba.HasAnySet());
    Assert.Equal(0, ba.PopCount()); // Fails here
}

This test will fail. Despite HasAnySet returning false, PopCount will return 1.

Likewise manually counting the bits will disagree with PopCount

[Fact]
public static void PopCount_LeftShiftLeavesDanglingBits()
{
    BitArray ba = new(33);
    ba[^1] = true;

    Assert.Equal(ManualCountBits(ba), ba.PopCount());

    ba = ba.LeftShift(1);
    Assert.False(ba.HasAnySet());
    Assert.Equal(ManualCountBits(ba), ba.PopCount());


    static int ManualCountBits(BitArray ba)
    {
        int count = 0;

        foreach (bool bit in ba)
        {
            if (bit)
            {
                count++;
            }
        }

        return count;
    }
}
```</issue_description>

<agent_instructions>Fix leftshift to ensure the extra bits are cleared if necessary</agent_instructions>

## Comments on the Issue (you are @copilot in this section)

<comments>
<comment_new><author>@tannergooding</author><body>
Looks like `LeftShift` (and potentially other operations) are not clearing the "unused" upper bits of the most significant storage element in the backing array.

There is probably a balance between every operation ensuring that it only writes to the valid data (i.e. fixing LeftShift) and ensuring operations which might read "unused" bits mask them off (i.e. fixing PopCount, RightShift, etc)</body></comment_new>
</comments>

Fixes BitArray ShiftLeft and PopCount produce unexpected behavior #123281

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: stephentoub <[email protected]>

src/libraries/System.Collections/tests/BitArray/BitArray_OperatorsTests.cs

Copilot

Pull request overview

This pull request fixes a bug in BitArray.LeftShift where unused high-order bits in the most significant storage element were not being cleared after the shift operation, causing PopCount() to return incorrect values when bits were shifted out of range.

Changes:

Added a call to ClearHighExtraBits() in the LeftShift method to clear dangling high bits after shifting
Added regression test LeftShift_PopCountReturnsCorrectCount to verify the fix

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
src/libraries/System.Private.CoreLib/src/System/Collections/BitArray.cs	Added `ClearHighExtraBits()` call after left shift operation to clear unused high bits
src/libraries/System.Collections/tests/BitArray/BitArray_OperatorsTests.cs	Added regression test that verifies PopCount returns 0 after shifting the last bit out of range

Co-authored-by: stephentoub <[email protected]>

tannergooding

Are there any other missed edge cases where the upper bits might be set?

We exposed Span<byte> CollectionsMarshal.AsBytes(BitArray array) as well, in which case a user can set the "high extra bits" themselves. Are we leaving that as undefined behavior or is it something we want to explicitly handle for APIs like PopCount?

stephentoub · 2026-02-05T19:36:08Z

Are we leaving that as undefined behavior or is it something we want to explicitly handle

We've been leaving that as undefined behavior, and methods that change them need to clear them to leave those bits 0s. The alternative or addition if we want is every method that might care needs to clear those bits first.

Are there any other missed edge cases where the upper bits might be set?

@copilot, have we missed any other cases anywhere in BitArray where these upper unused bits should be cleared and aren't?

Co-authored-by: stephentoub <[email protected]>

Copilot · 2026-02-05T19:58:46Z

Are we leaving that as undefined behavior or is it something we want to explicitly handle

We've been leaving that as undefined behavior, and methods that change them need to clear them to leave t...

I audited all methods in BitArray and found one additional case: the deserialization constructor was copying data from the serialized int[] array without clearing the high bits. If the serialized data came from an older version or was malformed, the high extra bits could be set.

I've added ClearHighExtraBits() at the end of the deserialization constructor in commit 611026f.

The other operations are safe:

And/Or/Xor: These rely on both operands having clean high bits (the invariant), so they preserve the invariant
RightShift: Already masks the high bits explicitly before shifting
Set/Get: Bounds-checked to only access valid bit positions
Constructors from byte[]/bool[]/int[]: Either copy exact multiples of their element sizes, or only write to valid bit positions

stephentoub · 2026-02-11T04:14:02Z

/ba-g failures are unrelated

Initial plan

51b9d17

Copilot AI assigned Copilot and stephentoub Jan 16, 2026

Copilot started work on behalf of stephentoub January 16, 2026 23:57 View session

Fix BitArray.LeftShift to clear dangling high bits

60a3afb

Co-authored-by: stephentoub <[email protected]>

Copilot AI changed the title ~~[WIP] Fix BitArray ShiftLeft and PopCount unexpected behavior~~ Fix BitArray.LeftShift to clear dangling high bits Jan 17, 2026

Copilot AI requested a review from stephentoub January 17, 2026 00:25

Copilot finished work on behalf of stephentoub January 17, 2026 00:25

build-analysis bot mentioned this pull request Jan 17, 2026

browser-wasm linux LibraryTests build failure - NETSDK1022: Duplicate 'Content' items #123237

Closed

stephentoub reviewed Jan 17, 2026

View reviewed changes

src/libraries/System.Collections/tests/BitArray/BitArray_OperatorsTests.cs Show resolved Hide resolved

stephentoub marked this pull request as ready for review January 17, 2026 16:30

Copilot AI review requested due to automatic review settings January 17, 2026 16:30

stephentoub added the area-System.Collections label Jan 17, 2026

Copilot started work on behalf of stephentoub January 17, 2026 16:30 View session

Copilot started reviewing on behalf of stephentoub January 17, 2026 16:31 View session

Copilot AI reviewed Jan 17, 2026

View reviewed changes

Copilot AI and others added 2 commits January 17, 2026 16:51

Add comprehensive shift+PopCount tests as Theory with various patterns

b649b14

Co-authored-by: stephentoub <[email protected]>

Improve test data generator readability

066cb7c

Co-authored-by: stephentoub <[email protected]>

Copilot AI requested a review from stephentoub January 17, 2026 16:54

Copilot finished work on behalf of stephentoub January 17, 2026 16:54

Merge branch 'main' into copilot/fix-bitarray-shift-and-popcount

0fc8e50

tannergooding approved these changes Feb 5, 2026

View reviewed changes

Copilot started work on behalf of stephentoub February 5, 2026 19:36 View session

Add ClearHighExtraBits call in deserialization constructor

611026f

Co-authored-by: stephentoub <[email protected]>

Copilot AI changed the title ~~Fix BitArray.LeftShift to clear dangling high bits~~ Fix BitArray to clear dangling high bits in LeftShift and deserialization Feb 5, 2026

Copilot finished work on behalf of stephentoub February 5, 2026 19:59

stephentoub enabled auto-merge (squash) February 5, 2026 20:55

build-analysis bot mentioned this pull request Feb 5, 2026

System.Transactions.Tests.OleTxTests.Recovery timeout #122621

Open

stephentoub merged commit 6641d63 into main Feb 11, 2026
144 of 147 checks passed

stephentoub deleted the copilot/fix-bitarray-shift-and-popcount branch February 11, 2026 04:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix BitArray to clear dangling high bits in LeftShift and deserialization #123296

Fix BitArray to clear dangling high bits in LeftShift and deserialization #123296

Copilot AI commented Jan 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

tannergooding left a comment

Uh oh!

stephentoub commented Feb 5, 2026

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

stephentoub commented Feb 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fix BitArray to clear dangling high bits in LeftShift and deserialization #123296

Fix BitArray to clear dangling high bits in LeftShift and deserialization #123296

Conversation

Copilot AI commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

tannergooding left a comment

Choose a reason for hiding this comment

Uh oh!

stephentoub commented Feb 5, 2026

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

stephentoub commented Feb 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jan 16, 2026 •

edited

Loading