Skip to content

refactor: make random-order CLI tests deterministic#9998

Merged
michalsn merged 3 commits intocodeigniter4:developfrom
michalsn:refactor/cli-tests
Feb 24, 2026
Merged

refactor: make random-order CLI tests deterministic#9998
michalsn merged 3 commits intocodeigniter4:developfrom
michalsn:refactor/cli-tests

Conversation

@michalsn
Copy link
Member

Description
This PR stabilizes CLI tests under random execution order and fixes some static analysis types.

Ref: #9968

Checklist:

  • Securely signed commits
  • Component(s) with PHPDoc blocks, only if necessary or adds value (without duplication)
  • Unit testing, with >80% coverage
  • User guide updated
  • Conforms to style guide

@michalsn michalsn added the refactor Pull requests that refactor code label Feb 24, 2026
@michalsn michalsn merged commit 0b1490d into codeigniter4:develop Feb 24, 2026
49 checks passed
@michalsn
Copy link
Member Author

Thanks!

@paulbalandan
Copy link
Member

I am creating a test that verifies the components are truly running now in random, and this CLI test randomly fails:

PHPUnit 11.5.55 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.3 with Xdebug 3.5.1
Configuration: /Users/paul/Workspace/CodeIgniter4/phpunit.xml.dist
Random Seed:   1772303183

F.stty: stdin isn't a terminal
...........stty: stdin isn't a terminal
........stty: stdin isn't a terminal
............stty: stdin isn't a terminal
.....stty: stdin isn't a terminal
stty: stdin isn't a terminal
..stty: stdin isn't a terminal
..stty: stdin isn't a terminal
..stty: stdin isn't a terminal
..................stty: stdin isn't a terminal
... 65 / 72 ( 90%)
.stty: stdin isn't a terminal
.stty: stdin isn't a terminal
.....                                                           72 / 72 (100%)

Time: 00:03.497, Memory: 26.00 MB

There was 1 failure:

1) CodeIgniter\CLI\CLITest::testWriteForegroundWithColorAfter
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'�[0;31mred �[0m�[0;32mgreen�[0m
+'
+�[0;31mred �[0m�[0;32mgreen�[0m
 '

/Users/paul/Workspace/CodeIgniter4/tests/system/CLI/CLITest.php:367

FAILURES!
Tests: 72, Assertions: 159, Failures: 1.

Generating code coverage report in Clover XML format ... done [00:00.521]

Generating code coverage report in HTML format ... done [00:03.959]

@michalsn
Copy link
Member Author

michalsn commented Mar 1, 2026

I will try to look into this - thanks!

@michalsn
Copy link
Member Author

michalsn commented Mar 1, 2026

I have sent a PR with a fix: #10014

@michalsn michalsn deleted the refactor/cli-tests branch March 2, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

refactor Pull requests that refactor code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants