HBASE-29838 Run Hadoop Check as a GitHub Action #7651

ndimiduk · 2026-01-20T07:05:05Z

No description provided.

ndimiduk · 2026-01-20T14:09:02Z

Looks like we occupied the runner for 6h and then it was aborted.

Apache9 · 2026-01-20T16:27:28Z

https://infra.apache.org/github-actions-policy.html

The policy here does not say about the 6 hours timeout...

We can ask infra about the rules and the size of the github runners, our jenkins runners finished in "343m 23s", which was very close to 6 hours, so if the machine of the github runner is weaker, the build will be very easy to cost more than 6 hours...

ndimiduk · 2026-01-20T16:35:32Z

My action itself has timeout-minutes: 600 . i'll try bumping this up.

ndimiduk · 2026-01-20T16:38:31Z

or not. 6h is GH's hard limit, https://docs.github.com/en/actions/reference/limits

Apache9 · 2026-01-21T01:52:22Z

Then maybe we should try self hosted github runners? For self hosted runners the execution time limit is 5 days...

ndimiduk · 2026-01-23T07:07:12Z

Then maybe we should try self hosted github runners? For self hosted runners the execution time limit is 5 days...

Yes we should bring this back to our CI discussions with Infra. Maybe we can borrow from the pool of new Jenkins workers while we continue to build this out.

Yetus is supposed to provide smart, selective detection of module changes when it decided which tests to run. I think the new .github directory broke that for this run, so I've pushed a change to exclude it, maybe that will help.

I'm also going to see if I can manually parallelize the unit test runs -- maybe break out three separate checks for the three main unit test groups or something like that.

Apache9 · 2026-01-23T09:05:27Z

Then maybe we should try self hosted github runners? For self hosted runners the execution time limit is 5 days...

Yes we should bring this back to our CI discussions with Infra. Maybe we can borrow from the pool of new Jenkins workers while we continue to build this out.

Yetus is supposed to provide smart, selective detection of module changes when it decided which tests to run. I think the new .github directory broke that for this run, so I've pushed a change to exclude it, maybe that will help.

I'm also going to see if I can manually parallelize the unit test runs -- maybe break out three separate checks for the three main unit test groups or something like that.

Maybe we could split the UTs run as a seperated github check? Or even more, we could split the UTs run as two seperated check, one has -PrunDevTests(for small/medium tests) and one has -PrunLargeTests(for large tests).

ndimiduk · 2026-01-23T12:37:28Z

Maybe we could split the UTs run as a seperated github check? Or even more, we could split the UTs run as two seperated check, one has -PrunDevTests(for small/medium tests) and one has -PrunLargeTests(for large tests).

Yep, that's exactly my thinking as well. Landing these other cleanup issues and I'll be back.

ndimiduk · 2026-01-23T13:20:52Z

Okay this is better. Module selection chose only hbase-examples for running the unit tests.

-1 overall
  
  | Vote |      Subsystem |  Runtime   | Comment
  ============================================================================
  |   0  |        reexec  |   1m 13s   | Docker mode activated. 
  |  -0  |         yetus  |   0m  4s   | Unprocessed flag(s): --brief-report-file 
  |      |                |            | --spotbugs-strict-precheck
  |      |                |            | --author-ignore-list
  |      |                |            | --blanks-eol-ignore-file
  |      |                |            | --blanks-tabs-ignore-file
  |      |                |            | --quick-hadoopcheck
  +---------------------------------------------------------------------------
  |      |                |            | Prechecks 
  +---------------------------------------------------------------------------
  +---------------------------------------------------------------------------
  |      |                |            | master Compile Tests 
  +---------------------------------------------------------------------------
  |  +1  |    mvninstall  |   3m 25s   | master passed 
  |  +1  |       compile  |   0m 19s   | master passed 
  |  +1  |       javadoc  |   0m 17s   | master passed 
  |  +1  |    shadedjars  |   4m 47s   | branch has no errors when building our 
  |      |                |            | shaded downstream artifacts.
  +---------------------------------------------------------------------------
  |      |                |            | Patch Compile Tests 
  +---------------------------------------------------------------------------
  |  +1  |    mvninstall  |   3m  7s   | the patch passed 
  |  +1  |       compile  |   0m 18s   | the patch passed 
  |  +1  |         javac  |   0m 18s   | the patch passed 
  |  +1  |       javadoc  |   0m 15s   | the patch passed 
  |  +1  |    shadedjars  |   4m 51s   | patch has no errors when building our 
  |      |                |            | shaded downstream artifacts.
  +---------------------------------------------------------------------------
  |      |                |            | Other Tests 
  +---------------------------------------------------------------------------
  |  -1  |          unit  |   0m 24s   | hbase-examples in the patch failed. 
  |      |                |  19m 12s   |
  
  
  || Subsystem || Report/Notes ||
  ============================================================================
  |  unit  | /home/runner/work/hbase/hbase/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-examples.txt |
  | Docker | ClientAPI=1.48 ServerAPI=1.48 base: /home/runner/work/hbase/hbase/yetus-jdk17-hadoop3-check/output/Dockerfile |
  | GITHUB PR | https://github.com/apache/hbase/pull/7651 |
  | Optional Tests | javac javadoc unit compile shadedjars |
  | uname | Linux 9b6b1363e88a 6.11.0-1018-azure #18~24.04.1-Ubuntu SMP Sat Jun 28 04:46:03 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
  | Build tool | maven |
  | Personality | dev-support/hbase-personality.sh |
  | git revision | master / adfd0560c941109283928946bd671a5836604e46 |
  | Default Java | Eclipse Adoptium-17.0.11+9 |
  | Max. process+thread count | 72 (vs. ulimit of 30000) |
  | modules | C: hbase-examples U: hbase-examples |
  | versions | git=2.34.1 maven=3.9.8 |
  | Powered by | Apache Yetus 0.15.0 https://yetus.apache.org/ |

ndimiduk · 2026-01-23T15:51:15Z

Looking over out last successful nightly on master, the Large tests on hbase-server still took 7h. I'm looking for other ways to partition this up.

a stop-gap until we have the wave generation action deployed

… api

… work

Apache-HBase · 2026-02-01T17:57:50Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/12/console in case of problems.

Apache-HBase · 2026-02-01T17:58:16Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/12/console in case of problems.

Apache-HBase · 2026-02-01T18:43:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	3m 14s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+0 🆗	yamllint	0m 1s		yamllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ master Compile Tests _
+1 💚	mvninstall	2m 58s		master passed
+1 💚	compile	8m 26s		master passed
+1 💚	spotless	0m 42s		branch has no errors when running spotless:check.
-0 ⚠️	patch	0m 50s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 56s		the patch passed
+1 💚	compile	8m 21s		the patch passed
+1 💚	javac	8m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	pylint	0m 3s	/results-pylint.txt	The patch generated 52 new + 0 unchanged - 0 fixed = 52 total (was 0)
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xmllint	0m 0s		No new issues.
+1 💚	hadoopcheck	11m 18s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
-1 ❌	spotless	0m 8s	/patch-spotless.txt	patch has 55 errors when running spotless:check, run spotless:apply to fix.
			_ Other Tests _
-1 ❌	asflicense	0m 11s	/results-asflicense.txt	The patch generated 5 ASF License warnings.
		45m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/12/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7651
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs spotless pylint markdownlint yamllint javac xmllint hadoopcheck compile
uname	Linux 1fd2af9c33b4 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1822dd9`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	188 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/12/console
versions	git=2.34.1 maven=3.9.8 pylint=2.15.5 shellcheck=0.8.0 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2026-02-01T22:08:27Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 13s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	2m 22s		master passed
+1 💚	compile	1m 43s		master passed
+1 💚	javadoc	1m 32s		master passed
+1 💚	shadedjars	4m 28s		branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	patch	4m 35s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 22s		the patch passed
+1 💚	compile	1m 42s		the patch passed
+1 💚	javac	1m 42s		the patch passed
+1 💚	javadoc	1m 31s		the patch passed
-1 ❌	shadedjars	0m 6s	/patch-shadedjars.txt	patch has 7 errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	230m 33s	/patch-unit-root.txt	root in the patch failed.
		251m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.53 ServerAPI=1.53 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/12/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7651
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 75b381895cd7 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1822dd9`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/12/testReport/
Max. process+thread count	5835 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/12/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

ndimiduk · 2026-02-02T09:18:52Z

The check details pages should be nicer with apache/yetus#362