Upgrading 32 dependencies on main branch#4305
Conversation
…g-runner to v2.8.4
dropwizard-metrics 4.2.34+ requires Jetty 12.1.x, which is incompatible with Solr's embedded Jetty 12.0.x. Downgrade to 4.2.33 (last version supporting Jetty 12.0.x) and upgrade eclipse-jetty from 12.0.27 to 12.0.34. jersey-container-jetty-http:4.0.2 is compatible with Jetty 12.0.x. Regenerate lockfiles and update license checksums for new Jetty version.
BC 1.84 added several new Security.getProperty() calls for internal configuration properties (ec.max_f2m_field_size, asn1.max_cons_depth, etc.) that were blocked by the test security manager. Add all property permissions found in the BC 1.84 jar to avoid future one-by-one failures.
BC 1.84 added two new Security.getProperty() lookups that are denied by the test and server security managers. Discovered by running JWTAuthPluginIntegrationTest in a loop until all denials surfaced: - org.bouncycastle.ec.max_f2m_field_size (EC F2m field size limit) - org.bouncycastle.asn1.max_cons_depth (ASN.1 constructed depth limit) Added to both solr-tests.policy and server/etc/security.policy.
|
I ran the entire test suite locally and after updating security policy for BouncyCastle upgrade, it all pass. So I'll take this out of draft mode and ready for review, first by Copilot... |
|
I like this innovative approach to dealing with dependencies... It has felt to me like more and more time has been going into dependency management and that it was a Sisyphean task. On other projects that are smaller like Quepid I basically do a two day spasm of updating dependencies, manually test, and then call it good about three times a year ;-). I don't have the intertwined issues that we get in Solr! I wonder if dependency upgrades really need to be in our changelog? If our changelog is for end users, unless a dependency is fixing some big exciting thing, or it's a big new feature, I suspect it's just noise. |
You will see for most changelogs out there, dep upgrades will be a major part of the changelog, and I believe it makes sense. Perhaps more so for libraries such as Solrj. For Solr server it will alert users about whether they can expect a certain bug/cve to be fixed in a given version, which they otherwise would have to scan source code or git to find out. Speaking about SolrJ, since we don't publish it as a separate artifact (should we?), it would perhaps be useful to have a separate changelog section for solrj dependency upgrades. Eh, well folks can see dependencies in maven central too, I don't know.. |
This is the happy-path. Dealing with low hanging fruits. After merging this, there will still be plenty of dep upgrades that did not pass checks, that need manual tweaks, added LICENSE files etc. My thinking is that volunteer can work on each individual problematic solrbot PR to figure out each of them, with or without AI help, and once the PR branch is green, merge it, or gather a bunch of such into a common branch like her. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 300 out of 346 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…v2.42.34 AWS SDK v2.42.34 creates a background thread named response-input-stream-timeout-scheduler whenever getObject() is called without an explicit ResponseTransformer timeout parameter. This thread is not cleaned up promptly, causing ThreadLeakError in tests. Fix by passing ResponseTransformer.toInputStream(Duration.ZERO) to all getObject() calls in S3StorageClient.pullStream() and the S3 test classes, disabling the timeout mechanism that spawns the scheduler thread (see aws/aws-sdk-java-v2#6567). Also remove deprecated @SuppressWarnings("removal") and add .silent() to S3MockRule builders to suppress Spring Boot startup noise in test output.
…v2.42.34 AWS SDK v2.42.34 creates a background thread named response-input-stream-timeout-scheduler whenever getObject() is called without an explicit ResponseTransformer timeout parameter. This thread is not cleaned up promptly, causing ThreadLeakError in tests. Fix by passing ResponseTransformer.toInputStream(Duration.ZERO) to all getObject() calls in S3StorageClient.pullStream() and the S3 test classes, disabling the timeout mechanism that spawns the scheduler thread (see aws/aws-sdk-java-v2#6567). Also remove deprecated @SuppressWarnings("removal") and add .silent() to S3MockRule builders to suppress Spring Boot startup noise in test output. � Conflicts: � solr/modules/s3-repository/src/java/org/apache/solr/s3/S3StorageClient.java
…v2.42.34 AWS SDK v2.42.34 creates a background thread named response-input-stream-timeout-scheduler whenever getObject() is called without an explicit ResponseTransformer timeout parameter. This thread is not cleaned up promptly, causing ThreadLeakError in tests. Fix by passing ResponseTransformer.toInputStream(Duration.ZERO) to all getObject() calls in S3StorageClient.pullStream() and the S3 test classes, disabling the timeout mechanism that spawns the scheduler thread (see aws/aws-sdk-java-v2#6567). Also remove deprecated @SuppressWarnings("removal") and add .silent() to S3MockRule builders to suppress Spring Boot startup noise in test output. � Conflicts: � solr/modules/s3-repository/src/java/org/apache/solr/s3/S3StorageClient.java
- JGit changelog: 7.5.0 -> 7.6.0.202603022253-r (actual resolved version) - Mockito changelog: 5.21.0 -> 5.23.0 (actual resolved version) - checker-qual changelog: 3.53.1 -> 3.54.0 (matches license sha1 file) - onnx version in libs.versions.toml: 1.23.0 -> 1.24.3 (matches license sha1 file) - dropwizard changelog: remove spurious PR#4093 link (manual upgrade, no solrbot PR)
|
@janhoy Was there a specific reason you updated kotlin-datetime to to A bit of the background: some classes were moved to the Kotlin standard library, which caused some conflicts in some libraries. Therefore they had to introduce a compatibility version for consumers that solves any issues of incompatible libraries. More infos can be found here. |
|
That’s an oversight. I thought look at it but forgot. Will change it to 0.7.1, thanks. |
Bulk dependency upgrade for main branch. Asked Claude to merge all "clean" solrbot PRs for main into this feature branch. There will be another batch of "clean" upgrades later.
This PR also "manually" upgrades Jetty to 12.0.34 (latest 12.0.x version as 12.1 is breaking chage), and dropwizard-metrics to 4.2.33 (since later will require Jetty 12.1). The solrbot PR branches merged into this are the following:
Resolves #3075, #3105, #3114, #3129, #3131, #3293, #3313, #3422, #3677, #3681, #3697, #3715, #3733, #3757, #3794, #3796, #3827, #3850, #3853, #3855, #3861, #3894, #3917, #3942, #4092, #4094, #4095, #4097, #4155, #4156 and #4158