CAMEL-23336: camel-mongodb - Expose Resume Token (Change Streams Consumer)

[Michwo]

Description

Adds option to configure the change streams resume token in change streams consumer endpoint (property resumeToken) and exposes the current resume token in exchange header (MongoDBConstants.RESUME_TOKEN).

[github-actions]

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

• First-time contributors require MANUAL approval for the GitHub Actions to run
• You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
• You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
• Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

[oscerd]

Please run a full mvn clean install -DskipTests from root folder.

[davsclaus]

you need to build the source code for the entire project as when you add new options there are code changed elsewhere

modified:   catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/mongodb.json
modified:   components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbConstants.java
modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/MongoDbEndpointBuilderFactory.java

[apupier]

I think it will be worthy to add also a test when the resume token is valid to check for the happy path

[Michwo]

Makes sense, I will add that

[Michwo]

you need to build the source code for the entire project as when you add new options there are code changed elsewhere

modified:   catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/mongodb.json
modified:   components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbConstants.java
modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/MongoDbEndpointBuilderFactory.java

Hi, thanks for the info.

I think my problem is that I've done the build on Windows and it messed up all the line endings in generated source (over 3k files changed). I've commited the changes inside components/camel-mongodb but that seems not enough.

I will try to run the build on linux next week.

[Croway]

Hi @Michwo I think that this feature can be implemented with Camel's resume strategy (ResumeAware Consumer). Using the resume strategy, the Camel route would look like:

from("mongodb:myDb?consumerType=changeStreams&database=mydb&collection=mycol")
      .resumable(myStategy)
      .process(this::process);

The resumeToken handling will be offloaded to the framework, and hopefully it will be easier to use.

The ResumeAware implementation can be done in another PR, no need to change the current implementation, but please open an issue about it.

fyi @orpiske