Skip to content

Tasks: Standardise testing for AI Tasks API testing #1362

@SasSwart

Description

@SasSwart

coderd/aitasks_test.go contains a few different testing strategies. Some of the tests are also overly specific and should be generalized and then moved to the appropriate abstraction layer.

This PR standardises our testing strategy for coderd/aitasks_test.go.

Feedback to address:

coder/coder#21889 (comment)

Most, perhaps all, task endpoints get a workspace at some point. We have endpoint specific testing to ensure that if a workspace cannot be gotten, we fail correctly. Feedback from reviewers is that we'd like to enforce this failure mode for all task endpoints instead of testing it per endpoint. This is to avoid the risk of inconsistent behaviour between different endpoints.

Furthermore, Task Pause and Resume endpoints use a wrapper that induces the failure by overriding the method that we need to fail. Elsewhere in the tests, dbgen and dbfake are used to induce failures using real state in a database.

The benefits to this are:

  • we know which states in the database induce failures by enumerating them
  • we avoid the burden of testing failures that are impossible right now

The drawbacks are:

  • tests are more tightly coupled to the database layer, which introduces friction to future changes in this space
  • tests assume the current state of the schema. As the codebase evolves, new failure modes will become possible that these tests will not account for.

Action items

  • Write a test to enumerate all task endpoints and ensure they fail correctly if a workspace cannot be retrieved.
  • Remove the aiTaskStoreWrapper and use dbgen/dbfake instead

coder/coder#21889 (comment)

The original issue stated requirements related to authorization. These requirements were explicitly tested at the endpoint level, but those tests are deemed too specific and too granular. Per reviewer's request, these tests should be generalised and rather done at the rbac/dbauthz layer. At the endpoint level, it is sufficient to test using a second member user and ensure that the response is correct.

Action items

  • Ensure that all task, workspace and workspace build resources used by the AI tasks feature have sufficient RBAC enforced and tested at the dbauthz layer.
  • Simplify authz tests at the endpoint layer to only check the response type using a second member user.

coder/coder#21889 (comment)

Similar to the feedback above, reviewers request that we generalize testing for when resources cannot be found across all AI Task endpoints and ensure that the 404 response is correct.

coder/coder#21948 (comment)

Define variables closer to first use.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions