WIP N+1 detector gem Prosopite (replacing gem Bullet)

Gemfile

@@ -61,9 +61,9 @@ gem "tzinfo-data", platforms: %i[mingw mswin x64_mingw jruby] # Windows does not
 gem "view_component" # View components for reusability
 gem "wicked" # Multi-step form wizard for Rails
 
-group :development, :test do
+group :development, :test, :production do
   gem "brakeman" # Security inspection
-  gem "bullet" # Detect and fix N+1 queries
+  gem "prosopite" # N+1 query detection via SQL pattern analysis
   gem "byebug", platforms: %i[mri mingw x64_mingw] # Debugger console
   gem "dotenv-rails" # Environment variable management
   gem "erb_lint", require: false # ERB linter

Gemfile.lock

@@ -116,9 +116,6 @@ GEM
     bugsnag (6.28.0)
       concurrent-ruby (~> 1.0)
     builder (3.3.0)
-    bullet (8.1.0)
-      activesupport (>= 3.0.0)
-      uniform_notifier (~> 1.11)
     bundler-audit (0.9.3)
       bundler (>= 1.2.0)
       thor (~> 1.0)
@@ -424,6 +421,7 @@ GEM
       actionpack (>= 7.1)
     prettyprint (0.2.0)
     prism (1.9.0)
+    prosopite (2.1.2)
     pry (0.16.0)
       coderay (~> 1.1)
       method_source (~> 1.0)
@@ -668,7 +666,6 @@ GEM
     unicode-display_width (3.2.0)
       unicode-emoji (~> 4.1)
     unicode-emoji (4.2.0)
-    uniform_notifier (1.18.0)
     useragent (0.16.11)
     view_component (4.2.0)
       actionview (>= 7.1.0)
@@ -713,7 +710,6 @@ DEPENDENCIES
   blueprinter
   brakeman
   bugsnag
-  bullet
   bundler-audit
   byebug
   capybara
@@ -757,6 +753,7 @@ DEPENDENCIES
   pg_query
   pghero
   pretender
+  prosopite
   pry
   pry-byebug
   puma (~> 7.0)

PROSOPITE_TODO.md

@@ -0,0 +1,145 @@
+# Prosopite N+1 Query Issues
+
+Issues detected by Prosopite during test suite run. Fix by adding eager loading (`includes`, `preload`) or restructuring queries.
+
+## High Priority (20+ occurrences)
+
+### app/decorators/casa_case_decorator.rb:34
+- **Method:** `map` in decorator
+- **Likely fix:** Add `includes` for associated records being accessed in iteration
+
+### app/models/user.rb:84
+- **Method:** `create_preference_set`
+- **Query:** `SELECT "preference_sets".* FROM "preference_sets" WHERE "preference_sets"."user_id" = $1`
+- **Likely fix:** Check if preference_set already loaded before querying
+
+### app/models/concerns/api.rb:11
+- **Method:** `initialize_api_credentials`
+- **Query:** `SELECT "api_credentials".* FROM "api_credentials" WHERE "api_credentials"."user_id" = $1`
+- **Likely fix:** Check if api_credentials already loaded before querying
+
+### app/lib/importers/file_importer.rb:50
+- **Method:** `create_user_record`
+- **Queries:** Multiple user lookups during import
+- **Likely fix:** Batch lookups or use `Prosopite.pause` for intentional import operations
+
+## Medium Priority (10-19 occurrences)
+
+### app/validators/user_validator.rb:56
+- **Method:** `validate_uniqueness`
+- **Query:** `SELECT "users".* FROM "users" WHERE "users"."type" = $1 AND "users"."email" = $2`
+- **Likely fix:** Consider if validation is necessary during bulk operations
+
+### app/lib/importers/supervisor_importer.rb:47
+- **Method:** `block in assign_volunteers`
+- **Query:** `SELECT "users".* FROM "users" INNER JOIN "supervisor_volunteers"...`
+- **Likely fix:** Preload volunteers before assignment loop
+
+### app/lib/importers/supervisor_importer.rb:51
+- **Method:** `block in assign_volunteers`
+- **Query:** `SELECT "users".* FROM "users" WHERE "users"."id" = $1`
+- **Likely fix:** Batch load users by ID before iteration
+
+### app/controllers/case_contacts/form_controller.rb:156
+- **Method:** `block in create_additional_case_contacts`
+- **Likely fix:** Eager load case contact associations
+
+## Lower Priority (5-9 occurrences)
+
+### app/models/court_date.rb:32
+- **Method:** `associated_reports`
+- **Likely fix:** Add `includes` for court report associations
+
+### app/lib/importers/supervisor_importer.rb:23
+- **Method:** `block in import_supervisors`
+- **Query:** `SELECT "users".* FROM "users" WHERE "users"."type" = $1 AND "users"."email" = $2`
+- **Likely fix:** Batch check existing supervisors before import loop
+
+## Lower Priority (2-4 occurrences)
+
+### app/lib/importers/file_importer.rb:57
+- **Method:** `email_addresses_to_users`
+- **Likely fix:** Batch load users by email
+
+### app/lib/importers/case_importer.rb:41
+- **Method:** `create_casa_case`
+- **Likely fix:** Preload or batch casa_case lookups
+
+### app/decorators/contact_type_decorator.rb:14
+- **Method:** `last_time_used_with_cases`
+- **Likely fix:** Eager load case associations
+
+### app/datatables/reimbursement_datatable.rb:25
+- **Method:** `block in data`
+- **Query:** `SELECT "addresses".* FROM "addresses" WHERE "addresses"."user_id" = $1`
+- **Likely fix:** Add `includes(:address)` to reimbursement query
+
+### app/services/volunteer_birthday_reminder_service.rb:7
+- **Method:** `block in send_reminders`
+- **Likely fix:** Eager load volunteer associations
+
+### app/models/contact_topic.rb:27
+- **Method:** `block in generate_for_org!`
+- **Likely fix:** Batch operations or use `Prosopite.pause` for setup
+
+### app/models/casa_org.rb:82
+- **Method:** `user_count`
+- **Likely fix:** Use counter cache or single count query
+
+### app/models/casa_org.rb:62
+- **Method:** `case_contacts_count`
+- **Likely fix:** Use counter cache or single count query
+
+### app/lib/importers/volunteer_importer.rb:23
+- **Method:** `block in import_volunteers`
+- **Likely fix:** Batch check existing volunteers before import loop
+
+### app/lib/importers/case_importer.rb:20
+- **Method:** `block in import_cases`
+- **Likely fix:** Batch check existing cases before import loop
+
+### app/controllers/case_contacts/form_controller.rb:26
+- **Method:** `block (2 levels) in update`
+- **Likely fix:** Eager load contact associations
+
+## Single Occurrence
+
+### app/services/missing_data_export_csv_service.rb:40
+- **Method:** `full_data`
+
+### app/policies/contact_topic_answer_policy.rb:18
+- **Method:** `create?`
+
+### app/models/casa_case.rb:152
+- **Method:** `next_court_date`
+
+### app/models/all_casa_admins/casa_org_metrics.rb:16
+- **Method:** `map`
+
+### config/initializers/sent_email_event.rb:7
+- **Method:** `block in <top (required)>`
+- **Query:** `SELECT "casa_orgs".* FROM "casa_orgs" WHERE "casa_orgs"."id" = $1`
+- **Note:** Initializer callback - consider caching org lookup
+
+## Notes
+
+- **Importers:** Many N+1s occur in import code. Consider wrapping entire import operations in `Prosopite.pause { }` if the N+1 pattern is intentional for per-record processing, or batch-load records before iteration.
+
+- **Decorators:** Add `includes` at the controller/query level before passing to decorators.
+
+- **Callbacks:** User model callbacks (`create_preference_set`, `initialize_api_credentials`) fire on each create. Consider if these can be optimized or if the N+1 is acceptable for single-record operations.
+
+## How to Fix
+
+```ruby
+# Before (N+1)
+users.each { |u| u.orders.count }
+
+# After (eager loading)
+users.includes(:orders).each { |u| u.orders.count }
+
+# For intentional batch operations
+Prosopite.pause do
+  records.each { |r| process_individually(r) }
+end
+```

config/environments/development.rb

@@ -68,12 +68,9 @@
   # Raises error for missing translations.
   # config.i18n.raise_on_missing_translations = true
 
-  config.after_initialize do
-    Bullet.enable = true
-    Bullet.console = true
-    Bullet.rails_logger = true
-    Bullet.bullet_logger = true
-  end
+  # Prosopite N+1 query detection
+  config.prosopite_enabled = true
+  config.prosopite_min_n_queries = 5  # More lenient for development
 
   # Annotate rendered view with file names.
   config.action_view.annotate_rendered_view_with_filenames = true

config/environments/test.rb

@@ -63,14 +63,6 @@
   # Raises error for missing translations.
   config.i18n.raise_on_missing_translations = true
 
-  config.after_initialize do
-    Bullet.enable = true
-    Bullet.console = true
-    Bullet.bullet_logger = true
-    Bullet.rails_logger = true
-    # Bullet.raise = true # TODO https://github.com/rubyforgood/casa/issues/2441
-  end
-
   # Annotate rendered view with file names.
   # config.action_view.annotate_rendered_view_with_filenames = true
 

config/initializers/prosopite.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+# Rack middleware for development only — in test, scanning is handled by RSpec hooks
+if Rails.env.development? &&
+    Rails.configuration.respond_to?(:prosopite_enabled) &&
+    Rails.configuration.prosopite_enabled
+  require "prosopite/middleware/rack"
+  Rails.configuration.middleware.use(Prosopite::Middleware::Rack)
+end
+
+# Development configuration — test config lives in spec/support/prosopite.rb
+Rails.application.config.after_initialize do
+  next unless Rails.env.development?
+
+  Prosopite.enabled = Rails.configuration.respond_to?(:prosopite_enabled) &&
+    Rails.configuration.prosopite_enabled
+
+  Prosopite.min_n_queries = Rails.configuration.respond_to?(:prosopite_min_n_queries) ?
+                            Rails.configuration.prosopite_min_n_queries : 2
+
+  Prosopite.rails_logger = true
+  Prosopite.prosopite_logger = true
+end

spec/.prosopite_ignore

@@ -0,0 +1,20 @@
+# Directories excluded from Prosopite N+1 raise (will still log).
+# Remove paths as you fix the underlying N+1s.
+#
+# See PROSOPITE_TODO.md for the full list of known issues.
+spec/models
+spec/services
+spec/lib
+spec/system
+spec/requests
+spec/controllers
+spec/views
+spec/decorators
+spec/policies
+spec/datatables
+spec/helpers
+spec/seeds
+spec/mailers
+spec/presenters
+spec/config
+spec/notifications

spec/support/prosopite.rb

@@ -0,0 +1,73 @@
+# frozen_string_literal: true
+
+return unless defined?(Prosopite)
+
+# Test configuration — this file owns all Prosopite settings for the test env
+Prosopite.enabled = true
+Prosopite.raise = true
+Prosopite.rails_logger = true
+Prosopite.prosopite_logger = true
+
+# Allowlist for known acceptable N+1 patterns (e.g., test matchers)
+Prosopite.allow_stack_paths = [
+  "shoulda/matchers/active_record/validate_uniqueness_of_matcher.rb",
+  "shoulda/matchers/active_model/validate_presence_of_matcher.rb",
+  "shoulda/matchers/active_model/validate_inclusion_of_matcher.rb",
+  "shoulda/matchers/active_model/allow_value_matcher.rb"
+]
+
+# Load ignore list from file for gradual rollout — directories listed in
+# .prosopite_ignore are scanned but won't raise, only log.
+PROSOPITE_IGNORE = if File.exist?("spec/.prosopite_ignore")
+  File.read("spec/.prosopite_ignore")
+    .lines
+    .map(&:chomp)
+    .reject { |line| line.empty? || line.start_with?("#") }
+else
+  []
+end
+
+RSpec.configure do |config|
+  # Pause Prosopite during factory creation to prevent false positives
+  # from factory callbacks and associations
+  config.before(:suite) do
+    if defined?(FactoryBot)
+      FactoryBot::SyntaxRunner.class_eval do
+        alias_method :original_create, :create
+
+        def create(*args, **kwargs, &block)
+          if defined?(Prosopite) && Prosopite.enabled?
+            Prosopite.pause { original_create(*args, **kwargs, &block) }
+          else
+            original_create(*args, **kwargs, &block)
+          end
+        end
+      end
+    end
+  end
+
+  config.around do |example|
+    if use_prosopite?(example)
+      Prosopite.scan { example.run }
+    else
+      original_enabled = Prosopite.enabled?
+      Prosopite.enabled = false
+      begin
+        example.run
+      ensure
+        Prosopite.enabled = original_enabled
+      end
+    end
+  end
+end
+
+def use_prosopite?(example)
+  # Explicit metadata takes precedence
+  return false if example.metadata[:disable_prosopite]
+  return true if example.metadata[:enable_prosopite]
+
+  # Check against ignore list
+  PROSOPITE_IGNORE.none? do |path|
+    File.fnmatch?("./#{path}/*", example.metadata[:rerun_file_path].to_s)
+  end
+end