Skip to main content

Security boundary

This rollout controls only whether internal users see navigation and global actions that their effective permissions do not allow. Direct-route guards, tRPC procedures, REST routes, service-layer tenant and entity checks, and external collaborator containment remain permission enforced in every stage.
Never use the visibility rollout state to authorize a read or write. A UI rollback intentionally restores legacy controls, which can still produce a server-side denial.
Case-scoped and external collaborators always receive enforced visibility. Neither the platform stage nor an organization rollback can restore internal navigation, tenant switchers, notifications, global actions, or search to those principals.

Controls

Set these values on every web node and restart or roll the nodes:
An unset stage defaults to enforce. An invalid explicit stage fails closed to enforce. Empty or malformed list entries are ignored. The controls contain organization IDs, so manage them as deployment configuration and do not expose them to browser telemetry.

Stages

  1. Broad enforcement (default): Permission-driven hiding applies to every internal organization except an explicit UI rollback organization.
  2. Canary enforcement: Set the stage to canary and list opted-in organization IDs. Permission-driven hiding applies only to those organizations. Other internal organizations remain in observation mode.
  3. Observe/audit: Internal users retain legacy visibility. Permission route containment remains active. A visit to a visible but unauthorized route emits a hidden_route_mismatch signal and renders Access Denied.
Promote the same configuration to all web nodes. Mixed stages make telemetry ambiguous and can cause navigation to change between requests.

Telemetry and privacy

The authenticated, CSRF-protected telemetry endpoint re-resolves the principal and rollout before recording a signal. It accepts only an event type and a capability from the static manifest. It discards spoofed allowed-capability or wrong-stage signals. Monitor these OpenTelemetry counters and corresponding structured warning logs:
  • permission_visibility.hidden_route_mismatch: an observation-stage control was visible but the permanent route guard denied it;
  • permission_visibility.authorization_denial: an enforced or external principal reached a route that its permissions deny.
Labels are limited to rollout stage, enforcement state, rollout reason, manifest capability, and internal/external principal class. User IDs, organization IDs, route paths, entity IDs, query strings, tokens, and content are not accepted or emitted.

Promotion gates

Before each stage transition:
  1. Run the authorization manifest, rollout, telemetry-route, external collaborator, and sensitive-entrypoint policy tests.
  2. Verify a no-permission internal user can see a legacy control in observe but receives Access Denied and a server-side denial from direct API calls.
  3. Verify a canary organization hides the same control while a non-canary organization does not.
  4. Verify external collaborators see only the case-scoped shell in every stage.
  5. Review mismatch and denial rates by capability. Investigate unexpected increases or a capability that has no expected authorized role.
  6. Keep a stage through one deployment cycle, one permission-cache TTL, and one fresh-login cycle before expanding.

UI-only rollback

For one organization, add its exact ID to PERMISSION_VISIBILITY_ROLLBACK_ORGANIZATION_IDS. For a platform rollback, set PERMISSION_VISIBILITY_ROLLOUT_STAGE=observe. Roll all web nodes and verify the resolved reason label is organization-rollback or platform-observe. Do not revert authorization middleware, permission requirements, tenant/entity scope checks, route guards, or external collaborator restrictions. Do not interpret restored controls as restored access. Remove the rollback exception after the denial or role-template issue is understood and corrected.

Operator follow-up

Operators must supply real canary organization IDs, configure collection and alerts for the two counters, choose acceptable mismatch/denial thresholds, record Security and Product approval for each promotion, and rehearse the UI-only rollback in staging. Preserve rollout logs through the post-release review window.