Security boundary
This rollout controls only whether internal users see navigation and global
actions that their effective permissions do not allow. Direct-route guards,
tRPC procedures, REST routes, service-layer tenant and entity checks, and
external collaborator containment remain permission enforced in every stage.
Never use the visibility rollout state to authorize a read or write. A UI
rollback intentionally restores legacy controls, which can still produce a
server-side denial.
Case-scoped and external collaborators always receive enforced visibility.
Neither the platform stage nor an organization rollback can restore internal
navigation, tenant switchers, notifications, global actions, or search to those
principals.
Controls
Set these values on every web node and restart or roll the nodes:
An unset stage defaults to enforce. An invalid explicit stage fails closed to
enforce. Empty or malformed list entries are ignored. The controls contain
organization IDs, so manage them as deployment configuration and do not expose
them to browser telemetry.
Stages
- Broad enforcement (default): Permission-driven hiding applies to every
internal organization except an explicit UI rollback organization.
- Canary enforcement: Set the stage to
canary and list opted-in
organization IDs. Permission-driven hiding applies only to those
organizations. Other internal organizations remain in observation mode.
- Observe/audit: Internal users retain legacy visibility. Permission route
containment remains active. A visit to a visible but unauthorized route
emits a
hidden_route_mismatch signal and renders Access Denied.
Promote the same configuration to all web nodes. Mixed stages make telemetry
ambiguous and can cause navigation to change between requests.
Telemetry and privacy
The authenticated, CSRF-protected telemetry endpoint re-resolves the principal
and rollout before recording a signal. It accepts only an event type and a
capability from the static manifest. It discards spoofed allowed-capability or
wrong-stage signals.
Monitor these OpenTelemetry counters and corresponding structured warning logs:
permission_visibility.hidden_route_mismatch: an observation-stage control
was visible but the permanent route guard denied it;
permission_visibility.authorization_denial: an enforced or external
principal reached a route that its permissions deny.
Labels are limited to rollout stage, enforcement state, rollout reason,
manifest capability, and internal/external principal class. User IDs,
organization IDs, route paths, entity IDs, query strings, tokens, and content
are not accepted or emitted.
Promotion gates
Before each stage transition:
- Run the authorization manifest, rollout, telemetry-route, external
collaborator, and sensitive-entrypoint policy tests.
- Verify a no-permission internal user can see a legacy control in
observe
but receives Access Denied and a server-side denial from direct API calls.
- Verify a canary organization hides the same control while a non-canary
organization does not.
- Verify external collaborators see only the case-scoped shell in every stage.
- Review mismatch and denial rates by capability. Investigate unexpected
increases or a capability that has no expected authorized role.
- Keep a stage through one deployment cycle, one permission-cache TTL, and one
fresh-login cycle before expanding.
UI-only rollback
For one organization, add its exact ID to
PERMISSION_VISIBILITY_ROLLBACK_ORGANIZATION_IDS. For a platform rollback, set
PERMISSION_VISIBILITY_ROLLOUT_STAGE=observe. Roll all web nodes and verify the
resolved reason label is organization-rollback or platform-observe.
Do not revert authorization middleware, permission requirements, tenant/entity
scope checks, route guards, or external collaborator restrictions. Do not
interpret restored controls as restored access. Remove the rollback exception
after the denial or role-template issue is understood and corrected.
Operator follow-up
Operators must supply real canary organization IDs, configure collection and
alerts for the two counters, choose acceptable mismatch/denial thresholds,
record Security and Product approval for each promotion, and rehearse the
UI-only rollback in staging. Preserve rollout logs through the post-release
review window.