Skip to main content
This runbook governs rollout of the authentication, session, checklist, alert, workflow, Defender, storage, and authorization changes delivered by the enterprise remediation program. Security hotfixes must remain separate from schema-heavy capability releases.

​
Release trains

​
Security hotfix

Ship the bootstrap and Pub/Sub authentication fixes first. Release gates:
  • unauthenticated bootstrap and alert-ingestion requests are rejected;
  • Google OIDC signature, issuer, audience, expiry, and service-account mapping tests pass;
  • only one bootstrap activation can win under concurrent requests;
  • security audit evidence has been reviewed and unauthorized accounts, sessions, and credentials have been revoked or rotated;
  • deployment configuration contains an explicit Pub/Sub audience and identity mapping.
Rollback is not permitted to a build containing either authentication bypass. If the hotfix cannot be promoted, disable the affected endpoints at the ingress until a corrected build is available.

​
Correctness release

Ship session-policy enforcement, time-entry authorization, case-list menu behavior, and checklist reconciliation next. Pilot one organization before wider rollout. Preview checklist reconciliation before applying it and retain policy snapshots and completed-item history. Rollback:
  1. Pause new reconciliation jobs.
  2. Allow in-flight jobs to finish or reach an idempotent checkpoint.
  3. Disable automatic reconciliation while retaining the manual preview path.
  4. Roll back application code. Do not delete checklist history or downgrade a database schema until the down-migration has been reviewed on a production snapshot.

​
Capability release

Enable team summaries, workflow tag actions, alert unlinking, and Defender enrichment independently. Start with one organization and one workflow or Defender connection. Alert unlink uses detach-only semantics. Copied case evidence, observables, techniques, time records, and promotion provenance remain on the case.

​
Authorization acceptance matrix

Validate each changed API and UI path with:
  • unauthenticated user;
  • suspended or expired user;
  • SOC analyst tiers 1, 2, and 3;
  • SOC lead and manager;
  • organization administrator;
  • platform super administrator;
  • user with an active case access grant;
  • user with an expired or revoked case access grant;
  • case-scoped external collaborator;
  • organization-scoped API key and service principal.
Tests must cover the expected successful operation and direct-object attempts against another organization, team, case, alert, task, attachment, and search result. Privilege reductions and session revocations must take effect without waiting for the browser token to expire.

​
Required monitoring

Create alerts and dashboards for:
  • rejected bootstrap and Pub/Sub authentication requests;
  • OIDC failures grouped by issuer, audience, and mapped identity without recording tokens;
  • session idle and absolute expirations, forced revocations, and concurrent session denials;
  • time-entry authorization failures by permission and role;
  • checklist preview, reconciliation lag, retired-item count, retries, and terminal failures;
  • workflow tag retries and dead-lettered executions;
  • alert unlink attempts, denials, and successes;
  • Defender throttling, consent failures, partial enrichment, malware rejection, and artifact deduplication;
  • external collaborator grant creation, expiry, revocation, and access denials;
  • custom-role changes, assignment impact, and authorization shadow mismatches.
Do not place credentials, bearer tokens, email content, attachment content, or raw provider payloads in metrics or logs.

​
Pilot exit criteria

A pilot can expand only when:
  • no cross-tenant or cross-case access is observed;
  • error and denial rates are understood and within the agreed service objective;
  • reconciliation has no unexplained active-item drift;
  • workflow and Defender retries are idempotent;
  • all privileged operations have complete chained audit records;
  • backup restore and application rollback have been rehearsed;
  • product, security, operations, and customer support owners sign off.

​
Post-release review

Within one business day of each security or authorization release, review audit events, denial anomalies, queue failures, and customer reports. Keep the pilot feature controls available until the observation window closes. Record any exception with an owner, expiry date, compensating control, and remediation deadline.