Platform Demo Scenarios and Regression Test Paths¶

This document defines the end-to-end demo scenario and the regression test path for each platform phase. Each scenario is executable against a real environment; each regression test is the automated check that proves the phase works as new code ships.

Phases are cumulative. Regression tests from earlier phases must continue passing as later phases are delivered.

Phase 0 — Foundation Hygiene¶

Goal: Prove that the local-dev stack starts cleanly, the core data models enforce tenancy and soft-delete, and every mutation returns the standard envelope.

Demo walkthrough¶

Prerequisites: Docker Desktop running, Python 3.12, make.

# Clone and enter the control plane
git clone git@github.com:calliopeai/astrolift-app.git
cd astrolift-app

# Start the full local stack (Postgres + Redis + Django + Temporal dev server)
make dev
# Expected: "Astrolift ready on http://localhost:8000" in the last few lines.
# Temporal UI is at http://localhost:8088.

Create a tenant and verify isolation.

Open a second terminal:

# Create the first org and its owner
make shell

Inside the Django shell:

from organizations.models import Organization
from accounts.models import User

org_a = Organization.objects.create(name="Acme Corp", slug="acme")
user_a = User.objects.create_user(email="alice@acme.com", organization=org_a)

org_b = Organization.objects.create(name="Widgets Inc", slug="widgets")
user_b = User.objects.create_user(email="bob@widgets.com", organization=org_b)

# Tenancy guardrail: querying org_a's apps as org_b must return empty
from apps.models import App
App.objects.create(name="acme-api", organization=org_a)

# This must raise or return nothing — not acme's app
result = App.objects.for_organization(org_b)
assert result.count() == 0, "Tenancy leak detected"
print("Tenancy isolation: OK")

Create an app and exercise soft-delete.

app = App.objects.create(name="demo-app", organization=org_a)
print(f"App created: {app.pk}, deleted_at={app.deleted_at}")  # None

app.delete()
print(f"After delete: deleted_at={app.deleted_at}")  # timestamp, not None

# Hard-delete path must be explicit
assert App.objects.filter(pk=app.pk).count() == 0   # default manager excludes soft-deleted
assert App.all_objects.filter(pk=app.pk).count() == 1  # unfiltered manager sees it
print("Soft-delete: OK")

Verify mutation envelope via GraphQL.

curl -s -X POST http://localhost:8000/graphql/ \
  -H "Content-Type: application/json" \
  -H "Authorization: Token $(make -s dev-token)" \
  -d '{
    "query": "mutation { createApp(input: {name: \"envelope-test\", organizationSlug: \"acme\"}) { ok errors { field message } data { id name } } }"
  }' | python3 -m json.tool

Expected response shape (values will differ):

{
  "data": {
    "createApp": {
      "ok": true,
      "errors": [],
      "data": {
        "id": "QXBwOjE=",
        "name": "envelope-test"
      }
    }
  }
}

Verify structured logs and audit trail.

# Tail Django log — every request line must be JSON with a trace_id field
make logs | python3 -c "
import sys, json
for line in sys.stdin:
    try:
        obj = json.loads(line)
        assert 'trace_id' in obj, f'Missing trace_id: {line}'
        print('OK:', obj.get('event', obj.get('message')))
    except json.JSONDecodeError:
        pass  # non-JSON lines (startup banners) are acceptable
" &

# Trigger a mutation to produce an audit entry
curl -s -X POST http://localhost:8000/graphql/ \
  -H "Content-Type: application/json" \
  -H "Authorization: Token $(make -s dev-token)" \
  -d '{"query":"mutation { deleteApp(id: \"QXBwOjE=\") { ok errors { message } } }"}'

# Check the audit log
make shell -c "
from audit.models import AuditEntry
entry = AuditEntry.objects.order_by('-created_at').first()
print('actor:', entry.actor_email)
print('action:', entry.action)
print('target:', entry.target_repr)
"

Success criteria:

make dev reaches ready state with no errors.
Tenancy assertion passes — cross-org query returns empty.
Soft-delete assertion passes — deleted_at set, default manager hides the row.
Every mutation response has { ok, errors, data? } at the top level.
Every log line produced by a request is valid JSON containing trace_id.
Every write operation produces an AuditEntry row.

Regression test path¶

make test PYTEST_ARGS="-m 'phase0' -x"

The phase0 marker covers:

tests/tenancy/test_orm_guardrails.py — cross-org query returns empty; queryset raises on missing org context
tests/audit/test_audit_entries.py — every mutation type writes an audit entry
tests/core/test_soft_delete.py — soft-delete on every registered model; default manager excludes; all_objects includes
tests/schema/test_mutation_envelope.py — every registered mutation resolver returns MutationResult
tests/logs/test_structured_logs.py — request middleware emits valid JSON with trace_id on every response

CI gate: this suite runs on every PR and must be green before merge.

Phase 1 — Platform Foundation¶

Goal: Prove that a real cluster can be registered, a cloud provider plugin loads, an identity provider issues tokens the platform accepts, secrets are read from a real backend, and an app reaches its ingress URL with a valid TLS cert.

Demo walkthrough¶

Prerequisites: A running Astrolift install (see Installing Astrolift). For a local demo, follow the kind dev profile in INSTALL-k8s-native.md. The astro CLI binary must be in PATH.

Step 1: Register a cluster.

# kind dev profile — create a local cluster
kind create cluster --name astrolift-demo

# Register it with the control plane
astro cluster register \
  --name demo-cluster \
  --plugin k8s_native \
  --kubeconfig ~/.kube/config \
  --context kind-astrolift-demo

# Verify
astro cluster list
# Expected output includes:
# NAME            PLUGIN       STATUS
# demo-cluster    k8s_native   healthy

For an AWS cluster, replace --plugin k8s_native with --plugin aws and provide the EKS cluster ARN via --cluster-arn.

Step 2: Verify the provider plugin loaded.

astro provider list
# Expected:
# PROVIDER     STATUS    DRIVERS
# k8s_native   loaded    cluster, registry, ingress, tls

Step 3: Configure an identity provider.

# Register a GitHub OIDC provider (replace with your org)
astro idp create \
  --name github-oidc \
  --type oidc \
  --issuer https://token.actions.githubusercontent.com \
  --client-id <your-github-app-client-id> \
  --client-secret <your-github-app-client-secret>

# Test that a token from that IdP is accepted
TOKEN=$(gh auth token)
curl -sI https://<base-domain>/api/auth/verify \
  -H "Authorization: Bearer $TOKEN" \
  | grep "HTTP/"
# Expected: HTTP/2 200

Step 4: Configure a secrets backend.

# For the kind dev profile, the built-in env-var backend is active by default.
# Confirm it's registered:
astro secrets backend list
# Expected:
# NAME       TYPE     STATUS
# local      env      active

# Write a secret and read it back
astro secrets set --app demo-app --key DATABASE_URL --value "postgres://..."
astro secrets list --app demo-app
# Expected: DATABASE_URL   [encrypted]   set 5s ago

For a production backend (Vault, AWS Secrets Manager), follow the operator runbook for that backend type.

Step 5: Create an app, deploy, and verify ingress + TLS.

# Create a minimal app manifest
mkdir demo-app && cd demo-app
cat > astrolift.toml <<'EOF'
[app]
name = "demo-app"
org  = "acme"

[build]
dockerfile = "Dockerfile"

[[workloads]]
name    = "web"
command = ["python", "-m", "http.server", "8080"]
port    = 8080

[ingress]
enabled = true
EOF

# Register and deploy
astro app register
astro app deploy --cluster demo-cluster

# Wait for healthy
astro app status --watch
# Expected final state:
# WORKLOAD   REPLICAS   STATUS
# web        1/1        running

# Verify HTTPS
curl -sI https://demo-app.acme.<base-domain>/
# Expected: HTTP/2 200
# cert is valid (curl does not warn)

Success criteria:

Cluster appears in astro cluster list with healthy status.
astro provider list shows the plugin with all expected drivers loaded.
An OIDC token from the configured IdP returns HTTP 200 from /api/auth/verify.
A secret written via astro secrets set is returned (masked) by astro secrets list.
The deployed app responds over HTTPS at its assigned subdomain with a valid cert.

Regression test path¶

make test PYTEST_ARGS="-m 'phase1' -x"

The phase1 marker covers:

tests/providers/test_plugin_loader.py — entry-point discovery loads the k8s_native driver; missing plugin raises PluginNotFound
tests/providers/test_cluster_driver.py — ClusterDriver.register() writes a Cluster row and the health-check probe returns healthy
tests/auth/test_idp_oidc.py — a JWT signed by the test OIDC issuer is accepted; a JWT from an unknown issuer is rejected
tests/secrets/test_secrets_backend.py — write + read round-trip on the env backend; absent key raises SecretNotFound
tests/ingress/test_ingress_driver.py — IngressDriver.provision() creates the expected Ingress object in the test cluster; TLS secret is present

CI gate: runs on every PR; integration subtests (real cluster) run on every push to main using the kind dev profile in CI.

Phase 2 — Onboarding and Deploy Loop¶

Goal: Prove the full webhook-driven path: a GitHub push triggers onboarding, builds the image, provisions a managed Postgres, injects secrets, deploys, assigns a subdomain, passes the healthcheck, and rollback returns to the prior revision.

Demo walkthrough¶

Prerequisites: Phase 1 complete. A GitHub repository with an astrolift.toml and a Dockerfile. Ngrok or a public URL pointing at your control plane for webhook delivery.

Step 1: Connect the GitHub repository.

astro app connect-repo \
  --app demo-app \
  --repo github.com/acme/demo-app \
  --branch main

# The CLI prints the webhook URL and secret to register in GitHub:
# Webhook URL: https://<base-domain>/webhooks/github/<install-id>
# Secret:      <hmac-secret>

In GitHub → Settings → Webhooks → Add webhook: - Payload URL: the URL printed above - Content type: application/json - Secret: the secret printed above - Events: select "Pushes"

Step 2: Declare a managed Postgres in the manifest.

# astrolift.toml — add the managed service block
[[services]]
name    = "db"
type    = "postgres"
version = "16"
plan    = "shared-1"

[[workloads]]
name    = "web"
command = ["gunicorn", "app:wsgi"]
port    = 8000
env_from_service = ["db"]   # injects DATABASE_URL automatically

Step 3: Push to trigger the full pipeline.

git add astrolift.toml
git commit -m "feat: add managed postgres"
git push origin main

Watch the deploy unfold:

astro app status --watch
# Timeline lines appear in order:
# [onboard]   cloning repo                  done
# [build]     building image                done  (sha256:abc123)
# [provision] provisioning postgres/db      done  (host: db-xxxx.internal)
# [secrets]   injecting DATABASE_URL        done
# [deploy]    rolling out web               done  (1/1 healthy)
# [dns]       assigning subdomain           done  (demo-app.acme.<base-domain>)
# [health]    GET /healthz -> 200           done
# STATUS: running

Step 4: Verify subdomain and healthcheck.

curl -s https://demo-app.acme.<base-domain>/healthz
# Expected: {"status": "ok"}

Step 5: Trigger a rollback.

# List revisions
astro app revisions --app demo-app
# REV   SHA        DEPLOYED AT           STATUS
# 2     sha256:… 2026-05-30T12:01:00Z  current
# 1     sha256:… 2026-05-30T11:45:00Z  superseded

astro app rollback --app demo-app --revision 1

astro app status --watch
# [rollback]  rolling out web@rev1      done
# STATUS: running (rev 1)

# Verify the previous image is live
curl -s https://demo-app.acme.<base-domain>/version
# Expected: {"revision": 1}

Success criteria:

A GitHub push triggers the control plane within 10 seconds of delivery (check webhook delivery log in GitHub).
Timeline shows onboard → build → provision → secrets → deploy → dns → health all done.
curl to the app's HTTPS subdomain returns HTTP 200 from the workload.
astro app rollback completes without error and the prior image is live.

Regression test path¶

make test PYTEST_ARGS="-m 'phase2' -x"

The phase2 marker covers:

tests/workflows/test_onboard_app_workflow.py — Temporal test env runs OnboardAppWorkflow end-to-end against a fake registry and cluster; asserts App.status == running
tests/workflows/test_deploy_app_workflow.py — DeployAppWorkflow progresses through all activities; final state is running
tests/workflows/test_rollback_workflow.py — rollback to a prior revision activates the earlier image and marks the current revision superseded
tests/webhooks/test_github_webhook.py — a POST to /webhooks/github/<id> with a valid HMAC signature enqueues a deploy signal; invalid HMAC returns 403
tests/managed_services/test_postgres_provision.py — ProvisionManagedServiceWorkflow creates a ManagedService row and injects DATABASE_URL into the app's environment
tests/ingress/test_subdomain_assignment.py — deploy workflow assigns a unique subdomain and the DNS record is created

CI gate: runs on every PR. Temporal workflow tests use the Temporal Go test server (not mocked). Database tests use real Postgres.

Phase 3 — Operations and Observability¶

Goal: Prove that an operator can observe a running app end-to-end in the UI (logs, metrics, traces), trigger an alert, receive a webhook, view the audit log, detect drift, and exec into a pod — all in one session.

Demo walkthrough¶

Prerequisites: Phase 2 complete. The demo app is deployed and receiving traffic.

Step 1: Open the single-pane-of-glass dashboard.

Navigate to https://<base-domain> and sign in. Select org "Acme Corp", then app "demo-app".

The overview panel shows: - Deployment timeline with current revision, deploy time, and deployer identity - Pod count and status (green = running) - Live CPU and memory sparklines

Step 2: Stream live logs.

Click "Logs" in the tab bar. Logs stream in real time. Use the filter bar:

level:error

Expected: only ERROR-level log lines appear. Clear the filter to see all levels.

From the CLI:

astro app logs --app demo-app --follow --filter "level=error"
# Lines stream as they arrive; Ctrl-C to stop

Step 3: View metrics and traces.

Click "Metrics". The dashboard renders: - Request rate (req/s) - P50 / P95 / P99 latency - Error rate (4xx, 5xx)

Click "Traces". The trace explorer shows recent spans. Click any trace to expand the waterfall.

Step 4: Trigger an alert and receive a webhook.

# Create an alert rule: error rate > 5% for 1 minute
astro alerts create \
  --app demo-app \
  --name "high-error-rate" \
  --condition "error_rate > 0.05" \
  --window 1m \
  --webhook https://webhook.site/<your-id>

# Inject errors to trip the alert (demo: call an endpoint that 500s)
for i in $(seq 1 50); do
  curl -s https://demo-app.acme.<base-domain>/error > /dev/null
done

# Within ~90 seconds the webhook fires
# Expected payload at webhook.site:
# {
#   "event": "alert.triggered",
#   "alert": "high-error-rate",
#   "app": "demo-app",
#   "org": "acme",
#   "value": 0.62,
#   "fired_at": "2026-05-30T..."
# }

Step 5: View the audit log.

In the dashboard, navigate to Org → Audit Log. Filter by "app = demo-app". Expected entries:

2026-05-30T12:01:00Z  alice@acme.com  app.deploy     demo-app   rev 2
2026-05-30T12:03:00Z  alice@acme.com  alert.create   demo-app   high-error-rate
2026-05-30T12:05:00Z  system          alert.trigger  demo-app   high-error-rate

Export to CSV: click "Export" → "CSV". Verify the file downloads with the same rows.

Step 6: Detect drift and pause deploys.

# Manually patch a pod annotation outside of Astrolift (simulates drift)
kubectl annotate pod -n acme-demo-app \
  $(kubectl get pod -n acme-demo-app -o name | head -1) \
  manual-patch=true

# Drift detection picks this up within its next scan cycle (default: 5 min)
astro app drift --app demo-app
# Expected:
# RESOURCE                      FIELD              EXPECTED   ACTUAL
# Pod/web-7f9d5b-xxxxx          annotations        {}         {manual-patch: true}
# Drift detected: 1 resource(s)

# Pause deploys for the environment
astro env pause --app demo-app --env production
astro deploy  # attempt a new deploy
# Expected error:
# Error: deploys are paused for production. Run `astro env resume` to re-enable.

astro env resume --app demo-app --env production

Step 7: Exec into a pod.

astro app exec --app demo-app --workload web -- /bin/sh
# Drops into an interactive shell inside the container.
# Session is logged: every command is recorded to the audit log.
# Exit with Ctrl-D.

# Verify the exec session appears in audit log
astro audit --app demo-app --action pod.exec --limit 5
# Expected:
# 2026-05-30T12:10:00Z  alice@acme.com  pod.exec  demo-app/web  duration=42s

Success criteria:

Log streaming in the dashboard shows lines within 3 seconds of emission; filter by level works.
Metrics panels render within 10 seconds of page load; P95 latency value is plausible.
Alert fires within 2 minutes of the error injection; the webhook payload arrives with HMAC signature.
Audit log shows every action taken during this demo session.
Drift detection reports the manually-patched annotation.
Deploy attempt to a paused environment is rejected with a clear error.
astro app exec opens an interactive shell; the session appears in the audit log.

Regression test path¶

make test PYTEST_ARGS="-m 'phase3' -x"

The phase3 marker covers:

tests/observability/test_log_stream.py — LogStreamDriver emits structured JSON lines; filter by level returns only matching lines; stream terminates cleanly on disconnect
tests/observability/test_metrics_driver.py — MetricsDriver.query_range() returns a valid Prometheus-compatible response for CPU/memory/request-rate queries
tests/alerts/test_alert_rules.py — creating an alert rule persists the rule; injecting a metric above threshold fires alert.triggered; the outbound webhook is dispatched with a valid HMAC signature
tests/audit/test_audit_export.py — exporting audit entries returns all rows in CSV with correct headers
tests/drift/test_drift_detection.py — DriftDetectionWorkflow compares desired state against a patched cluster state and emits a drift event
tests/exec/test_pod_exec.py — astro app exec creates an audited session; every command is appended to the session record; session end is recorded on disconnect

CI gate: runs on every PR. Log/metrics driver tests use a stub backend. Drift and exec tests use the kind cluster in CI.

Phase 4 — CLI and Polish¶

Goal: Prove the full astro CLI workflow on a freshly downloaded binary: init, deploy, logs, and rollback — across macOS, Linux, and Windows.

Demo walkthrough¶

Prerequisites: Nothing installed except the OS. No existing astro binary in PATH.

Step 1: Install the CLI.

# macOS / Linux — one-liner
curl -fsSL https://astrolift.dev/install.sh | sh
# Expected: "astro 0.x.y installed to /usr/local/bin"

# Verify
astro version
# Expected: astro version 0.x.y (darwin/arm64)

# Self-update check
astro update
# Expected: "already up to date" or "updated to 0.x.z"

On Windows (PowerShell):

# Via Scoop
scoop bucket add astrolift https://github.com/calliopeai/scoop-astrolift
scoop install astro

astro version
# astro version 0.x.y (windows/amd64)

Step 2: Authenticate.

astro auth login --url https://<base-domain>
# Opens browser; user completes OIDC login
# Expected: "Logged in as alice@acme.com"

astro auth whoami
# alice@acme.com  (org: acme)

Step 3: Init a new app from an existing directory.

mkdir my-python-api && cd my-python-api

# Add a minimal Python app
cat > main.py <<'EOF'
from http.server import HTTPServer, BaseHTTPRequestHandler
class H(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"hello")
HTTPServer(("0.0.0.0", 8080), H).serve_forever()
EOF

echo "requests" > requirements.txt

# Auto-discovery: detects requirements.txt → Python web app
astro app init
# Expected output:
# Detected ecosystem: python
# Generated astrolift.toml
# Generated Dockerfile

cat astrolift.toml
# [app]
# name    = "my-python-api"
# org     = "acme"
# ...

Step 4: Deploy.

astro app deploy
# [build]   building image …
# [deploy]  rolling out web (1/1) …
# [dns]     my-python-api.acme.<base-domain>
# Deploy complete. https://my-python-api.acme.<base-domain>

Step 5: Stream logs with filtering.

# Stream all logs, grep for access log lines
astro app logs --follow --grep "GET /"
# Lines matching "GET /" stream in real time; others are suppressed

# Filter by log level (structured logs only)
astro app logs --follow --filter "level=error"

# Pipe to jq for JSON logs
astro app logs | jq '.message'

Step 6: Rollback via CLI.

astro app revisions
# REV   SHA         DEPLOYED AT            STATUS
# 2     sha256:…   2026-05-30T14:00:00Z   current
# 1     sha256:…   2026-05-30T13:45:00Z   superseded

astro app rollback --revision 1
# [rollback]  rolling out web@rev1 …  done
# Rollback complete. Running revision 1.

Step 7: Permission diagnostic.

# Check what a deploy token can and cannot do
astro permissions check --token <deploy-token> --action app.deploy
# Expected:
# PERMISSION      RESULT
# app.deploy      ALLOWED  (via role: deployer)
# app.delete      DENIED   (not in role)
# secrets.reveal  DENIED   (not in role)

Success criteria:

curl | sh installer completes without errors and the binary is in PATH.
astro version reports the correct platform triple (darwin/arm64, linux/amd64, windows/amd64).
astro update runs without error.
astro app init correctly detects the Python ecosystem and emits a valid astrolift.toml and Dockerfile.
astro app deploy completes and the app is reachable at its URL.
astro app logs --filter and --grep both suppress non-matching lines.
astro app rollback succeeds and the app serves the prior revision.
astro permissions check accurately reflects the token's role bindings.

Regression test path¶

make test PYTEST_ARGS="-m 'phase4' -x"

The phase4 marker covers:

tests/cli/test_app_init.py — discovery on each ecosystem (Python via requirements.txt / Pipfile, Node via package.json, Go via go.mod, Dockerfile passthrough) generates a valid astrolift.toml
tests/cli/test_deploy_command.py — astro app deploy against the test control plane exits 0 and the app reaches running
tests/cli/test_logs_command.py — --filter suppresses non-matching lines; --grep passes through matching lines; --follow streams until SIGINT
tests/cli/test_rollback_command.py — rollback to a prior revision exits 0; subsequent astro app revisions shows the correct current rev
tests/cli/test_permissions_check.py — astro permissions check returns ALLOWED for permitted actions and DENIED for others; output is parseable (JSON mode via --output json)
tests/release/test_binary_matrix.py — CI build matrix produces signed binaries for darwin/arm64, darwin/amd64, linux/amd64, linux/arm64, windows/amd64; astro version on each returns expected output

CI gate: runs on every PR. Binary matrix test runs on every tag push.

Phase 5 — Advanced and Beyond Parity¶

Goal: Prove a PR-triggered preview environment with a custom domain, cost tracking, a vulnerability scan gate, and GitOps reconciliation.

Demo walkthrough¶

Prerequisites: Phase 2 complete (GitHub webhook connected). A custom domain you control (e.g. preview.acme.com). A container vulnerability scanner (Trivy) accessible from the build worker. GitOps mode enabled (ArgoCD or Flux).

Step 1: Enable preview environments in the manifest.

# astrolift.toml
[previews]
enabled        = true
base_domain    = "preview.acme.com"
ttl            = "7d"
auto_gc        = true

[cost]
budget_monthly_usd = 500
alert_threshold    = 0.8   # alert at 80% of budget

Step 2: Open a pull request.

git checkout -b feat/new-endpoint
echo '# change' >> main.py
git add main.py
git commit -m "feat: new endpoint"
git push origin feat/new-endpoint
# Open a PR on GitHub

The platform receives the pull_request webhook and starts the preview pipeline. Watch from the CLI:

astro preview list --app demo-app
# PR    BRANCH               STATUS
# 42    feat/new-endpoint    provisioning…

astro preview status --app demo-app --pr 42 --watch
# [scan]     trivy scan sha256:…         passed  (0 critical CVEs)
# [build]    image built                 done    sha256:def456
# [provision] preview env created        done
# [dns]       pr-42.preview.acme.com     done
# [tls]       cert issued                done
# STATUS: running

If the scan finds a critical CVE the pipeline halts:

# [scan]  trivy scan sha256:…  FAILED (3 critical CVEs)
# Deploy blocked. Fix vulnerabilities or override with `astro preview scan-override --pr 42 --reason "..."`.

Step 3: Verify the preview environment.

curl -sI https://pr-42.preview.acme.com/
# Expected: HTTP/2 200
# cert is valid (curl does not warn)

# The preview URL also appears as a GitHub PR status check
# Status check name: "astrolift/preview"
# Target URL: https://pr-42.preview.acme.com

Step 4: Check cost tracking.

astro cost show --app demo-app
# RESOURCE             MONTHLY EST.   PERIOD
# preview/pr-42        $4.20          2026-05-30 → now
# production/main      $38.10         2026-05-01 → now
# TOTAL                $42.30 / $500  budget (8.5%)

astro cost show --app demo-app --breakdown
# Compute:   $26.80
# Storage:   $8.40
# Egress:    $7.10
# (prices pulled from cloud pricing API)

Step 5: Verify GitOps reconciliation.

With GitOps mode enabled, every deploy writes the desired state to the GitOps config repo. Confirm:

# Check the config repo (ArgoCD example)
git clone git@github.com:acme/astrolift-gitops.git
ls astrolift-gitops/apps/demo-app/
# production.yaml   preview-pr-42.yaml

# Manually edit a value to simulate drift
# ArgoCD / Flux detects the deviation and reconciles back within the sync interval (default: 2 min)
kubectl patch deployment web -n acme-demo-app \
  --patch '{"spec":{"replicas":3}}'

# After the sync interval, replicas return to the manifest-declared value
kubectl get deployment web -n acme-demo-app
# READY   UP-TO-DATE   AVAILABLE
# 1/1     1            1        ← back to 1

Step 6: Preview environment garbage collection.

# Merge or close the PR → GC fires automatically
# Or trigger manually:
astro preview gc --app demo-app --pr 42

astro preview list --app demo-app
# PR    BRANCH               STATUS
# (empty — gc complete)

# DNS record is gone
nslookup pr-42.preview.acme.com
# NXDOMAIN

Success criteria:

A PR push triggers the preview pipeline within 10 seconds.
A Trivy scan with zero critical CVEs passes; one with critical CVEs blocks the deploy.
Preview URL is reachable over HTTPS with a valid cert within 3 minutes of the push.
A GitHub PR status check appears with the preview URL as its target.
astro cost show returns a non-zero monthly estimate; prices are from the cloud pricing API.
GitOps reconciliation reverts a manual patch within the sync interval.
Closing the PR triggers GC; DNS record is gone within 5 minutes.

Regression test path¶

make test PYTEST_ARGS="-m 'phase5' -x"

The phase5 marker covers:

tests/previews/test_preview_workflow.py — PreviewEnvironmentWorkflow provisions a preview env on PR open, creates a DNS record, and tears it down on PR close
tests/previews/test_scan_gate.py — a build with zero critical CVEs passes; a build with critical CVEs halts the workflow and records a scan.failed event; scan-override creates an override record and allows re-run
tests/previews/test_custom_domain.py — preview URL is derived from base_domain in manifest; cert-manager Certificate CR is created
tests/cost/test_cost_estimates.py — CostEstimator calls the cloud pricing API (mocked in unit tests, real in integration); returns a non-zero estimate; budget alert fires when estimate exceeds threshold
tests/gitops/test_reconciliation.py — GitOpsWriter commits desired state to the config repo on deploy; ArgoCD (or Flux) test server detects and reconciles drift back to desired state within configured interval
tests/previews/test_gc.py — PreviewGCWorkflow runs on PR close; preview namespace, DNS record, and TLS secret are all removed; Preview.status is gc_complete

CI gate: runs on every PR. Preview and GitOps integration tests run on every push to main using the kind cluster and a local ArgoCD install in CI.

Running the full regression suite¶

# All phases, stop on first failure
make test PYTEST_ARGS="-m 'phase0 or phase1 or phase2 or phase3 or phase4 or phase5' -x"

# All phases, continue on failure, with coverage report
make test PYTEST_ARGS="--cov=. --cov-report=term-missing"

Phase markers are defined in pytest.ini:

[pytest]
markers =
    phase0: Foundation hygiene regression tests
    phase1: Platform foundation regression tests
    phase2: Onboarding and deploy loop regression tests
    phase3: Operations and observability regression tests
    phase4: CLI and polish regression tests
    phase5: Advanced and beyond-parity regression tests

Tests are additive: adding a phase N test must not break any phase < N test. Maintain this invariant in CI by running the full suite on every PR.