Manual QA Test Plan

Run this checklist before every submission or significant release. Use a fresh development store unless noted.

For full editing coverage, execute and annotate the companion matrix:

• full-editing-manual-matrix.md

Environment prep

• Use latest staging build (Render deployment or local tunnel).
• Confirm env vars (TEST_MODE=0) so usage gating and billing behave realistically.
• Seed store with sample products (different titles, tags, variants, prices).

Smoke tests

• [ ] Install app via Shopify Admin → Apps → Bulk Product Editor.
• [ ] Verify embedded nav renders without console errors.
• [ ] Confirm session token/auth flow succeeds (no redirect loops).

Bulk edit workflows

• [ ] New bulk edit:
  • Build filters using UI controls (title contains “Sample”).
  • Preview count updates.
  • Add a single action (tags add/remove, price round).
  • Create task and confirm status transitions: QUEUED → RUNNING → COMPLETED.
  • Verify job logs (info/warn entries) and diff records.
• [ ] Variant-only SKU/barcode actions (automated in tests/bulk-tasks/sku-and-barcode.pw.spec.ts; run Playwright before full regression, then spot-check manually as needed):
  • Configure SKU replace and barcode append actions with variant filters.
  • Ensure task creation redirects to Recent tasks and logs completed jobs.
• [ ] Save & reuse filter groups/templates: create saved filter group and template, reapply on new job.
• [ ] Variant targeting – selection modes:
  • Create a product with ≥3 variants (different options).
  • Add a Price or SKU action. Confirm the default summary reads “Variants to edit: All variants on each matching product”.
  • Click Change to open Variant Filter & Selection.
  • Add a simple variant filter (e.g., Variant option 2 = Red) and verify:
    • All variants: all matching variants on each product are edited.
    • First variant only: only the first matching variant by position is edited.
    • Last variant only: only the last matching variant by position is edited.
    • By number: entering 2 in “Variant number (1 = first match)” edits only the second matching variant.
    • By option: supplying Option 1/2/3 values (e.g., Option 2 = Red) edits only variants whose options match exactly.
  • Verify the “Variants to edit …” summary line reflects the current filter + selection.
• [ ] Filters – Shopify search alignment:
  • Add a product-level filter using Variant price and confirm it behaves like Shopify’s price search (e.g., between 10 and 20 finds the expected products).
  • Add variant-only filters (Variant title, Variant option 1/2/3, Variant position) and confirm they affect preview rows and which variants are edited, but not which products initially appear when only product-level fields are used.
• [ ] Metafield filters (if enabled in this release):
  • On two test products, set a metafield such as custom.review_note = ok on one, and leave it unset or different on the other.
  • In the Filter products card, add a Metafield rule custom.review_note equals ok.
  • Confirm only the product with the matching metafield appears in preview and is affected by a simple action (e.g., add a tag).

CSV workflows

• [ ] Export CSV (product ID, handle, title).
• [ ] Import CSV with valid row → job created successfully.
• [ ] Import CSV with invalid data → validation error surfaced, job not created.
• [ ] Large CSV (≥50 rows) processes without UI hang (monitor queue logs).

Usage & billing

• [ ] Run jobs until current plan cap reached (or override cap via /app/usage in non-prod) → redirect to Usage page shows upgrade message and Upgrade button.
• Automated check: tests/non-bulk/usage-gating.pw.spec.ts asserts the error banner + Upgrade CTA render when ?error= query parameters are present.
• [ ] Approve billing charge in development store (Shopify billing page).
• [ ] Verify active subscription detected and usage cap lifted.
• [ ] Cancel subscription from Shopify Admin and confirm app reflects downgrade (next visit to Usage page).

Error handling & resilience

• [ ] Simulate Shopify throttling (trigger via multiple jobs) → backoff and retry logs recorded, job completes eventually.
• [ ] Cancel a running job from Jobs list and confirm it halts further processing.
• [ ] Force a mutation failure (e.g., attempt to set invalid price) → change marked failed, logs show error, rest of job continues.

Privacy & compliance

• [ ] Trigger mandatory privacy webhooks via Shopify CLI; ensure HTTP 200 and data cleanup:
  • customers/data_request
  • customers/redact
  • shop/redact
• [ ] Uninstall app → sessions, usage, and jobs marked uninstalledAt are cleaned (verify DB or logs).
• [ ] Reinstall app → auth flow succeeds; no stale state.

UI/UX checks

• [ ] Verify empty states for Jobs, Saved filters, Templates.
• [ ] Confirm skeleton/loading states appear during network calls.
• [ ] Validate accessibility basics: tab navigation, aria labels on buttons (ActionEditor).
• [ ] Ensure Polaris theming (light/dark) works if available.

Regression sanity

• [ ] Job queue continues running after server restart (check logs).
• [ ] Health page /app/health reports env and DB status correctly.
• [ ] Background job metrics logged (monitor console/host logs).

Document any failures with screenshots/logs and fix before submission. Update this plan as new features ship.