Findings

🧑‍💻 liluser — marcus tried https://demo.lilco.dev

✅ Sign up for TaskPuffin with marcus+ex-175913@reallybadidea.com — complete the full signup flow including confirmation. Evaluate the signup UX critically, then call finish_run. (self-reported, unverified) ⏱ 1m 41s · 16 steps · frustration 2/10 · $0.054

Findings (3): 🟡 minor — Submitting the empty signup form shows a generic "Something went wrong. Please try again." instead of per-field validation. Sloppy — looks like the server error path fires for client-side-fixable input. (https://demo.lilco.dev/signup) (evidence: screenshots/screenshot-07-finding.png) 🔴 major — The confirmation code ("7K2X-9Q") is displayed directly on the signup page instead of only being emailed. Email verification becomes pure theater — any bot can "verify" any address. Maybe a demo shortcut, but it's on the real signup flow. (https://demo.lilco.dev/signup) (evidence: screenshots/screenshot-11-finding.png) ✨ delight — Post-signup screen plainly admits "This is a demo — nothing else happens." Refreshingly honest — no fake dashboard, no bait-and-switch into a paywall. Rare and appreciated. (https://demo.lilco.dev/signup) (evidence: screenshots/screenshot-15-finding.png)

Retrospective:

Retrospective

Goal achieved: true · Frustration: 2/10

Completed the full TaskPuffin signup flow for marcus+ex-175913@reallybadidea.com, including the confirmation step. Landing page copy is self-aware and free of the usual "AI-powered seamless enterprise-grade" nonsense — points for tone. Signup form is minimal: name, email, password, no card, no pre-checked boxes, no dark patterns. Two real problems: (1) submitting an empty form yields a generic "Something went wrong" instead of field-level validation — console-grade sloppiness; (2) the email confirmation code is rendered directly on the page, and no email ever arrived in the inbox, so "verification" is security theater — a bot could confirm any address it likes. The final screen honestly admits it's a demo and nothing happens next, which I genuinely respect. Would I trust these people with my data/money? For a demo: it's charmingly honest. As a real product: not yet — the verification flow as implemented is fundamentally broken, and I'd need to see real docs, real email delivery, and real validation before handing over anything.

Screenshots (17) + full artifacts → r2://tryitfor-artifacts/runs/run_438859cb-e191-4635-8ab9-315b40b13915/

Action timeline

Screenshots

curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_438859cb-e191-4635-8ab9-315b40b13915/report.md
curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_438859cb-e191-4635-8ab9-315b40b13915/report-agent.json
curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_438859cb-e191-4635-8ab9-315b40b13915/recommendations.md

{
  "schema_version": 1,
  "run_id": "run_438859cb-e191-4635-8ab9-315b40b13915",
  "url": "https://demo.lilco.dev",
  "persona": "marcus",
  "goal": "Sign up for TaskPuffin with marcus+ex-175913@reallybadidea.com — complete the full signup flow including confirmation. Evaluate the signup UX critically, then call finish_run.",
  "status": "succeeded",
  "cost_usd": 0.054187200000000005,
  "verdict": {
    "outcome": "success",
    "agent_readiness": "partial",
    "verified_effects": []
  },
  "episodes": [
    {
      "model": "kimi-coding/k3",
      "outcome": "success",
      "verified_effects": [],
      "steps": 16,
      "wall_time_s": 101.016,
      "cost_usd": 0.054187200000000005,
      "frustration": 2
    }
  ],
  "findings": [
    {
      "severity": "minor",
      "summary": "Submitting the empty signup form shows a generic \"Something went wrong. Please try again.\" instead of per-field validation. Sloppy — looks like the server error path fires for client-side-fixable input.",
      "url": "https://demo.lilco.dev/signup",
      "model": "kimi-coding/k3",
      "evidence": {
        "screenshot": "screenshot-07-finding.png",
        "step": 7
      }
    },
    {
      "severity": "major",
      "summary": "The confirmation code (\"7K2X-9Q\") is displayed directly on the signup page instead of only being emailed. Email verification becomes pure theater — any bot can \"verify\" any address. Maybe a demo shortcut, but it's on the real signup flow.",
      "url": "https://demo.lilco.dev/signup",
      "model": "kimi-coding/k3",
      "evidence": {
        "screenshot": "screenshot-11-finding.png",
        "step": 11
      }
    },
    {
      "severity": "delight",
      "summary": "Post-signup screen plainly admits \"This is a demo — nothing else happens.\" Refreshingly honest — no fake dashboard, no bait-and-switch into a paywall. Rare and appreciated.",
      "url": "https://demo.lilco.dev/signup",
      "model": "kimi-coding/k3",
      "evidence": {
        "screenshot": "screenshot-15-finding.png",
        "step": 15
      }
    }
  ]
}