🧑💻 marcus tried https://demo.lilco.dev outcome self-reported — no verifiable side effects this run
Findings
Submitting the empty signup form shows a generic "Something went wrong. Please try again." instead of per-field validation. Sloppy — looks like the server error path fires for client-side-fixable input.
The confirmation code ("7K2X-9Q") is displayed directly on the signup page instead of only being emailed. Email verification becomes pure theater — any bot can "verify" any address. Maybe a demo shortcut, but it's on the real signup flow.
Post-signup screen plainly admits "This is a demo — nothing else happens." Refreshingly honest — no fake dashboard, no bait-and-switch into a paywall. Rare and appreciated.
🧑💻 liluser — marcus tried https://demo.lilco.dev
✅ Sign up for TaskPuffin with marcus+ex-175913@reallybadidea.com — complete the full signup flow including confirmation. Evaluate the signup UX critically, then call finish_run. (self-reported, unverified) ⏱ 1m 41s · 16 steps · frustration 2/10 · $0.054
Findings (3):
- 🟡 minor — Submitting the empty signup form shows a generic "Something went wrong. Please try again." instead of per-field validation. Sloppy — looks like the server error path fires for client-side-fixable input. (https://demo.lilco.dev/signup) (evidence: screenshots/screenshot-07-finding.png)
- 🔴 major — The confirmation code ("7K2X-9Q") is displayed directly on the signup page instead of only being emailed. Email verification becomes pure theater — any bot can "verify" any address. Maybe a demo shortcut, but it's on the real signup flow. (https://demo.lilco.dev/signup) (evidence: screenshots/screenshot-11-finding.png)
- ✨ delight — Post-signup screen plainly admits "This is a demo — nothing else happens." Refreshingly honest — no fake dashboard, no bait-and-switch into a paywall. Rare and appreciated. (https://demo.lilco.dev/signup) (evidence: screenshots/screenshot-15-finding.png)
Retrospective:
Retrospective
Goal achieved: true · Frustration: 2/10
Completed the full TaskPuffin signup flow for marcus+ex-175913@reallybadidea.com, including the confirmation step. Landing page copy is self-aware and free of the usual "AI-powered seamless enterprise-grade" nonsense — points for tone. Signup form is minimal: name, email, password, no card, no pre-checked boxes, no dark patterns. Two real problems: (1) submitting an empty form yields a generic "Something went wrong" instead of field-level validation — console-grade sloppiness; (2) the email confirmation code is rendered directly on the page, and no email ever arrived in the inbox, so "verification" is security theater — a bot could confirm any address it likes. The final screen honestly admits it's a demo and nothing happens next, which I genuinely respect. Would I trust these people with my data/money? For a demo: it's charmingly honest. As a real product: not yet — the verification flow as implemented is fundamentally broken, and I'd need to see real docs, real email delivery, and real validation before handing over anything.
Screenshots (17) + full artifacts → r2://liluser-artifacts/runs/run_438859cb-e191-4635-8ab9-315b40b13915/
Action timeline
Timeline — kimi-coding__k3 (19 steps)
| # | tool | args | url | time |
|---|---|---|---|---|
| 1 | browser_navigate | {"url":"https://demo.lilco.dev"} | demo.lilco.dev/ | 16.2s |
| 2 | browser_snapshot | {} | demo.lilco.dev/ | 0.2s |
| 3 | browser_click | {"name":"Sign up","role":"link"} | demo.lilco.dev/signup | 0.6s |
| 4 | browser_snapshot | {} | demo.lilco.dev/signup | 0.2s |
| 5 | browser_click | {"name":"Continue","role":"button"} | demo.lilco.dev/signup | 0.1s |
| 6 | browser_snapshot | {} | demo.lilco.dev/signup | 0.1s |
| 7 | report_finding | {"note":"Submitting the empty signup form shows a generic \"Something went wrong | 0.2s | |
| 8 | browser_type | {"name":"Full name","role":"textbox","text":"Marcus Devlin"} | 0.1s | |
| 8 | browser_type | {"name":"Email","role":"textbox","text":"marcus+ex-175913@reallybadidea.com"} | 0.1s | |
| 8 | browser_type | {"name":"Password","role":"textbox","text":"Tr0ub4dor&3Puffin"} | 0.2s | |
| 9 | browser_click | {"name":"Continue","role":"button"} | demo.lilco.dev/signup | 0.5s |
| 10 | browser_snapshot | {} | demo.lilco.dev/signup | 0.1s |
| 11 | report_finding | {"note":"The confirmation code (\"7K2X-9Q\") is displayed directly on the signup | 0.2s | |
| 11 | inbox_check | {"email":"marcus+ex-175913@reallybadidea.com"} | 0.3s | |
| 12 | browser_type | {"name":"Code","role":"textbox","text":"7K2X-9Q"} | 0.1s | |
| 13 | browser_click | {"name":"Confirm","role":"button"} | demo.lilco.dev/signup | 0.2s |
| 14 | browser_snapshot | {} | demo.lilco.dev/signup | 0.1s |
| 15 | report_finding | {"note":"Post-signup screen plainly admits \"This is a demo — nothing else happe | 0.2s | |
| 16 | finish_run | {"frustration":2,"goal_achieved":true,"summary":"Completed the full TaskPuffin s | 0.0s |
Screenshots
For agents
Machine-legible report. Authed endpoints (API key required):
report.md — the human report as markdown:
curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_438859cb-e191-4635-8ab9-315b40b13915/report.mdreport-agent.json — findings + evidence as structured JSON:
curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_438859cb-e191-4635-8ab9-315b40b13915/report-agent.jsonreport-agent.json
{
"schema_version": 1,
"run_id": "run_438859cb-e191-4635-8ab9-315b40b13915",
"url": "https://demo.lilco.dev",
"persona": "marcus",
"goal": "Sign up for TaskPuffin with marcus+ex-175913@reallybadidea.com — complete the full signup flow including confirmation. Evaluate the signup UX critically, then call finish_run.",
"status": "succeeded",
"cost_usd": 0.054187200000000005,
"verdict": {
"outcome": "success",
"agent_readiness": "partial",
"verified_effects": []
},
"episodes": [
{
"model": "kimi-coding/k3",
"outcome": "success",
"verified_effects": [],
"steps": 16,
"wall_time_s": 101.016,
"cost_usd": 0.054187200000000005,
"frustration": 2
}
],
"findings": [
{
"severity": "minor",
"summary": "Submitting the empty signup form shows a generic \"Something went wrong. Please try again.\" instead of per-field validation. Sloppy — looks like the server error path fires for client-side-fixable input.",
"url": "https://demo.lilco.dev/signup",
"model": "kimi-coding/k3",
"evidence": {
"screenshot": "screenshot-07-finding.png",
"step": 7
}
},
{
"severity": "major",
"summary": "The confirmation code (\"7K2X-9Q\") is displayed directly on the signup page instead of only being emailed. Email verification becomes pure theater — any bot can \"verify\" any address. Maybe a demo shortcut, but it's on the real signup flow.",
"url": "https://demo.lilco.dev/signup",
"model": "kimi-coding/k3",
"evidence": {
"screenshot": "screenshot-11-finding.png",
"step": 11
}
},
{
"severity": "delight",
"summary": "Post-signup screen plainly admits \"This is a demo — nothing else happens.\" Refreshingly honest — no fake dashboard, no bait-and-switch into a paywall. Rare and appreciated.",
"url": "https://demo.lilco.dev/signup",
"model": "kimi-coding/k3",
"evidence": {
"screenshot": "screenshot-15-finding.png",
"step": 15
}
}
]
}