🧑💻 dana tried https://demo.lilco.dev self-reported, unverified
🧑💻 liluser — dana tried https://demo.lilco.dev
✅ Sign up for TaskPuffin with dana+ex-175912@reallybadidea.com — complete the full signup flow including confirmation. Evaluate the signup UX critically, then call finish_run. (self-reported, unverified) ⏱ 2m 9s · 13 steps · frustration 2/10 · $0.043
Findings (2): 🔴 major — Confirmation code (7K2X-9Q) is displayed directly on the page instead of being emailed. No email ever arrived in my inbox. That defeats the point of confirming my email — I could have typed a fake address. Fast for me, but feels sketchy. (https://demo.lilco.dev/signup) (evidence: screenshots/screenshot-09-finding.png) 🟡 minor — Friendly welcome screen confirms my account, but post-signup there is literally no product — "This is a demo — nothing else happens." No dashboard, no board, nothing to evaluate. Also 'workspace' is jargon to me. (https://demo.lilco.dev/signup) (evidence: screenshots/screenshot-12-finding.png)
Retrospective:
Goal achieved: true · Frustration: 2/10
TaskPuffin: signed up and confirmed successfully in about 2 minutes. Landing page was excellent — "task tracker, free for small teams," understood in under 10 seconds, no jargon, funny self-aware copy ("tasks stay where you drop them, usually"). The signup form was dead simple: name, email, password, one button. Two critiques: (1) no inline password rules — I guessed and it worked, but if it had failed I'd have no idea why; (2) the "confirmation" is fake — it just shows the code (7K2X-9Q) on the page itself. I checked my email and nothing ever arrived. That's convenient but totally pointless as verification; I could've used a fake email. After confirming I got a cheery welcome screen and then... nothing. No actual product, just "this is a demo, nothing else happens," so I couldn't judge whether the value is real. Time-to-clarity: great (~10s). Trust: decent but dinged by the fake confirmation. Overall a fast, low-friction signup with a security-theater email step.
Screenshots (14) + full artifacts → r2://tryitfor-artifacts/runs/run_33bbb2f1-a618-4f59-8019-b31caafe0880/
| # | tool | args | url | time |
|---|---|---|---|---|
| 1 | browser_navigate | {"url":"https://demo.lilco.dev"} | demo.lilco.dev/ | 4.3s |
| 2 | browser_snapshot | {} | demo.lilco.dev/ | 0.7s |
| 3 | browser_click | {"name":"Start free","role":"link"} | demo.lilco.dev/signup | 1.5s |
| 4 | browser_snapshot | {} | demo.lilco.dev/signup | 0.6s |
| 5 | browser_type | {"name":"Full name","role":"textbox","text":"Dana Reyes"} | 0.6s | |
| 5 | browser_type | {"name":"Email","role":"textbox","text":"dana+ex-175912@reallybadidea.com"} | 0.9s | |
| 5 | browser_type | {"name":"Password","role":"textbox","text":"PuffinNest2025!"} | 0.6s | |
| 6 | browser_click | {"name":"Continue","role":"button"} | demo.lilco.dev/signup | 1.5s |
| 7 | browser_snapshot | {} | demo.lilco.dev/signup | 0.7s |
| 8 | inbox_check | {"email":"dana+ex-175912@reallybadidea.com"} | 0.2s | |
| 9 | report_finding | {"note":"Confirmation code (7K2X-9Q) is displayed directly on the page instead o | 1.0s | |
| 9 | browser_type | {"name":"Code","role":"textbox","text":"7K2X-9Q"} | 0.7s | |
| 10 | browser_click | {"name":"Confirm","role":"button"} | demo.lilco.dev/signup | 1.4s |
| 11 | browser_snapshot | {} | demo.lilco.dev/signup | 0.6s |
| 12 | report_finding | {"note":"Friendly welcome screen confirms my account, but post-signup there is l | 0.9s | |
| 13 | finish_run | {"frustration":2,"goal_achieved":true,"summary":"TaskPuffin: signed up and confi | 0.1s |
Machine-legible report. Authed endpoints (API key required):
curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_33bbb2f1-a618-4f59-8019-b31caafe0880/report.md
curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_33bbb2f1-a618-4f59-8019-b31caafe0880/report-agent.json
curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_33bbb2f1-a618-4f59-8019-b31caafe0880/recommendations.md{
"schema_version": 1,
"run_id": "run_33bbb2f1-a618-4f59-8019-b31caafe0880",
"url": "https://demo.lilco.dev",
"persona": "dana",
"goal": "Sign up for TaskPuffin with dana+ex-175912@reallybadidea.com — complete the full signup flow including confirmation. Evaluate the signup UX critically, then call finish_run.",
"status": "succeeded",
"cost_usd": 0.042692400000000005,
"verdict": {
"outcome": "success",
"agent_readiness": "partial",
"verified_effects": []
},
"episodes": [
{
"model": "kimi-coding/k3",
"outcome": "success",
"verified_effects": [],
"steps": 13,
"wall_time_s": 128.796,
"cost_usd": 0.042692400000000005,
"frustration": 2
}
],
"findings": [
{
"severity": "major",
"summary": "Confirmation code (7K2X-9Q) is displayed directly on the page instead of being emailed. No email ever arrived in my inbox. That defeats the point of confirming my email — I could have typed a fake address. Fast for me, but feels sketchy.",
"url": "https://demo.lilco.dev/signup",
"model": "kimi-coding/k3",
"evidence": {
"screenshot": "screenshot-09-finding.png",
"step": 9
}
},
{
"severity": "minor",
"summary": "Friendly welcome screen confirms my account, but post-signup there is literally no product — \"This is a demo — nothing else happens.\" No dashboard, no board, nothing to evaluate. Also 'workspace' is jargon to me.",
"url": "https://demo.lilco.dev/signup",
"model": "kimi-coding/k3",
"evidence": {
"screenshot": "screenshot-12-finding.png",
"step": 12
}
}
]
}1 concrete fix available on the paid tier. Key holders: GET https://api.liluser.dev/v1/runs/run_33bbb2f1-a618-4f59-8019-b31caafe0880/recommendations.md · upgrade