Findings

🧑‍💻 liluser — dana tried https://liluser.lilco.dev

❌ Explore this product's website: read the landing page and try to sign in / get started as far as you can. Note any friction you hit, then call finish_run. — goal not achieved ⏱ 56s · 9 steps · frustration 4/10 · $0.042

Findings (3): 🟡 minor — I get the gist fast — it tests whether an AI can actually use your website, which is clever. But it's full of jargon: "API", "CLI", "GitHub Action", "AGENTS.md", "oracle-verified". As a first-time operator I have no idea how I'd even trigger a test. Time-to-clarity on the concept: ~20 seconds. Time-to-clarity on how I would use it: unclear. (https://liluser.lilco.dev/) ✨ delight — The live sample report is great marketing — plain English, shows time (1m56s), frustration score, and a delight finding. THIS made the value concrete for me way better than the jargon-heavy landing page. Wish the homepage led with this. (https://api-liluser.lilco.dev/r/run_39691223-655e-4bf0-8811-06fa35e04bf7) 🔴 major — The ONLY way to get access is "Sign in with GitHub" — a developer tool I don't have an account for. There's no email signup, no waitlist form, no "request an invite" for non-developers. As an operator evaluating tools, I'm a dead end here even though the report page convinced me of the value. Also "beta is invite-only" but there's no visible way to request an invite. (https://liluser.lilco.dev/)

Retrospective:

Retrospective

Goal achieved: false · Frustration: 4/10

I landed on liluser and understood the core idea in about 20 seconds: it sends an AI "persona" to use your website like a real new user and tells you where it gets stuck. That headline ("Your product's first user is an agent") actually worked for me — clear and intriguing. But the page then assumes I'm a developer: "API key", "CLI", "GitHub Action", "AGENTS.md", "npx liluser init". As a first-time operator with a new LLC, none of that means anything to me, and there's no plain-language path explaining how a non-technical person would run a test. The best moment was the linked live report — a real run written in plain English with time, steps, frustration score, and findings. THAT sold me on the value more than anything else on the homepage; I'd lead with it. Then I hit the wall: the only way in is "Sign in with GitHub". I don't have GitHub, I'm not a developer, and there's no email signup, no waitlist, no "request an invite" form despite the page saying beta is invite-only. So I bounced at the door. Verdict: strong concept, great proof artifact, but the product is clearly for developers only right now and doesn't say so upfront — I spent my 30 seconds of goodwill before discovering I'm not the audience.

Full artifacts → r2://tryitfor-artifacts/runs/run_1fde8340-713c-4ef8-982d-edb72dd07134/

Screenshots

No screenshots captured for this run.

curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_1fde8340-713c-4ef8-982d-edb72dd07134/report.md
curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_1fde8340-713c-4ef8-982d-edb72dd07134/report-agent.json
curl -H "authorization: Bearer $LILUSER_API_KEY" https://api.liluser.dev/v1/runs/run_1fde8340-713c-4ef8-982d-edb72dd07134/recommendations.md

{
  "schema_version": 1,
  "run_id": "run_1fde8340-713c-4ef8-982d-edb72dd07134",
  "url": "https://liluser.lilco.dev",
  "persona": "dana",
  "goal": "Explore this product's website: read the landing page and try to sign in / get started as far as you can. Note any friction you hit, then call finish_run.",
  "status": "succeeded",
  "cost_usd": 0.041528999999999996,
  "verdict": {
    "outcome": "failed",
    "agent_readiness": "fail",
    "verified_effects": []
  },
  "episodes": [
    {
      "model": "kimi-coding/k3",
      "outcome": "failed",
      "verified_effects": [],
      "steps": 9,
      "wall_time_s": 56.02,
      "cost_usd": 0.041528999999999996,
      "frustration": 4
    }
  ],
  "findings": [
    {
      "severity": "minor",
      "summary": "I get the gist fast — it tests whether an AI can actually use your website, which is clever. But it's full of jargon: \"API\", \"CLI\", \"GitHub Action\", \"AGENTS.md\", \"oracle-verified\". As a first-time operator I have no idea how I'd even trigger a test. Time-to-clarity on the concept: ~20 seconds. Time-to-clarity on how *I* would use it: unclear.",
      "url": "https://liluser.lilco.dev/",
      "model": "kimi-coding/k3"
    },
    {
      "severity": "delight",
      "summary": "The live sample report is great marketing — plain English, shows time (1m56s), frustration score, and a delight finding. THIS made the value concrete for me way better than the jargon-heavy landing page. Wish the homepage led with this.",
      "url": "https://api-liluser.lilco.dev/r/run_39691223-655e-4bf0-8811-06fa35e04bf7",
      "model": "kimi-coding/k3"
    },
    {
      "severity": "major",
      "summary": "The ONLY way to get access is \"Sign in with GitHub\" — a developer tool I don't have an account for. There's no email signup, no waitlist form, no \"request an invite\" for non-developers. As an operator evaluating tools, I'm a dead end here even though the report page convinced me of the value. Also \"beta is invite-only\" but there's no visible way to request an invite.",
      "url": "https://liluser.lilco.dev/",
      "model": "kimi-coding/k3"
    }
  ]
}