❓Questions

To research how we could improve an agent-based development workflow with Storybook, we defined a set of goals:

1. Whenever the user prompts for UI to be generated or modified, the agent should know to write/update stories appropriately.
2. The stories should be of high quality, with correct CSF syntax and interaction testing where necessary.
3. The stories should match the project's setup in terms of imports, language, etc.
4. At the end of the UI writing process, the agent should respond with one or more links to stories that have been affected, for the user to visually inspect and review.

To reach these goals we defined a set of questions we needed to answer:

1. How do we make the LLM consistently write and update stories when working with UI?Can we change an LLMs behavior so the user doesn’t need to explicitly ask the LLM to write stories, but instead the LLM just knows that it’s a core part of any tasks that requires UI work?

2. How can we make an LLM write “good” stories?

   • What context or information will make an LLM write the best stories possible, including valid syntax and using best practices around args, play-functions, etc.?

3. How can we make an LLM send valid story links to the user when relevant?

Investigating these questions and trying out different solutions is what has ultimately led to the experimental release of @storybook/addon-mcp :

📖 UI Building Instructions Tool

We found that “hiding” these instructions behind an MCP tool was an effective way of managing the LLMs’ context as much as possible. In general, minimising the usage of LLMs’ context windows is critical to get good results. Not only are there hard limits and costs associated with having too many tokens in the context. LLMs perform worse and worse the more context they have - similar to how humans struggle to focus on a task if they get bombarded with irrelevant information. Our findings show that generally the LLM will only request the UI instructions when it’s working on UI, ensuring that it doesn’t consider it when it’s working on other tasks eg. backend only.

The combination of the system prompt, the tool’s description, and the paragraph about when to write stories have proven to be an effective way to nudge the LLM to write and update stories when working on UI components, without requiring a specific user prompt for it. However AI’s are unreliable, models, agents and clients all behave different, so it’s not a surefire way to achieve this. We believe that the reliability can be improved in the future by iterating on the prompts (feedback and ideas welcome! 🙏).

We found that generally speaking, LLMs have an okay understanding of CSF and how to write stories already embedded in their training data from all the publicly available content there already exists about Storybook today. However they struggle to stay up to date, and so providing the migration guide and instructions on new features was a way to ensure it wrote valid and modern stories. It had a noticeable impact on the validity of the output, however there’s still much to be desired when it comes to writing “good” stories. More Prompt Engineering™ is needed to make it understand best practices better, eg. relevant usage of assertions in play-functions and useful args combinations.

Finally, including clear instructions on when and how to provide story links to the users proved to be very effective. Our findings indicate that with the MCP server the LLM will consistently end tasks with a set of links that the user can visit to inspect the UI work the LLM has performed. Not only when creating new stories, but also when working on UI only without any story modifications.

🔗 Story URL Tool

An exact URL to a story requires a few things:

• The origin, eg. http://localhost:6006
• The path to the stories-file, eg. ./src/Button.stories.ts
• The export name of the story, eg. WithLongLabel
• Optionally the custom name of the story, eg. Long Label

With this information it is possible for the tool to programmatically look up the story index and find the stories’ IDs. From there it’s just about combining the IDs with the origin correctly.

To get an understanding of the LLMs knowledge and capabilities we first explored if it could generate these URLs on its own, without assistance from a tool. We found that it worked in the most basic scenarios, where stories-files had explicit meta titles, no custom story name, and where running on localhost:6006. That was too limiting for practical, but interesting nonetheless that it understood the Storybook URL structure.

We then gave the LLM information about how Storybook internally constructs the story IDs, by giving it the raw code that does it, or alternatively an LLM generated description of the code. This was very reliable, and the LLM correctly gathered all the necessary information and produced valid links in all advanced scenarios. But it struggled to get the origin right when it was not localhost:6006, which led to the creation of the tool that does it programmatically and reliably.

In future iterations we need to make sure the LLM always passes stories’ custom names into the tool. Currently it’s a two-shot process where the tool will return an error reminding the LLM to pass in a story’s custom name, and then the LLM will get it right the second time.

🤔 Why an addon

There are many ways to distribute an MCP server. Here’s a quick overview of what we explored and the pros and cons:

• Remote MCP Server, that we host somewhere. It’s easy to set up for users as they just provide a link, but it also has very limited capabilities, as it doesn’t have access to anything local (filesystem, local servers), it only have access to the information that the agent provides during a tool call. Local files are the main input here, so this was a no-go.
• Local MCP Server, that the user configures with an npx command or similar. This is the regular way to distribute MCP servers, and is capable of accessing anything locally. The downside of this is that if it needs access to anything from the Storybook runtime - stories, the index, statuses, etc. - it would have to request that from a running Storybook dev server, and we would have to expose that in API endpoints in Storybook core. This would be cumbersome to maintain, it would be slower to iterate on and release, and the MCP server would only be compatible with specific Storybook versions.
• Storybook addons have access to a lot of information from within the Storybook runtime. The can access the channel, modify presets, read configurations, etc. The Story URL tool makes use of this, by determining the Storybook framework from the user config, and adding that to the UI Building Instructions so the LLM knows that automatically. You can imagine in the future it could modify the manager UI, read story statuses, inject afterEach-hooks, etc. The MCP server will be tied to a specific instance, and in the future we’d need to figure out how that would play out in monorepo situations where there might be multiple Storybooks running at the same time.