Out of Sight, Out of Mind: Improving Visualization of AI Info

Editor's Notes

• #3: Introduction to ReView, some motivation behind it, then taking a look at the visualizing runtime flow, moving on to recording arbitrary data, then visualization of algorithms and closing with quick look behind the scenes.
• #4: Even when you have different means to visualize data it does not mean that it is easy to figure out how to visualize it best. Too often the easy way is chosen instead of the best way. Also, there is no silver bullet, when you want to have a good visualization that most likely means getting hands dirty in order to implement it. Especially consider how time is being represented in the visualizations, you can collapse multiple phases / states into one single image, have interactive manipulation of the time frame or perhaps use a timeline to show events which can then be selected for closer inspection. There are plenty of good books out there to help you figuring out how to create simple but powerful visualizations, one of my favorites is ’The Visual Display of Quantitative Information’ by Edward Tufte.
• #5: These are some of the principles that I’ve had when working with visualization of game data. Minimizing the impact on client is important to have as little effect as possible on the gameplay itself, game should be able to run all the levels with roughly the same speed as without collecting all this information. Also, API should be very lean to have as little code added into the client as possible; this helps when code changes and you need to update the visualizing code as well. This can be achieved by having the tool running as a separate process and not store any debug information in client-side. UI for the tool / visualizer should not need user’s manual, also it should not distract the user from inspecting the actual data that is being shown. Further, it will help to keep visualization simple as well by way of showing a good example.
• #6: One of the first things that I try to understand when starting to work with a new system or code base, is how does the runtim flow look like and what are the major components. This was the case also when I started working with Killzone Shadowfall MP Bots, it took me only few days to hook up the visualization while I was reading documentation, asking questions and browsing through the code. After that I had the ability to see live how the runtime flow of various AI systems looked like. Furthermore, even if you know how it should work, it might be difficult to verify that it actually does what it is supposed to do.
• #7: First thing I wanted to solve was the runtime flow visualization of different components. Since I wanted to show both structure, in other words how to visualize ownership or dependencies of components, and full history of these individual instances, it made perfect sense to create a sequencer view similar to video editing tools. You get a temporal and structural view on your data simultaneously. It gives a good overview and at the same time you can focus your gaze to bring details about some specific component.
• #9: There are two components in this view, tracks and items. Tracks have a parent which forms the hierarchy and items always belong to a track. Items may not overlap, to have a simple interface to deal with this constraint it is possible to let the tool to create child tracks automatically if no available track can be found for item. This was demonstrated in the video with ’anonymous concurrent tracks’. Each item in turn contains log entries which can be viewed when clicking on item. Each entry has a 32bit flag parameter which can be used to set different attributes for them. Tool lets user to assign a color for each attribute (32 in total) which allows easy and generalized interface to color items on the tracks. Since there might still be a lot of data, as was shown in the video, filtering can be done by using regular expressions or flags. Many game engines already have some kind of sequencer available so creating a view like this would probably be quite simple.
• #10: After using the timeline view to get a good overview of the runtime flow and taking advantage of the history it became quickly annoying that it was not possible to ’rewind’ the game and observe how things looked at a specific point in time. Creating a system that would record everything or requiring full determinism from a game engine would, in most cases, be a tall order. Solution had to be something simple which would not require massive changes on the client side. Idea for Binary Storage came from multiplayer code which was sending information to other clients about state of an object. Why not just send this information to a server that marks the sender ID, time and stores it for later re-use. When sending information back the component serializing the information receives it back and can then deserialize it and set the current state to match. It would not often be a huge change to add serialize and deserialize functions for a game object which in addition would have a double-buffered state that is used in the game logic and rendering. This simply means that when receiving information back you can set that to be in the buffer to use while retaining the actual current state in the object. In other words not overwriting current state with debug state.
• #13: The most recent addition into ReView came after I got frustrated working with few ’trivial’ problems; convex hull generation and Delaunay triangulation. I completed both of them eventually with the usual way of debugging and observing end results. While working with these I run into several visualizations in the web of different algorithms solving these problems and started wondering, why couldn’t I have a simple way to visualize and observer step-by-step what my algorithms were doing. First solution I tried was to utilize the Binary Storage and then write a separate simple rendering of that data but very soon it became too cumbersome to send different rendering primitives to the storage and create the visualizations separately. Nicely packaged remote debug renderer sounded like a much better idea and that’s what I started working with.
• #16: Try to keep the interface as simple and clean as possible. ReView assumes that users are able to extend the interface and tool when the need arises, this allows for a more specific interface.
• #17: RPC system was refactored into ReView after realizing how much boiler-plate code needed to be done for every new feature that was added to the original concept. This is important to allow for users to extend functionality easily. RPC system nicely forces separation of the tool / visualization from the client side code while it makes it still easy to do the communication. Tool is made with C# using WinForms. VS Designer together with C# makes it nice to work with the UI side of things.