Musing on Visual Shader Tools
Recently, I started working on a visual shader construction tool; something along the lines of the Unreal material editor, the now-unavailable ShaderWorks, and the visual render path tools present in modeling packages like Maya and Blender. I also borrowed heavily from some concepts presented in Morgan McGuire’s “Abstract Shade Trees” paper.
Such tools are uncommon. Unreal’s tool is available only to those with an Unreal Engine license; other studios, such as the Project Offset team, Activision, and Snowblind Studios have tools that are proprietary. McGuire’s paper does not provide a demonstration application. Essentially, there are few such tools available to the public, although they are becoming more prevalent (FX Composer 2, for example, now includes such a tool).
My tool, which I’ve oh-so-creatively dubbed “Shade,” is more-or-less functional, although not exactly ready for public consumption. I still feel it’s an open question whether or not these kinds of tools are actually relevant or useful. The user interface for a tool like this involves, primarily, a directed acyclic graph; nodes in the graph represent operations, elements or states, and edges represent connections or dependencies between nodes. The actual execution path of the shader would be the topological ordering of the graph nodes. The crux, I think, is how the tool uses the nodes — where those nodes fit within the abstraction continuum between low and high level operations.
A low-level approach would have each node represent an operation (add, multiply, lerp, invert, et cetera). Connections between nodes represent the flow of data from one operation to the next. In my opinion, this is far too low-level to be of any use to anybody. There is no benefit to programmers, because they could probably type the code directly in a fraction of the time. Nor is there a benefit to non-programmers; while the tool is somewhat more accessible due to its visual nature, the user still essentially writing a program and the “programmer’s mindset” is still required to make the most of the tool.
Yet at the other end of the spectum is the completly high-level approach, where nodes represent large, complex features of a shader. While this approach is probably easier to work with, it suffers from the fact that the abstraction is probably too high-level; so much so that there is little practical purpose in placing nodes in the graph in any arbitrary order. Only a handful of placements would produce a useful looking shader, and so a more efficient GUI for such a system would probably be a simple list box. The successful compromise, hopefully (as it’s what I’ve bet my implementation on) lies in giving the tool the flexibility to scale the abstraction of the nodes so that the graphics programmers can tailor the available node set to the style of requests they get from the art team.
A typical workflow bottleneck between artist and programmer manifests itself in the form of an artist’s request to a programmer for a new or modified shading effect. Usually the artist will be prevented from completing whatever he was originally working on until the programmer can implement the effect, and the programmer, obviously, must take time away from his current task to write the shader. Furthermore, implementation of the effect may require tedious and time-consuming iteration, even after the artist has initially signed off on the implementation.
An artist’s request tends to be of the form, “I’d like to put our snow shader on our new unit, but we want all units to self-shadow and the snow shader doesn’t support that, we need a new shader,” or “It would be cool if the building’s glow effect was tinted by its team color, and maybe we could have the intensity of the glow vary base on the viewing direction.” We can break these down into required features (snow, self-shadowing, glow, tinting, and Fresnel term) and relations among those features (apply self-shadowing to the snow, use team color as the tint input, et cetera).
The features themselves would be implemented by a graphics programmer using actual shader code. The key would be to provide flexibility in the way the tool handled the implementation of these features so that the programmer could, as mentioned before, tailor them to the language and abstraction level used by the team. This isn’t something that can be automated, really, and would definitely require some practice and false-starts to get right. The success or failure of the visual shader tool probably depends on whether or not the programmers can effectively use the tool to create the correct set of nodes for their project.
The artists would use the tool to define the relationships between nodes that the programmers have written. The workflow optimization would be when the majority of requests tended towards alterations of existing feature interactions (as typically happens later in the development process when art production begins to ramp up); at this point the artists would be reusing more of the existing feature nodes and would need to make trips down the programmers less often. The blocking artist-programmer interaction would only occur when an artist needs a completely new feature not currently supported by the engine.
By allowing the artist to experiment directly with shaders, instead of interacting with and waiting for the programmer, the impact to his workflow is minimal, which should translate to an overall minimization of the development time of a new effect; given the amount of the content generation required for modern game development, this can be a huge time saver. Of course, it should be clear such a tool benefits artists much more than programmers. Programmers would be getting the benefit of a more streamlined workflow (due to less frequent requests for shader development), of course, but they aren’t neccessarily getting any kind of improvements to the way they write shader code. But that shouldn’t be too big of a deal, since programmers are comfortable writing code and can do it efficiently.
It’s difficult for me to decide if these sorts of tools have a future. So much of their utility seems to rest on getting the node set correct, which is a delicate balancing act. A poorly constructed node collection can severely limit the usable combinations available, making experimentation with the node graph an interesting toy, but not neccessarily an effective tool for game development. It may very be that these kinds of tools aren’t popular because they just aren’t practical. It might also be that they’re simply hard to get right. What say you, reading public? Have any of used these sorts of tools before, and have positive or negative experiences?
