Renderer - Materials

I'm about to start my final year at university, and this year the largest module is our individual project. We get to choose to do whatever we want for this project, provided that it is a non-trivial programming problem; so I am going to take this as an opportunity to write the deferred renderer I have been planning for some time - and have it contribute towards my degree. I'm pretty exited about writing this, but unfortunately a university project and a long report are synonymous.

I hate writing reports. It means I have to write notes on what I'm doing.. thats almost like writing documentation! No matter how well my intentions are to document what I'm doing, when I finish a module or piece of functionality, I want to get right on into the next one. Writing it all up seems such a chore.

So, thinking of ways to make this bit a little more interesting, I thought I could keep a project log on this blog. I've learned probably most of what I know about game development from reading various blogs, so it feels good to give a little back to the community.

Materials
Everything you draw in XNA (and the last couple versions of DirectX) has to be drawn with a shader. These shaders contain code which executes on the graphics card, which ultimately calculates the colour of each rasterised pixel.

Shaders usually require a set of parameters. Some of these describe properties of the material being rendered; for example the diffuse texture to describe the colour. Others are information about the object being rendered, like the world transform matrix, or camera view matrix.

The XNA framework represents shaders with the Effect class. By default, each effect in a model is pre-loaded with the material parameters for the model. However, the rest need to be set by the game as things are drawn. This is fine if you always know exactly what shader parameters need to be set every time you use an effect, but it means you have to effectively hard-code what parameters are available to a particular model when it is drawn.

What I really wanted was some way of data binding information which the game is aware of, into whatever parameters the shader wants. Conveniently, HLSL provides two ways of adding metadata to parameters. Semantics, and Annotations. There is already a standard for using annotations to bind values to parameters, called Standard Annotations and Semantics (SAS). However, SAS is only really used in shader development programs like FXComposer and RenderMonkey. The annotations in SAS are a little verbose, and don't include many of the bindings which I'm likely to want to use in my renderer. Following this standard doesn't really make sense, then. Instead, I'm just going to use semantics, which are simple strings attached to the parameters. The engine can look at these semantics, and lookup the corresponding value to assign from a dictionary.

There are a couple of implementation difficulties with this system. Firstly, the Compact Framework running on the  Xbox 360 has a crap (non-generational) garbage collector. That means that using a Dictionary to store metadata in the game is not a good idea, as most data which will go into it will be value types (such as matrices), and so will be boxed. Allocating during gameplay on the compact framework will cause the GC to run, which can cause the game to stutter. Instead of using a dictionary of objects, I used a dictionary of my own generic box class. This means that the values are manually boxed the first time something is assigned to that specific key, but from then on the box is re-used by simply replacing its internal value. This also allows shaders to cache the box, thereby avoiding the dictionary lookup.

The 2nd difficulty is that setting the value of a shader parameter can only be done through the SetValue methods. There is an overload of this method for each type which a shader parameter can store. The problem then is in calling the correct overload. We could look at the type of value we are assigning, and then use reflection to find the correct method. But calling methods via reflection is far too slow. Instead, I wrote an adapter class for each overload, and the shader instantiates the correct adaptor when it is constructed. This is not neat, but it works.

With this in place, I can now draw things with any arbitrary shader, and the shader will automatically go out and grab whatever data it needs from the engine, without me needing to know at the time of writing what that data will be.