That topic is how we’re not going to use IDisposable any longer.

More specifically, we’re not going to use it as much. Almost every SlimDX 1 object implements IDisposable, because IDisposable is all about cleaning up unmanaged resources and SlimDX objects are, for the most part, exactly that. But the sort of resources IDisposable is geared towards are those with very explicit lifetimes: you create it, do your thing, and then Dispose() of it. C# using-blocks allow you to implement this exact pattern for locally-held objects quite efficiently.

Long-time readers will recall that our IDisposable wrappers around COM objects have caused us a lot of trouble in the past, largely because the interface’s contract does not really jive with COM’s reference counting. To account for this, we implemented an object table based loosely on the same principles that the .NET RCW uses — specifically, we tried to maintain exactly one COM reference to any object known to SlimDX so that we could do the correct thing when Dispose() was called. This eventually led us to an overly-complicated system burdened with multiple creation code paths and inconsistent ownership semantics.

Our solution is to, essentially, side-step the issue entirely and just expose the reference counting directly. All SlimDX 2 objects that are backed by actual COM interfaces implement IComObject, which provides the familiar-looking methods AddReference() and Release(). For the trivial scenarios where you just want quick, scoped access to a COM object, we’ll be providing a simple IDisposable utility wrapper you can use in conjunction with a using-block to automatically drop the reference count when appropriate.

While it first might seem like this change is a step backwards, we think that it’s an improvement in the long run: it obviates the need for the exceedingly complex object ownership rules that existed with SlimDX 1 (which, we suspect, most of our users were getting wrong). Plus, it makes the SlimDX ownership semantics match those of the already-familiar native API.

I particularly enjoyed this post. Back in 2007, when SlimDX was a young API, we had the luxury of having very few serious users. When we realized we’d screwed something up, we just changed it, and it didn’t matter how much that change would impact a client code. There were no clients.

But now, three years later, that’s no longer the case. We have quite a few users; SlimDX has been used in commercially released games and applications, and helps power internal content production tools used by professional studios like Zipper Interactive. We no longer have the luxury of implementing sweeping changes on a whim, no matter how correct they may be. We have practical realities to consider: providing sufficient advance warning and documentation. Developing upgrade paths. Transitioning deprecated versions of the API to “support mode” for an appropriate period of time.

I believe that breaking changes are for the best when they actually improve the overall health of the API. But they’re also pretty scary.

The primary motivation for organizing the library in this fashion was to eliminate the problems with building “Any CPU” applications using SlimDX.

Managed applications built in “Any CPU” mode are essentially asking the CLR to load them as processes most-appropriate to the host machine: if you’re running on the x64 architecture, you get a 64-bit process. On x86, you’d get a 32-bit process.

For native code, however, the target architecture needs to be decided at compile-time, which is why there are distinct 32-bit and 64-bit versions of the SlimDX assembly. Unfortunately, you cannot load a 32-bit DLL into a 64-bit process.

If you’re using the copies of SlimDX that our installer places in the GAC, you can avoid running afoul of this problem by editing your .csproj. If you’re a user who prefers to reference the SlimDX DLLs themselves, you are out of luck. You have to pick a specific version to reference, and you have to set the machine type of your project to match.

By using a two-part distribution, we can ship a public assembly configured for “Any CPU” use. This assembly can detect whether 32- or 64-bit code would be more appropriate and load the correct private implementation DLL into your process.

This approach also allows us to write more of SlimDX in C#, which is far more pleasant language to work with than C++/CLI, and has a much better toolchain.

It also lets us improve the health of the code we do need to write in C++/CLI. Because of a peculiarity in how templates (not generics) interact with managed types, the presence of a template type anywhere in a class hierarchy results in a complete lack of IntelliSense support for that class hierarchy… but there’s a lot of redundant boilerplate code in SlimDX, exactly the kind of code one would like to use templates to write just once. We had to compromise and use macros to provide that code for SlimDX 1.

However, with the split-assembly approach — along with with our adoption of interfaces — the classes “polluted” by templates exist entirely in the private assembly and are never exposed by their concrete types; the lack of IntelliSense becomes a non-issue.

There is one small downside, unfortunately. Since the implementation DLL is loaded manually, clients who reference and ship private copies of SlimDX (that is, anyone not using the GAC version) will need to add or link the SlimDX implementation DLLs (and PDBs) in their .csproj files in order to allow the build system to copy those files into the directory of the built application.

The first is the question of how one initially obtains an interface. SlimDX 1 uses the constructors of its concrete objects to enable users to obtain new instances. Thus, in SlimDX 1 you would obtain a texture instance by doing new Texture2D( device, … ). This constructor will invoke the native ID3D10Device::CreateTexture(). In SlimDX 2, we’ll present a model that mirrors the native interface more closely, with an IDevice interface that demands a CreateTexture method.

To handle bootstrapping — obtaining that first interface — we’ll expose so-called “API objects” that will represent each of the primary subsystems SlimDX exposes to native code (for example, there will be a Direct3D10Api object that will let you obtain the initial D3D10 device interface. These API objects will also be a convenient location to present many of the native free functions. SlimDX doesn’t expose all of these now, and the ones it does provide are often tucked away, seldom-seen static members of one class or another.

The other half of the problem is translating those interfaces back into concrete, usable objects when they are passed back to SlimDX for one reason or another.

We could simply downcast the interface to the concrete type, but that relies on the assumption that such a cast would always succeed — and that isn’t true in general. SlimDX has always supported interoperation with other DirectX-wrapping managed libraries, and to make the assumption that our interfaces were only ever implemented by our own library would work against that.

Fortunately, as a result of that interoperability feature, we’ve had to have all of our objects expose their IUnknown or similar native pointer (wrapped up in a System.IntPtr). Thus, we can rely on an accessor for that pointer being available from the interfaces, and we can use that to obtain the actual interface we’re interested in working with in the SlimDX code.

In the future, our intention is to create a firm split between official and community samples. Only the former will be checked into source control. The community samples will be hosted on our website somewhere, and when warranted we will fold appropriate community samples into the official distribution, after appropriate modification. One of the things we want to ensure is that the official samples have a uniform style (to make them easy to follow and compare) and that they meet certain quality criteria.

The other thing we’re going to do is refactor the sample framework itself. The current framework feels like a “mini-XNA,” which has caused some users to adopt it as a more generalized “game” framework, which is problematic for us. We do not want to be in the business of writing — more importantly, of maintaining — anything resembling an engine. There are other projects out there that fit that bill; SlimDX is a lightweight wrapper, and we want to preserve its minimalist ideals even across aspects of the project that are not the core library itself.

The sample framework should be about making it easy for us to build samples. The native SDK uses a pretty robust sample framework, and we’ve seen that hamper it as an educational tool. When the sample is trying to demonstrate some concept or technique, it should demonstrate that in the language of the API itself. API calls should be used as often as is reasonable in order to demonstrate ‘the simplest thing that could work,’ which is usually what a user is looking at the sample code for.

Basically, we want a relative lack of abstraction. Users are just going to peel away the abstraction layers anyway, so we’d just be creating more work for them.

It’s still going to be a framework, though! We’re still going to wrap the creation of the primary window and the ‘game loop,’ and we’ll provide extensibility points allowing a sample to customize the main phases of application execution: pre-initialization configuration, initialization and resource load, logic update, render update, and shutdown/cleanup. Simplifying such boilerplate code, code that is not directly related to the primary tasks one would use SlimDX for, is perfectly acceptable.

Similarly, we’ll probably wrap mesh loading (especially since there is no built-in method for creating ID3DX10Mesh objects from a file, and we’d like to use the same source assets when possible for homogeneity) and similar ancillary operations.

UI is another interesting issue. The native sample framework supports a simple collection of UI controls for tweaking parameters for a sample. I’d rather not implement (and thus maintain) a similar system in SlimDX, but the alternative of using Windows Forms controls means that tweaking won’t be available in samples running in fullscreen mode, which is a deal-breaker. So we’ll support a small set of basic controls useful for tweaking parameter values in the new framework as well.

These framework changes probably won’t make it into the next release (if indeed the next release is this month, as we suspect), but you should see them in the release immediately following that at the latest… as well as some other interesting changes. And of course, you can always live on the bleeding edge and work from the head of our repository.

In other SlimDX news, Arcen Games, LLC recently released AI War: Fleet Command. I’ve posted about this on Twitter already, and SlimDX lead Promit Roy has beaten me to the punch on a blog post, but I’ll repeat it here for the benefit of other readers: AI War is the first game available for purchase that was built using SlimDX! This is very cool. Developer Chris Park has written about his experience evaluating rendering solutions for C# projects, explaining how he ultimately chose SlimDX… head over and give his blog a read, and when you’re done, be sure to check out the game itself!

Now that that’s out of the way, let’s turn to the real issue: how to build engines, and more specifically, if you should even bother.

For any number of reasons, many neophyte game developers (and even some moderately experienced ones) seem to think that a game engine is a required, critical part of a game. They think that — in keeping with the mechanical analogy from which the term “engine” comes from in the first place — like a car, a game with no engine will simply not run. But that mechanical analogy starts to break down when you take it this far, because when you look at the reality of the situation, a game engine is about reusable components. Much like you can build a car with custom parts or generic ones, you can build a game with custom components or reusable ones. The fact that you must have an engine to build a non-trivial game is a fallacy, something perpetuated largely by people who don’t know any better.

Nonetheless, every so often comes the obligatory post on GDNet by somebody who has decided its time for them to write their engine, so they can start making some real games. These posts usually consist of requests for resources about the engine design and development process, and in fact chances are that if you’re reading this, it’s because I caught you making just such a post and linked you here so that I could crush your dreams with a minimum of effort on my part.

So my advice to you, if you’re trying to write an engine, is: Don’t. No matter what your reasons are — it doesn’t matter if you’re writing an engine so you can write your dream game, or if you’re writing an engine because you think it will be a good learning experience, or any number of similar reasons. They’re all wastes of time. You can sit down and write a game without writing a pre-written engine, and in fact this is very often the better approach, regardless of why you want to write an engine. The entire development process goes much more smoothly if you are focused on writing a game instead: a game is much easier to identify requirements for, much narrower in focused, much more rewarding when finished, and much, much more useful.

Most hobby developers who “finish” an “engine” that was designed and built in isolation (with the goal of having an engine, not a game, upon completion) can’t ever actually use it, and neither can anybody else. Since the project didn’t have any goals, any boundaries, or any tangible applications, it essentially attempted to solve aspect of the chosen problem space and consequently failed miserably. These “engines” tend to be little more than an inconsistent and messy amalgamation of random functionality that isn’t terribly practical, robust or reusable. Furthermore, the actual viewable end product of these “engines” tend to be a cute little map (loaded from one of the popular BSP formats) you can run around in and observe some cute little models (also loaded from a popular model format) rendered with some fairly basic lighting effects (maybe some per-pixel normal mapping or parallax mapping). Maybe you might even see some collision detection or animation. These projects are just tech demos, and not particularly compelling ones at that. There is nothing to them, and they feel rather flimsy and inflexible.

The solution, even if you really want to make an engine, is to make a game instead. I can hear you crying out in protest from here, but just bear with me for a minute. The game does not have to be an epic production. It does not have to be GTA5, Quake 17, the next Elder Scrolls game, or a WoW-killer. It just has to be a game, with well-defined (and ideally well-thought-out) gameplay and well-defined developmental scope. A game is much easier to identify requirements for, as I’ve already mentioned, and it is also a practical application of those requirements. Once you have made one game, make another. Then another. Each time you start a new project, you should identify functionality that could be used and pull it out into a common library of base code. You’ll probably have to refactor some of your code to remove explicit dependencies on other code or data that is game specific, but this is a good thing. It will help you generalize what is generalizable in a way that you can still test the generalized functionality against your finished game and confirm that it still works (obviously you might have to modify some game-specific code to adapt to the increased generality, as well).

If you repeat this process long enough, after a few games you’ll have the beginnings of a solid collection of reusable functionality that has been proven to have practical applications. You’ll also have a set of far more interesting demonstration applications that could also double as test harnesses for your engine. This method of growing an engine (rather than manufacturing it from whole cloth) is superior because it helps you shape your goals, it forces you to actually think about the problems you will face and their practical implementations, it forces you to think about the separation between the domain-specific logic of the engine and the game. It makes you learn from your mistakes rather than pretend that mistakes cannot be made.

Whatever the reason, the programmer will often decide that the solution is a complete re-write of her code base. “This time,” she says, “I’ll get the design right the first time! My new code will be better, cleaner, more robust, more generic, and overall more powerful and less buggy!”

Unfortunately, rewriting your code from scratch is very often a bad idea. A developer who is uncomfortable with the state of her code should refactor it, not rewrite it. The key difference between the two is that refactoring is about making many, usually small, iterative changes to the existing code in order to make the old implementation conform to a new design. Ideally, one would preserve the ability for the code to compile, run, and/or be consumed by client code as much as possible, within reason (adopting a new design may, of course, involve breaking the public interface of the subsystem).

Why is refactoring better? Refactoring actually forces you to think about the problems. It forces you to look at the current design, the current problems you have with that design, and what you could do to address those problems. When you don’t think about the problem, and choose instead to nuke the entire subsystem from orbit and rebuild from scratch, you’re probably just going to make the same mistakes again; you’re rewrite will most likely be superficially different from the original (for example, you might have more consistent or different looking naming conventions), but nothing more. Refactoring allows you to learn from your mistakes. Rewriting just blissfully ignores your mistakes, all but ensuring history will repeat itself.

Refactoring is an important skill to master. In the real world, full-on rewrites of existing subsystems are almost never cost-effective, so refactoring ends up being the only real sane option.