Code Repository Setup

I have always tempted fate when it has come to backing up my coding projects. Trusting that my computer hard drive would not fail me and that all my precious hours, blood, sweat and tears that I put into the code would not disappear into thin air. However as I grow older (and more synical) I'm begging to realise that eventually this will come back to bite me.

So after months of saying I'll do it and never getting around to it I have now finally uploaded all my development stuff into an online code repository and version control system, namely Google code. Now this is not my first attempt I actually created the account and repository over 2 years ago but I never actually uploaded any code to it (Which makes it pretty useless).

This time however some code has finally made it into Google Code, you can find my repository here http://code.google.com/p/bsx11mt/ if you're interested in having a look as per Google Code's requirements the code is completely free and open source.

So far I'm liking the version control for my game development stuff, the ability to trace out the development of indiviudal features and files is really interesting. You can see how stuff evolves over time or at what point it all started to go wrong.

I highly recommend Google Code and Tortoise SVN for any individual or small team of developers out there who are looking for something a little better than swapping and reserving files.

Assimp Loading Library

The Open Asset Import Library, or Assimp for short, is a portable Open Source library to import various well-known 3D model formats in a uniform manner. Written in C++, it is available under a liberal BSD license and there is also a C API as well as bindings to various other languages.

Assimp loads all input model formats into one straightforward data structure for further processing. This feature set is augmented by various post processing tools, including frequently-needed operations such as computing normal and tangent vectors.

I think the strength in the use of the Assimp library is that you don't limit your game meshes to be created in just a single format or program. You might have an artist who likes 3DS Max for making structures and buildings, while another prefers Blender for producing humanoid characters with animations. With Assimp you can accept all file formats but still have a consistent from inside your game engine.

I plan to have future posts on how my effort to integrate this into my Brainstorm engine goes. In the mean time you can download the Assimp library from here: http://assimp.sourceforge.net/main_downloads.html

PART II: Terrain Rendering with a Scene Graph

In the previous post we discussed some of the issues of Terrain rendering and started to improve and evolve our Terrain rendering system. Part II kicks off right from where Part I left off and it might actually cover some scene graph stuff this time.

So we discovered that a constant number of visibility checks can still be a limiting factor. It's Quad tree to the rescue. Instead of just cutting the grid up into 256 even bits by going across and down 33 verticies we could actually be clever in the way we build the patches.

Starting from the highest level we could divide it into 4 equal child parts which would result in 4 patches that are 257 X 257 vertices. Four patches is not enough so lets divide each of these patches into four more child patches. Now we have 16 patches, still not enough but if we continue to divide down a few more levels eventually we reach 256 patches that we add as leaves to the quad tree. If we keep this dividing information in a tree (Quad Tree to be exact) we can make assumptions that can help us with our visibility issues. Note none of the higher level nodes contain any Terrain information, only the leaves of the Quad tree.

For example because each level of children exists completely inside the bounding volume of the parent by definition we then know that if the parent node fails the visibility test then so will all the children. So no need to bother testing their visibility. Worst case all 256 patches would be visible and we'd still do 341 visibility tests but on average we should achieve about 64 tests instead of original 256. A great improvement on the efficiency of our algorithm for no change in the visual experience of the user.

In the previous version of my Brainstorm game engine this was the complete extent of my Terrain rendering code and it has done the job quite well, however there are still a number of issues with this implementation that I think I can improve on. Specifically there is no handling of level of detail, there is still wasted effort when parent nodes are completely visible inside the viewing frustrum and Tree and Grass rendering don't take any advantage of the terrains Quad Tree, each of the systems uses their own quad tree visibility checking.

Keep an eye out for Part III of this series that will try and cover these issues in more detail.

DirectX 11 Up and Running!

Success! I have managed to set up and compile my very first DirectX 11 App. This is an evolution from my previous game engine and I've tried so hard to get the structure correct from the start.

But funnily enough the more I programmed the more the elusive "correct structure" changed. Never the less I will push on with the hopes to come back and refactor and clean up things later.

I'd normally post a screen shot to celebrate but as it's nothing more than a black window it seems a little pointless at this stage. But don't worry as more interesting stuff starts to happen I will post screenshots.

Next on my list of things to convert over from my old engine is the sky rendering.

My CPU is melting! (well it nearly did)

I was having some issues with my computer last week where it would just restart for no apparent reason. I suspected that there were some temperature issues but I needed to be sure before I started ripping the case apart.

 

A quick run of 3DMark 11 and watching the temperatures inside SpeedFan I quickly worked out why my computer was shutting down. The CPU temperatures were through the roof, SpeedFan listed one of the cores as reaching a maximum of 94degC during the test!

 

Needless to say I shut it down as quickly as possible and gave it a quick cool down before rushing out to buy myself a new CPU cooler before my i7 melted. Lucky for the motherboard protecting the chip I guess. It's forgiven for shutting down and losing me a bunch of code.

 

I settled on the Artic Freezer 13, pictured below. It's nothing super special but heaps better than the Intel stock cooler that I was running.



I originally built this computer at the start of 2010 and it has been super reliable, when I opened the case I found the CPU cooler buried under dust and when I pulled the stock Intel cooler off, I found that the thermal paste had been reduced to thermal dust. Needless to say I don't think it was working too well anymore after 3 years.

The Artic Freezer 13 absolutely dwarfs the stock Intel cooler, it was a bit squeezy installing it amongst all my other components but I managed unscathed.

With the new cooler in place I've since re-run the 3DMark 11 test and I'm delighted to say that temperatures peaked at only a little over 50degC, so I'm chalking that one up to a win and who knows maybe I might dabble in some overclocking now.

PART I: Terrain Rendering using a Scene Graph

Now that I finally have my head around the concept of a Scene Graph the ideas of how best I can use it in my Brainstorm game engine has been keeping me awake at night. Here's hoping that getting my idea out on paper (well blog form actually) lets me sleep. I'm going to start from scratch and explain the evolution of my design.

So how does one represent a terrain in computer graphics? Well there are actually a number of ways, each with their own pros and cons. I'm not about to start a war of philosophies on this subject but to simply tell you that for my implementation I have chosen to use a regular grid heightmap because it suits my design best.

A regular grid heightmap has points that are evenly spaced in 2D with the height of them varying to represent the peaks and troughs of a terrain. In my case I'm using a 513 X 513 grid of heights. Each grid is 1 meter apart, giving a total area of 512m X 512m. Why 513 you might ask? Well it actaully makes it easier to divide up later as smaller grids always need to share a side of vertices to make them match up nicely. You can get to these numbers by taking 2^n + 1 eg. 2^9 + 1 = 513. I'll explain it a bit more later.

Now if we took a brute force approach to rendering this grid we would have 263,169 vertices and 524,288 triangles to attempt to render! Crickey that's a lot especially when if we're a character standing on the terrain we probably aren't actually able to see very much of the terrain at all (could be less than 10%). So that's a lot of wasted effort to render vertices and triangles we can't even see and our frame rate will suffer badly.

So the first step is to divide our large grid up into a series of smaller parts or patches. In my case I've divided it up into 256 individual terrain patches to represent the full 513 X 513 grid. That gives us patches that are 33 X 33 verticies across. This time to render the terrain we can check which patches are visible and only render those ones. It's a much better solution we now have 256 visibility checks and we only draw the terrain that is visible. But we will always have 256 visbility checks even if 3/4 of terrain is clearly behind us and not visible, can we do better?

Old Demo Video

Just reminising about my old game engine and what I managed to achieve. You can go check it out on Youtube, it's a Stargate based demo and the video shows a bit of a walk through the SGC before heading off world to play with the PhysX of wooden boxes.

I'm hoping my new generation game engine will eventually be able to pcik up where this left off and maybe one day turn it into an actual playable demo for people to enjoy.

New Hardware!

Very exciting news I've just ordered myself a new graphics card.
It's a Gigabyte GTX660 OC

If you want to check out the details you can do so here: http://www.gigabyte.com.au/products/product-page.aspx?pid=4361#ov

I'm looking forward to some serious DirectX 11 and PhysX 3.0 work with this once it arrives and I get it installed. I'll keep you up to date and maybe even show off a bit with some benchmarking.

PART I: Visual Studio is your BFF (if you set it up correctly)

I'm sorry I feel so dirty using BFF (Best Friends Forever for those programmers born before 1985) in my title but it seemed like a good idea at the time and I'm yet to come up with a better alternative. Don't worry no more Generation Y lingo I promise.

I wanted to cover in detail the benefits of correct setup of your IDE (that's Integrated Development Environment for those who don't know). It's going to be covered in a couple of parts as there is quite a bit to cover, so keep an eye out for the whole series.

IDE setup is a skill that is so often forgotten to be covered by programming books and even in training courses. It seems gone are the days where as much time is spent learning the compiler as is spent learning the code. i think because it's considered the "boring part" people just want to see graphics on their screen with a click of the compile button.

Programmers just seem to take it for granted that the defaults will be fine or that what a book uses for doing little demos will be sufficient for a full on game until it's too late. I'm convinced that programmers can do it properly! (with a little bit of help)

Now obviously there are a wide choice of IDEs available with a range of different features and costs etc. I'm going to cover specifically Microsoft Visual Studio 2010 in this series of posts as it is my IDE of choice but a lot of the principles I'll cover are relevant to all IDEs it's just a matter of working out how to do it. Disclaimer: I'm not an IDE expert this covers everything I have either worked out myself or learnt the hard way so if other have tips and tricks, that I don't cover, for setting up IDEs I encourage you to leave comments.

If you haven't been lucky (or unlucky depending on your point of view) enough to use the Visual Studio suite of tools before, I think it is well worth a look and if you would like to play along with this series you can download the express versions for free here. It doesn't have all the "bells and whistles" as the full version but for free you do get quite a bit.

So get your IDE fired up and check out Part II - What is an IDE and why do I need it

PART III: Terrain Rendering with a Scene Graph

The first two parts of this series of posts covered the development and evolution of my terrain rendering code from my previous generation Brainstorm game engine. This post will cover in details the current implementations limitations and my proposed ideas for improving the Terrain rendering for the new Brainstorm game engine.

Wasted effort when entire parent is within the viewing frustum. For example a parent which is completely within the viewing frustum contains 16 leaf nodes. despite knowing that the entire parent is visible we still go down the quad tree and conduct another 20 visibility checks before the decision is made to render the 16 leaf nodes.

Alternatively instead of returning a simple Boolean when testing a viewing volume for visibility we could return an enumerated value that could be represent as.

enum Visibility
{
     NOT_VISIBLE;
     PARTIALLY;
     COMPLETELY;
}

From this we could program the visiblity checking code to automatically pass all child nodes as visible and add them to the render list if the parent is determined to be completely visible. Alternatively if it is only partially visible then it should continue to check child visiblility as per normal. I believe that the extra overhead of calculating if a bounding volume is partially or completely visible would be less effort than the wasted visibility checks.

Level of Detail (LoD) considerations. The terrain pixel shader is quite intensive requiring a total of 15 texture look ups and blending operation on a per pixel basis. While this provides great detail when the terrain is up close, the distant patches still use the complicated pixel shader for not much visual benefit ie the terrain is too far away for the player to notice the amazing textures. A single triangle could be covered by just 1 pixel or even less.

So instead of the texture lookup on distant terrain patches the terrain textures can be simplied into a single average RGB colour that is blended, this takes some of the load off the pixel shader without any loss in experience by the player. but no matter how simple we make the pixel shader the amount of effort it takes to process the required vertices is there. Surely there is something we can do to improve that.

As the terrain patches get further away from the player so too do you lose the need for so many verticies to represent the complexity of the terrain, a number of vertices may start to appear as a single point so why not skip a couple of pointless vertices? Well that's exactly what we'll cover in the final part of this series as we tie LoD into our scene graph system to give us a complete Terrain rendering solution.

Scene Graphs: Why Bother?

This was my attitude towards scene graphs for a long while, for the most part what I did in my game engine was very linear using either a queue, vector or list and it seemed to work just fine. I'd add new objects and still my FPS would power along at over 60 until the straw (or maybe more accurately the 'grass') that broke the camels back. You can check my previous post on using a quad tree for grass rendering and keep an eye out for a future post on Terrain rendering using a scene graph.

I'd seen and read all sorts of information on scene graphs and why I needed one, but every article or post I found was so 'wishy washy' (technical term). It seemed nothing more than some fancy buzz word that was flavour of the month. They wouldn't give good details of how best to actually implement one and most gave completely different implementations. The ones that did give implementations seemed so overly slow and complicated and involving so much overhead that I didn't see the point still. So when I eventually ran into trouble with my linear approach I went to quad trees because I could understand those posts and articles and I thought to myself I'm always going to choose quad tree when it comes to the scene graph vs quad trees debate.

But how naiive I was. What I didn't know then that I do know now is that scene graphs and quad trees are not a one or the other decision like a DirectX or OpenGL. Infact a quad tree is a basic type of scene graph I'd been using a scene graph all along and not realised it, but worse than that by being blinkered to the fact I also missed out on other benefits.

But it's actaully more than that because an OctTree is also a type of scene graph, a DAG (Directed Acyclic Graph) is also a scene graph plus countless other possible implementations. Hence the reason why all the different articles could have such different implementations. So infact you can build all of these implementations from a scene graph and best of all that mean's you can design and build all sorts of hybrid graphs to suit your needs perfectly. Sound familiar? it's got inheritance written all over it.

Put simply a scene graph is made up of a series of nodes each of which has links to other nodes (however many you need) and in some cases other information. This other information can range from bounding volumes to meshes and vertices and everything in between. For example if you make every node have axis aligned bounding boxes and must include 4 nodes which divide the current nodes axis aligned bounding box into 4 equal parts bingo! you've got a traditional Quad Tree.

I now have ideas beyond a traditional quad tree for my terrain rendering. I'll still have 4 child nodes just like in a quad tree used for visiblility determination but now I can also have 4 additional nodes for Level of Detail (LoD). So it will actaully be a network of Quad Trees that will allow the rendering of the terrain even more efficiently taking into account both visibility and level of detail at the same time.

So in summary don't be afraid of the concept of a scene graph, you could very well be using one already and not realise it just like me. If you're not using one then jump in the deep end and have a go and keep tuned for future posts on my Scene Graph design, implementation and real world benefits for Terrain rendering. I promise to try not to be so 'Wishy Washy'.

Frank Luna's DirectX 11 Book Review

Once again Frank is back with his 4th installment of the "Introduction to 3D Game Programming with..." series and this time it dives into DirectX 11. The accompanying website d3dcoder.net stays the same for this 4th book and there you can find the details of all the books, accompaying code, solutions to selected problems and the forum.

I was actually lucky enough to get a sneak preview of some of the chapters from this book and do some early testing of the source code before it was released. Which for a series that I enjoy so much was a bit of a thrill and Frank even gave me a little mention in the thanks in the book (aww shucks Frank!)

But what do I actually think about the book? Personally I'm a big fan of this series and I own a copy of each of the 4 books. It slots in very nicely with the relaxed but informative style of the other three books and if you've read any of the others in the series you'll instantly recognise it as a "Frank Luna book".

DirectX 11 is again quite a change from its predeccesor so there are some new chapters to reflect it in this book especailly around the areas of the additional pipeline stages (Compute and Tesselation stages) and the new effects requirements. And I think given the complexity of the material associated with these new features frank does a good job of providing an introduction but don't expect to be able to implement top of the world effects with these the book is afterall an introduction.

The other big change is to do with DirectX's move away from using the D3DX utility library, no longer can you take advantage of the math and texture loading functions that we're all use to instead we now have XNAMath (actually changed names again to DirectXMath with the new Windows SDK). It takes a  little getting use to but there are some good benefits also.

The book is all about getting your appetite for learning game programming going and the basics are good enough to provide you a platform to go on and extend your knowledge if you want to. The demos aren't ground breaking but again they're a start and the rest is up to you.

You don't need to own all the other books in the series, there is more than enough information to pick up this new book and run with it, however I think straight out of the box a bit more effort has to be put in to get everything set up to compile and run the code. Believe me it's very possible (as I've run all of the samples) but don't just expect to double click on the solution file, click compile and run. I think at the very least you need to have some C++ skills under your belt the rest you can learn from this book.

The d3dcoder forum complements the books perfectly and is fantastic. Lots of programmers there (including me) always chip in with ideas and help and you'll even find Frank there answering questions when he gets a chance. So you should drop in and say hi, even if you don't own the books.

So do yourself a favour if you're looking at getting into game programming for the first time or you're keen to upgrade some previous work into DirectX 11 then this is well worth a look.

Back from the dead! Well sort of

Oh how time flies, it's been soooo long since I've posted anything and I've been getting itchy programming fingers (and blogging finges for that matter). And so much has changed there is a new version of Windows, a new version of visual studio and even a new version of DirectX (well not quite but the SDK is now built into the Windows 8 SDK and there are some changes).
Unfortunately life can just take priority over any programming sometimes and I haven't been able to spend as much time here or programming as I would have liked.

But the long break away has given me a new appreciation for wanting to get back into game programming again and what better time to once again wipe the slate clean and start my project from scratch. Well not completely from scratch a lot of the work I had complete will still be able to be used in some way shape or form but I will be harsh and only include good coding and rewrite anything that is messy or ugly or anything that i've always wanted to fix.

This time my priority is in improving design from the base level and increasing the priotirty of planning and initial setup. You know that whole "if you fail to plan, you plan to fail" stuff. Improve my design to be as API independant as possible and improve my ability to update individual components easier going forward so I spend less time doing complete from scratch changes when an API change happens.

It's not an easy task but I think I've worked out that I get the most enjoyment out of finding clever solutions to these problems and writing a useable engine that others can enjoy rather than neccesarily writing a game for others to play.

Keep tuned for hoepfully some more regular blogging of where I'm up to on this new game engine journey