Comparing RuneEngine to Unity3D - Test #1

Last night I performed my first official render comparison test with Unity3D's latest released version.  Unity 4.0 may be a significantly different story, but I use what I have access to for now.  I was unable to test against Unity Pro version which supports static instancing which may make a significant difference in the results, but here is my test case and what was found:

Test Case --
20,000 Cubes assigned to Diffuse texture
500 Point lights
0 moving objects
Deferred Rendering configured.
720x480 resolution
Physics Disabled
Input Disabled

Unity3D 3.5.6f4 --
Editor Crashed
Debug build Crashed
Release build Runs
FPS in Debug: n/a
FPS in Release: ~3-4

RuneEngine v2 --
Editor n/a
Debug Build Runs
Release Build Runs
FPS in Debug: not tested due to Unity3D results
FPS in Release: ~135-140 as Active Window ~35-40 as Inactive Window

Further tested to determine threshold for 30FPS with 500 deferred lights found that this can support up to 400,000 diffuse textured cubes in conjunction with 500 deferred lights non-moving while remaining above 30FPS.

Ok, a lot of abstract information.  Well, for those who are not aware 28+ is a solid framerate to avoid jerkiness, but some video formats only push 24/25 FPS and remains unnoticed.  Therefore RuneEngine is fully capable of pushing non-jerky frames in this test case and well beyond the specific test case while Unity is not.

My analysis from these results is as follows, the new Update Systems and the Rendering Pipeline are optimal, though are still receiving some improvements.  However, Unity3d is likely to have some overhead that RuneEngine at this time has not accounted for.  Such as the Quad tree sorting.  In the basic scene all objects are rendered, but Unity would still be processing this sort.  Depending on the nature of how they do this that could be the cause of what is seen.  However, I must note that prior to the new Update System in RuneEngine my framerate results were rather similar to what is seen in Unity3D suggesting that this optimization may be the sole reason for the substantial performance difference.

Time will tell if this landslide performance difference continues to be so, but at the moment RuneEngine outperforms Unity3D by ~45-46 times...

I intend to do some further testing with this as time progresses, as well as confirming the performance of Unity3D Professional in comparison.  For now this is a pretty big success for RuneEngine I believe.

RuneEngine V2's Flow System

I thought it would be kind of me to write a larger post about what this system is, how it will be used, and why it is a crucial part of the new RuneEngine V2 feature list.

If you're at all familiar with DirectShow basically imagine that, but re purposed for something... different.  For those who don't know, essentially we're working in Blocks that I call the IFlowBlock interface.  A block defines a procedure that handles a certain type of data.  This part is somewhat different from what DirectShow does.  The data is generic, where in DirectShow it's always a type of media passed by a buffer of bytes.  RuneEngine's Flow system can pass more or less whatever you see fit.  There are just a few rules. So, in DirectShow a sample or data is passed downstream all the way to the end, in every case.  Well... Flow is not going to do the same necessarily.  It can, but generally speaking the end block is going to be a renderer or finalizing state, so it should only be hit when the rest is complete.  Flow data starts upstream and passes downstream until it hits a breaking condition.  Data can be flagged for passthrough transit meaning, it's going all the way to the end or to a certain point before processing itself.

I could go on and on about how data moves and the different possibilities, but the key thing to understand about Flow vs DirectShow.  DirectShow has rules, while Flow is more user defined.  The both have Pins and "Filters" or "Blocks", even "Graphs", but ultimately they are completely different beasts.

Flow is not designed for processing video/image data, it is designed for processing some data in a specified sequence using a system of interchangeable blocks.  Within the upcoming particle system this will be used by the ParticleController.

The blocks though will control a state of a particle.  A particle graph may look rather complicated, but it houses some insane possibilities compared to other particle systems.  And it does this without creating 20 emitters and controllers.  Below is an example of what a simple particle system graph may look like:

This may not be the final example, but in this the idea is despite the appearance, when in one of the velocity blocks the object actually does not receive wind, it just uses velocity to update.  Wind control happens after the velocity state is complete.  So, for a certain time, a particle will be moving in a specified velocity, when reaching it's final output, it moves to wind control.  Particles internally hold a velocity value, so the old velocity should be there still.  Wind control is going to manipulate this.  So, you would see the particle slow down and react to the wind.  Imagine and explosion where the particles move out quickly then get carried away in the wind after.  That is what this somewhat the idea of what this effect would accomplish.

This demonstrates that this system is more sequence driven.  One block does not start til the following is complete, though that is not a forced behavior.  It was the design.  However, particles are small instances so the overall will almost seem like streaming update, it's not exactly.  Now, in theory could you create a loop back effect to handling particles, without changing the underlying system?  Absolutely!!! You could easily create a block that has a loop back output pin which send's items back to another block if the parameter to pass forward has not been met.

Given that this system is newish, and purpose is different than similar systems these are only speculations of ideas.  The actual implementation and design of graphs will come after some testing.  If I find some cool tricks to do with the FlowGraphs they will likely get shared as examples. =)

That's all I got for now though.

New API objects for RuneEngine V2

Ok, the title of this post is almost misleading, this is more about the correction of some features that were slightly flawed in their design which have now been corrected.  So I'm going to talk a little about it.

RuneEngine V2 has had a feature since the beginning referred to as a content reference system.  The idea and purpose of this system is to be able to dynamically load content and handle such implementation as transparently as possible.  Sadly, it only truly came to my attention last night it was not serving it's full purpose.  It wasn't exactly wrong.. but it was defeating it's purpose in existing with the existing implementation.

Now some MAJOR changes were made, everything from the outside looks almost the same, except it is a little easier to use now.  I should be taking that another notch further, but that's a completely different talk.  So, what was changed?  Here's a basic list from the far back perspective:

Removed--
IMaterialReference
IPostShaderReference
MaterialReferenceObject
PostShaderReferenceObject
MultimapTextureReferenceObject (rebuilt to interact with the TextureReferenceObject instead of separate)

Added--
IShaderReference (new material reference, but also serves as post shader reference)
RenderTargetReferenceObject( it was missing )
IContentContainer (this guy... does magic)
ShaderReferenceObject

Ok, that's about it.  As you can see it's a pretty even exchange, but the added were completely missing or are more of a rename.  The multimap texture reference and texture reference objects were completely unnecessary to be separate, I may have to do some wiggling to make it work, but it is done.  Material references and post shading references were extremely redundant hence the spark to make these changes.  In the process is how I found the issues.

IContentContainer was added to resolve the biggest of the issues as well as add a few functionalities that really make this feature sing.  The issue was, the reference objects were calling load methods on instantiation in most cases, and if they didn't.  The components were.  Why not use a normal load call if I'm not going to use these for what they were added for right?  So, first I stripped all calls that forced loading of the content references, then looked closely at what was missing.  IContentContainer was what I came up with.  This interface defines a collection of callbacks that the IContentReference objects call when they are loaded or unloaded.  It has a method for making sure the reference is associated.  That sort of thing.

One interface all it really took?  Not a chance! Now, the IContentReference interface also has some added methods for working with the IContentContainer.  Also, RuneContentManager was extended and improved to work more closely with this new interface as well.

So, rather than being super abstract about what happens let me just explain a few scenarios and how these changes have impacted them.

Scenario #1:
I'm building a game where the scene is rather simple, but the Content takes a long time to load.  I do not want to make a super long load time when a level is loaded.  But, I'm afraid to preload the content bogging down the CPU in the main menu.  In this case I can preload the Scene files but not the content.  Once the level is selected we can prioritize required content loading it first allowing the level to start with partial content and potentially even a partial scene.  The reference objects allow this separation of load times, but not in a way that is forced by the engine.  You can load it whenever you please.

Scenario #2:
I have a large collection of global resources that need to be loaded.  I can put them in a global RuneContentManager and link it to my SceneGraphs allow the components to search through it's content as well.  I can load all these global content items at startup, background or load screen.  And pull them using the RuneContentManager linking systems.  The amazing part here is that the contentReferences will manage unloading and reloading of those assets individually if they're not really being used.  Also, the ContentReferences will pull more directly from the contentManagers now than before.

Scenario #3:
I'm building an Elder Scrolls Clone.... GG!  The newest implementations actually allow full user control over when content is loaded and the IContentReference objects do all the tricky stuff.  You can load a resource in anyway RuneEngine allows and the components and the IContentReferences will receive that callback letting all places that need to know, know that the resource is available and can be used now.  No periodic or frame by frame update required.  All of this leading to a fully seamless platform capability in a very simple way at that.

TextureContentComponent.AssetName = "assetName";
RuneContentManager.Load( "assetName" );

Done.. This like much of RuneEngine, is order independent.  RuneEngine is unique in this element that I've spent a lot of time working around pre-requisite type issues.  The game will not fail when resources are missing it'll just respond to it internally or with user defined code.  Components can be added in any order.  SceneNodes can be activated before or after components are added, it doesn't matter.  The only real prequisite to any of RuneEngine is this line:

RuneManagementService.Startup();

I'm extremely enthusiastic lately watching all of these architectural designs in the engine being realized. So pardon me if I seem ranty.

Anyways, happy coding.

Thoughts after the RuneEngine V2 Code Review

Last night I spent a lot of time going through the RuneEngine V2 source code, updating documentation and reviewing the source for poor organization and garbage code left behind.  This is a good practice to act on with some regularity, but isn't always necessary.

Going through this I had quite a few thoughts.  Absolutely none of them involved considering change.  I found that for 210 code files currently they all seem to be in pretty good order and have great readability.  But that is aside from the point.  The real point is an issue of mentality gained from reviewing the source.

I have spent a lot of time over the past few months procrastinating about not working on the engine.  I am getting heavy back in development with it so I decided a cleanup pass was in my best interest.  I had been procrastinating because of "how much is left."  Well, this review showed me quite a it in the direction of how inaccurate that really is.  There are still plenty of unfinished areas and new features that need implementation, but reminding myself that there are systems and features already implemented that push into professional grade quality affected my opinion of where I am at.  It would appear that by the end of the week I could have any loose ends taken care of and begin development of new features next week.  By new features I specifically mean building the editor.

Now, I've spoken here about the goals of the engine as far as features, but here's a list of what I found that gives me such a confident outlook on the status.

- Deferred Rendering
- Post Processing System
- Dynamic Content system
- Custom material format
- User defined geometry system capable of working to the extent of a simple modeling utility
- Extendable file IO system
- Extendable XNA content resource extensibility (This is small, but huge at the same time, specifically it is    built that shaders with custom structs used can be applied via the material file format with some extension to this area)
- Input System, with plugin capabilities
- Updateless and per-frame draw call removed system (huge optimization)
- Custom SpriteBatch which renders into a deferred scene with textures that support the multiple channels.
- Architecture in a way that almost every part of the engine is 100% self reliant.
- Low-Level and High-Level implementations for most features

As a foundation, RuneEngine V2 has all the necessary pieces to be something really impressive.  Nearly the entire engine is designed for extendability.  This design is going to make moving forward significantly easier than it was in past iterations.

Very soon, after the loose ends are tied up I will be making a post here and on the RuneEngine V2 facebook page.  This post is going to be a demonstration of development with RuneEngine V2.  I want to showcase, how some of this stuff is compartmentalized and how, as a coder using it, things are going to be done.  In many cases a test project only consists of a few RuneEngine lines of code.  If it's much more then it is due to setting up parameters of objects for a less "defaulted" test.

Anyways, happy coding.

My new Development Pattern and why it works

I’m generally not one in the past who has studied other’s development patterns, so I generally just go with whatever works best for me.  I found not real long ago, that doesn’t really work… So, I did actually put some thought into how I worked on my projects.  This has been heavily exercised in the restructure of RuneEngine V2 over the past year.  The code base is the most organized and the most stable, but also the largest it has ever been.  Therefore, I am confident this is working well for me.  So, I decided to share some details in my new design process towards development.

Ok, I’ve always done this, but I’ve taken it more seriously.  Whether I write it down or just visualize it, or even go ahead and set it up in Visual studio.  The first thing I do is define objects that I need in a given area.  I also ensure that I know an approximation of what members I need.  Like I said, I’ve always done this; this is a pretty standard starting point for planning phases.  

The next part is where I determine which methods are used commonly across the objects and/or if there are methods/properties that may come in handy across other parts of the project.  Then, I place all of these common methods and properties in groups and apply them to interfaces accordingly.  Ideally, the first code I write is at this point.  If you’re not overly fond of interfaces and/or believe that they aren’t a necessary part of development you may want to re-evaluate that statement and review MSDN and observe the inheritance trees of .NET objects.  I would agree they’re not required, but when you practice this way you get a few amazing advantages you may not have thought of.

First of all, one example is extension methods for interfaces.  These are a great addition to an interface driven design pattern.  You can define behaviors with these interfaces and avoid rewriting the same logic in every implemented object.  This is more appropriate than inheritance; because it is flat there is no call stack for this.  So, this does become an optimization later.  One major con is that interface casting is expensive, you can control how much time you spend casting interfaces in many ways, so this is not overly an issue.  An example of a major pro would be extensibility of your code base.  Along with the extension methods, other methods that are geared to accept the interfaces and not objects can act on any object that implements them.  This gives phenomenal ability to add whatever you need to adhering to the rules in the interface, and all of your code base will support it.  This is my purpose in designing this way.  One issue that you run into that is not really a pro or a con is that you do have to define everything, every time.  So, it does take more work to add new objects than deep inheritance trees and abstract objects, however this does promote flatter code overall, so again it runs faster.

As I am designing the interfaces, I put a lot of thought into a few things.  Will it return an error code or throw exception?  Do these methods require specific implementation that is not optional?  Can I confine them into fewer methods?  Should this be publically accessed?  These are good questions to ask because they guide the process into making useful interfaces that are not too tedious and remain useful.  It is easy to make an interface that is a total waste.  The most useful interfaces in RuneEngine V2 are the IContentReference and derived interfaces.  The reason these are so useful is because they define rules that allow the RenderManager to use them without the need of any further information about the content.  IMeshContentReference for example, is used for custom geometry, models, animated models, and will be used for terrain as well.  One interface can handle all types of rendering.  To make it even more useful, this interface can be defined by a 3^rd party and game specific code, and guess what?  Now, that custom rendering logic is available to the RenderManager as well.  

After I do all of this and start rolling with the project and ensuring that my interfaces work in their desired environments and the objects I’m building can get their full behavior from this set up.  I go through again and make sure the interfaces don’t have any methods that across the board aren’t used.  The IMeshContentReference did have a few methods that weren’t being used by the RenderManager or the other objects so they were removed.  The objects that relied on them kept these methods, but the interface does not define them so they became object specific.

The last big key to the puzzle is actually one of the more important, that is done from the beginning, but mentioned last for its importance.  CHUNK IT, do not try to list out every interface and all of its methods from the beginning.  Build in chunks.  This gives you time to focus on each individual area closely.  It also allows you to overtime realize a more appropriate location for existing and new interfaces/classes.  I avoid moving things in too big of a hurry, but I’d rather do it now than later.  The longer you wait the more work it will require to do.  I generally define a chunk as a new feature or system.  Anytime I create a new DLL I tried to handle its major features right off the bat. 

A good example of how I work in chunks is this.  When I was starting the restructure of the content system, I knew I wanted to work in ContentReferences as before, but they were going to become interfaces.  So, that was an immediate.  I also, knew already from prior that the ContentReferences did require some uniqueness so I created the Mesh, Material and Texture reference interfaces.  These were the most important and I believed there was a chance I could confine all my rendering code to use them exclusively (I was right).  I didn’t start working on the implementation of any of the reference object implementations until after I had developed them into the RenderManager and the RuneContentManager objects.  I had to confirm their use and entry, what methods they would require before every implementing a single object.  I did find in implementing the objects that the methods required a little tweaking, but there was little to no change during this stage.

The more general stance on this is as follows, start with a list.  Use the list to define rules in the form of an interface.  Start the implementation at the most system level objects that use the interfaces as parameters.  Ensure the interfaces will work then implement the objects.  Make minor changes if necessary as the objects are implemented.  Only work in chunks of implementation as opposed to the entire code base at once.  If an object doesn’t apply in any of these systems and uses no methods that would be required in other objects, there is no purpose for an interface.  Basically, if you define IContentShaderReference when ContentShaderReference is the one and only object that will use it.  Well get rid of the interface.  IContentReference should be enough.  However, if there is any chance IContentShaderReference could be used in another case.  Yes I would include it.  Only exclude an interface definition if you know there is no chance another object will exist in need of those methods that will be passed into the same methods this object will.  Simpler put, if this object could be singing “I’m the only one” Then you probably don’t need an interface.

Most of this may seem pretty general, but as usual this may help you.  It’s just an approach I’ve been using.  You may want to review your approach and see if you do any of this and whether or not the ideas may benefit you.  If you ever find yourself cussing because you don’t have enough flexibility or you have to write large extensions to your entire code base every time a new object is added.  Then this is a good approach for you.

Happy coding.

Console Game Architecture: Part 1

This is something that I feel is poorly addressed to many students and aspiring game developers.  So, I just thought I would take a little time to write a little about some of the parts that make this work the way it's supposed to.  I am writing it against C# and posting some code snippets to make better sense of it.  But, it's not really a tutorial.

One crucial part to all games is the While loop.  So, in console games many people make this mistake in thinking.  Oh it's text based so I don't need that.  Sorry, you really do.  Granted you can avoid it, but you're only making it harder if you don't use the loop.  That means you need a boolean field which determines if the game is running and a while loop which utilizes that value.  here is the simplest way to approach this.

Nothing hard about that.  Making sure you start here is crucial to saving yourself thousands of headaches later.  Now, another big mistake I see is the lack of use for input handler methods.  I'm going to write a few examples and show you what I mean by this.

This is our global input handler, you use this one to handle input that you want for any and all input manipulation.  Pretty neat concept right?  There are two methods one for a one shot call, and the second which actually handles the input so it can be called by other handlers.







The ones below are for handling the concept of room navigation.  Same concept of the two methods, but notice in the RoomInputHandler method I call GlobalInputhandle.  The nRoomIndex argument is used to update the current room, this handler does all this for you with the room information set up accordingly.  The RoomInfo[] is a simple struct which holds the name and description.  All the rest of this is the same as the global handler.

The coolest part about all this so far.  That's majority of it.  The rest is pretty much whatever you want it to be.  I'm adding a few other parts to this as a series, but this covers 99% of what most students which consult me are missing.

Hope this helps.

P.S. I used screenshots so you can actually write the code yourself and not just copy it on purpose ;-)

State Of RuneEngine V2

This post is being directed here since it is rather large and easier to write in the environment of my blog as opposed to a Facebook post for example.

RuneEngine V2 is currently somewhere near the 50,000 line mark in C# and C++/CLI code.  All Visual Studio 2010 developed.  Most of which is 2008 compliant which I intend to update in order to be 100% 2008 compliant.  RuneEngine is developed against XNA 4.0 and .NET 4 runtimes.  I want to make a few minor changes to allow XNA 3.1 compilation, but I may never do this unless at some point I am asked to do so.  The C++ Native build is in a semi skeletal structure, but has no executable implementation at this time.

Recent Changes-


I have recently updated the assemblies and consolidated them to be more concise and promote stronger modularity. The new list of the core dlls is as follows:

• RuneEngine.dll
• RuneEngine.Content.dll
• RuneEngine.Components.dll
• RuneEngine.Components.Baseline.dll
• RuneEngine.Graphics.dll
• RuneEngine.IO.dll
• RuneEngine.Physics.dll
• RuneEngine.Scene.dll

I have removed the Math, Threading and Sorting dlls.  And moved their contents to the RuneEngine.dll.  These 3 DLLs were being used or intended for use across many parts of the engine and made more sense to be in the core kernel assembly.

RuneEngine.Components.dll and the RuneEngine.Scene.dll were rewritten against C++/CLI in order to improve performance.  This improvement will only apply to Windows Games if I cannot get the new dlls to compile for XBOX 360, though to my understanding this is possible.

RuneEngine.Components.Baseline.dll was added as a C# features dll or test bed for components that will be added.  In this DLL you're going to find core features needed for most to all games.  Such as health information components, event system related components, terrain system, and tile map systems for 2D and 3D.  In the RuneEngine.Components.dll there will be all minimal requirements for a game.  Core implementations for rendering, movement and sorting logic.  Everything here should be used in every game unless your game is strictly 2D or strictly 3D.  A 3D game will generally use every component in this DLL, but it does not HAVE to use the 2D features here.  Also, any base component objects are here and are available for inheritance to expand upon them.

Performance-

So, how is performance looking?  I can get, on my machine, 17,000 moving and rotating textured cubesrendering with a basic lambert lighting algorithm in a deferred rendering environment at 30FPS with ~20% CPU usage.  Machine specs are expected to vary, but this is greatly optimized from comparison to before.  65535 non-moving textured cubes can be rendered at ~300-400 FPS.

I have consolidated my largest bottlenecks in performance to XNA itself.  There are a few areas of overhead from the engine, but primarily they are minimal issues and can be ignored as they can not be improved easily or they already have to a point I have not found methods for further improving them at my current ability.

What's To Come? --

Ok, this is something not being mentioned enough.  What is all of this for and what will it be used for?  So, there has been some mention of a game being built with the engine.  At this time it is scoped to be a 2D side scroller.  If things change on a resources standpoint it will transition to 3D.  The game is still being kept pretty thoroughly underwraps as far as what it's about etc etc.  Once a playable prototype is running there will likely be a more official announcement and video for it.

Is RuneEngine only going to support core engine features?  This is not going to be the end result of RuneEngine V2.  As I develop the working title many of the components will go into a new assembly RuneEngine.Components.Sidescrolling.dll  This dll will contain many components and features necessary for any side scrolling game.  And yes, there will be side scrolling features for 3D games as well regardless of the projects end platform.  

That being said, will RuneEngine V2 only have features for sidescrollers?  I hope no one thought this would be yes.  I actually have plans and some layout for many other features such as this.  Here is a list of component assemblies which are intended to be added as well:

• RuneEngine.Components.Sidescrolling
• RuneEngine.Components.RolePlaying
• RuneEngine.Components.Adventure
• RuneEngine.Components.Strategy
• RuneEngine.Components.FirstPerson

Ok, this list has actually received a lot more thought than it may appear.  I know there is not Action, Shooter, Racing and many other genres.  However, if you pay attention to this list carefully and think about each of these basic subsets and what they may contain.  This list is a compilation of the genres which actually contain substantial unique elements to their genre.  Adventure for example is going to contain components for 3D platformer elements like puzzles and basic action combat.  Features easily extending to make the next hack and slash Action game like God Of War.  RolePlaying is going to contain primarily data components for handling statistics and algorithms for damage and experience.  Adjustable parameters for curve adjustments etc.  Strategy contains elements for unit manipulation and resource gathering.  There will be many unique data components here as well which are simpler than the RPG elements since they need to happen on a larger frequency.

Those reading this are probably starting to wonder, what if I don't want to make a turnbased RPG?  Like I mentioned earlier though this is mainly data and some UI components.  Basic movement can be handled by the baseline components however, if you use the Adventure components for movement and combat and RPG for stats you have the next Action RPG.  A fairly compelling approach to making games like Kingdom Hearts for example.

The Strategy components are not restricted to turn based or real-time either way.  There will be a few unique components for each, but overall we're looking at a pretty broad topic.

First Person should be self explanatory, but if not here it goes; IT WILL NOT EXIST!!!! First Person shooters only have one or two unique factors so they will not justify a unique Assembly.  The first person movement and the mechanics for firing a weapon will be between two separate components in the baseline assembly and can easily be implemented a new for custom behaviors.  If with this development pattern I do find that shooters are difficult to complete with the Engine as it is, I will add a Shooter components assembly, but it is not intended for initial features.

Time Line--

No, I'm not going to wait for all the above to be completed before making potential releases.  However, the working title game will be released first.  There may be some alpha/beta release builds of the engine though.

The components DLLs mentioned above will be built in the same order they were written above.  Which is funny since the next title I have in mind after the current one is a RTS.  It will pull from RolePlaying and Adventure features so they will come first.  Also, RolePlaying may be used for the current working title though that is undecided as even if its features are used the working title will have no visible exterior of a RPG.

At this time I do not have the means to put official dates or even presumable dates on any of this, but I did want to give a little scope as to how things are progressing and where things will progress towards.

Thanks,

Designing RuneEngine V2

This is something I’ve been meaning to discuss wider scale, but haven’t taken the time to do so. This post is going to be all about the ideas and design theories I am using as features are added to RuneEngine V2. I am also going to do my best to explain why these theories are proving to work well and how they will improve experience for use of the engine later when it is more completed.


Interfaces and Polymorphism

One of the biggest concepts that have stuck hard in the new structure of the engine is strong use of Polymorphing topics and interfaces. This is extremely important for users who would plan to extend the engine and add objects without having to write crazy wrappers around my rendering and content systems. Almost every object in the engine implements an interface, and almost all of the internal systems use the interfaces and NOT the objects directly to handle core features. For example, all content reference objects are interface driven and apply specific rules that the rendering pipeline requires. So, the rendering pipeline only requires the interface and proper implementation. Therefore, this can be extended very easily so long as the interfaces are implemented correctly. The most recent IO systems are nearly 100% interface driven, and probably show the extensibility of RuneEngine at it’s peak. The physics system exposes no objects, only interfaces and attaches to physics engine wrappers, so to add a new custom engine is trivial(well as far as making it work with RuneEngine goes).

As you can see those are some major area’s and they are very simple to plug into the existing API. This makes my life especially easier, but other’s using the engine could gain the same benefit as these interfaces are not internal and can be used by anyone.

Key reasons for the high focus here are obviously for extensibility and generic behaviors. This benefits mostly programmers, but that high benefit could allow a team to do much more with RuneEngine than what I allow them to.

Internal Code Cleanup/Optimization

This is a huge reason I started restructuring the engine. My goals for what I wanted it to be changed a few times during the first structure so the internals were getting haggard and the calling conventions were inconsistent.

So what are the new guidelines and why? First a little history of what it kept shaping around originally. When I first started the engine it was designed to be built like C++, well I took that a little too seriously. So, I backed off that and went way the other direction for a while. The object oriented model has always been very important through development of this, but some pieces were being written poorly, and as the ideas changed the legacy code was not getting removed. This made the code base fairly hard to work in as there was a mix of all sorts of styles. Before I started the restructure I gained an even better handle over C++ development and stay well practiced with it building many prototypes first in C++. The C++ iteration was started for RuneEngine. This is when I saw how poorly they were going to correlate. Some parts would work fine while others would need full rewrites. Therefore, the engine began restructuring. I use the term restructuring because I have not rewritten much of the internal code, just optimized and sorted it better. Much of it has not changed aside from areas where the objects used were changed greatly.

Ok, so let’s focus more on what has happened internally now. Originally there were a lot of methods calling methods and deeper callstacks, well now unless completely necessary every method is completely self maintained and does not internally call upon another. I’ve made my method argument lists more appropriate for C++ porting. However, I have also added higher level overloads which do not look as much like C++. Naming conventions have been held true, and many fields are marked internal now. Using statements have been cleaned up(this can actually be a huge optimization). All of this has made the performance better and the cleanliness much greater.

Exposure of High Level and Low Level methods

Before I only really exposed high level methods to access features. Although, I did have internal methods to do much of the low level portion as well. The biggest change here is that the low-level methods are now public and cleaned up. This also gets back into extensibility of the engine. If I don’t expose these methods then users of the engine will still have to handle things the XNA way. My methods just make that process a little simpler, even the low level ones.


IO vs Content Pipeline

This is a big change more so than original design at all. Originally there was a high focus on the IO system to load files which were unique to RuneEngine. Well, to adhere better to XNA and what XNA developers are used to, I am extending the ContentPipeline for many proprietary formats. Of course the C++ iteration will need some major differences here, but it should be manageable. So, pretty much all Rune specific content types are extended in the pipeline. And therefore could be used without any feature in RuneEngine at all.


Compartmentalized Design

This has been the plan since day one. Each DLL has been designed to be completely self maintained and features are being better designed in this new structure to hold true that they can work without the rest of the engine. I do this by writing majority of the objects completely independent from the rest of the engine. I had made some mistakes in the original structure, primarily the content systems. The content was reliant on every other part there, and the Graphics wrappers I had built as well. Now they’re self maintained and as stated above can be loaded through the content pipeline and be used with normal XNA projects.


Platform Target considerations

This part is simple, I’ve focused towards XBOX360, because it has more restrictions than PC and it’s not a mobile device. Not to hark on mobile, but just because market studies show it as the better revenue stream doesn’t mean it’s my primary focus. I prefer to focus on maximum quality I’m capable for. I’m not going to downgrade the output by adhering to mobile. This is a personal stubbornness of mine and I admit that, but I am NOT ignoring mobile. Just I will build the engine focused against everything else and make a mobile build later. Adhering to 360 and REACH profiles are going to keep me closer to what mobile will expect, so I am genuinely not concerned.


Development Tools

There is a fairly large concern here as well, but none of this matters if the engine itself behind it is not backing it. So, I’ve rebuilt a lot of the editor and have a lot of plans for working new features into this side of the engine. There is also a strong feel for extensibility here.




That about wraps it all up, there is not much more to point out at this time. Of course there has been a lot of time spent some days determining what a method should be called, and where it should be located. I didn’t think any of that detail was that important ;-D. Anyways, until next time…

Project Organization and File Structure

I was just thinking on this and realized that I preach about
this to everyone, all my student, anyone I get into dialog with about
programming development. This is something
I’ve gotten better about over the years and have also seen how big of a problem
it can be not doing so. ORGANIZE YOUR
FILES!!!!! Make a bulletproof file
structure. I can’t stress that any more
than I am right now. If you do not get
your files organized in a way you can find things quickly, you will hurt
yourself later.

Now, I don’t believe I have taken the time to give some big
tips and tricks that have worked before for me on a regular basis, so I’m going
to put them up here now. When I create a
new project I generally spend about 2-3 hours putting it together BEFORE
writing a single line of code. You may
say “WOW that’s a lot of time for preparation,” but I say 2-3 hours on a
project that you’re going to spend more than a week on… that 2-3 hours is
nothing. And if you didn’t do it I
wouldn’t be far from promising it would take 2 weeks instead of 1.

So, what I start a new project what do I do? First of all first and foremost even before
starting VS. I create a master directory
for the project. Typically on the root C
drive of my PC at home, if I had a D partition it would probably be there. For example RuneEngine V2’s master directory
is located at “C:\RUNE2.” Now what goes
in the master directory? The solution
right? NO!!! I stress that the solution
is not created here because with the rest I’m going to cover creating backups
would be nearly impossible or an extremely painful process otherwise.

The next folder I create is a child folder of the Master
directory “Bin”. This is where ALL my
projects output goes. In C:\RUNE2\Bin I have these sub directories:

REACH
HiDef
XBOX360
Mobile
DOCS (Yes I actually build documentation, it would not be
inappropriate for this go to outside of the bin in the master directory if you
prefer)
Editor
EditorAPI

As you may see above, these are all the platform targets,
not debug and release like VS typically creates. This is entirely intentional; my
Debug/Release folders are contained in these.
And this is where my projects target their output. This is why we do NOT put the solution in the
Master directory. When we create the
projects and configure their target paths we do not have to redo this for any
backups we generate. Also, Source
control is easier to… well control this way.

So, where are the projects created? They’re created in Subdirectories of the
Master Directory. The current working
solution is contained in C:\RUNE2\RuneEngine_2D_MAIN, but there are 4 other
backups and iterations of the engine there as well as the initiation of the
Native code etc etc blah blah. The
Master Directory contains anything and everything related to the project in
this setup, and it isn’t burrowed in C:\Users\Me\Documents\Visual Studio
2010\Projects….. Yep, not my favorite place to put things.

A note on source control, IF YOU ARE NOT CAREFUL YOUR
PROJECT CAN BE CORRUPTED, hence why I have multiple backups at all times. Usually it only gets corrupted if you’re using
it incorrectly, but better safe than sorry with large projects.

A note about creation of code files in the solution explorer
and handling folders. In C# you’ve
probably noticed that it adds namespace when you add to the folder. And sometimes you don’t want that. Of course you can just delete it and fix it
that way, but my tip is to just always add files in the namespace you desire
and if you want to move the file somewhere else to be more organized, it’s a
simple drag operation.

I hope this is a good read for other developers out there,
everyone has their own styles of handling files and I’ve tried quite a few
variations and this is the way I plan to stick to for a while. Below is a pseudo copy of my file structure
in RuneEngine v2’s project directories:

C:\
  RUNE2 – Master
    Bin
      REACH (Debug/Release)
      HiDef (Debug/Release)
      XBOX360 (Debug/Release)
      Mobile (Debug/Release)
      DOCS
      Editor (Debug/Release)
      EditorAPI (Debug/Release) – FYI I’m going to be
talking A LOT about this soon.
    Lib
    Includes (Lib and
includes are for the native build ignore them if you’re developing fully
managed)
    RuneEngine_2D_MAIN
(There are
actually multiple solutions here which load different combinations of the
projects)
      RuneEngine
      RuneEngine.Graphics
      RuneEngine.Content
      RuneEngine.Physics
      RuneEngine.ContentExtensions
      RuneEngine.Math
      RuneEngine.Scene
      RuneEngine.Threading
      RuneEngine.Components
    RuneEngineOLD2
    RuneEngineOLD1
    RuneEngineBACKUP
    RENativeDX

I wanted to make one last modification to this, though I suggest this methodology highly if it doesn't work for you don't do it. This ulimatley is a reminder of how important this is and a suggestion in what works for me. It's mostly about what you benefit from most. -Happy coding

DevConnections 2012 - Las Vegas

So, I was sent off to the DevConnections conference in Las Vegas recently. I was there as an exhibitor, but I also had a full attendee pass and was told to attend sessions as much as I could while there. There was a lot to learn there specifically on Metro and HTML 5. I'm writing now to summarize some of the things that were key points accross the sessions that I attended.

New C++ Features

I'm actually going to start with the new features of C++ in Visual Studio 11 as it is the shortest and probably in my opinion the most important.

One of the biggest surprises here was Microsofts statement regarding the state of C++ in software development today. IT'S BACK! That's right, as far as Microsoft claims to be seeing the interest and desire to work in C++ is growing again. It was thought for quite some time computers were surpassing the software we were putting on them so there was no need to go through the extra effort of C++/Native development.

The biggest things for me and I assume many developers with an interest and desire to work in C++ that are very good to hear are these two things:

• Intellisense has greatly improved
• Types are now color coded

I know these are not as big of a deal as they seem, but for many years C++ has been a painful experience while these two features greatly reduce just how painful that is. If you have used C++ in the VS 11 Beta already you have probably seen this.

There are a few things that are both really great and a little weird at the same time. For example there are a few new keywords added, one specifically I am not in full agreement on is the "auto" keyword. If you're familiar with var in C# or a few other various languages it basically does the same thing. Auto does not require a type definition, it requires a specific template based instantiation, but does not works very similar to var as it does not specify a type. It does delete itself, so it behaves a lot like a managed memory allocation. I refuse to use automatic type resolution myself so this was not appealing to me personally, and chances are if I'm developing in C++, it is to get away from managed and this sounds too much like managed behavior, without the CLR.

The last big implementation is that XAML can now be used with C++ when using WinRT. Yes, that's right Metro apps can be developed with C++ as well if you didn't already know this.

Something else really neat that threw me off a bit, mainly because they did not show much of what they meant was that supposedly the new Libs for C++ are more DirectX friendly and allow writing 2D/3D graphics in standard C++ libraries. It seemed to me all they really did was include DirectX in the standard Libs/Headers and that's just all it really seemed to be, still cool, but I don't know if it was a HUGE thing for me.

WinRT / Metro

This is where things start getting fun. WinRT and Metro are "practically" the same. So, let that sink in a moment...

Ok, unlike .NET WinRT communicates directly with the Operating System, but is also exclusive to Windows 8 and will not be ported for compatibility in Windows 7, which I think most everyone is aware of. What exactly is WinRT though? It is a new API for development which is NATIVE not managed for development of Metro applications in Windows 8.

WinRT is actually Native COM to be more specific. The DLLs/Assemblies are compiled with CLI metadata the same as .NET and actually... the code you write will very much resemble .NET. Another big kicker is that many of the .NET 4.5 objects ARE WinRT objects. Just in .NET they hit the CLR BEFORE going to WinRT.

INFORMATION OVERLOADED, take a breath...

Ok, now given the fact that WinRT is Native COM what does this give us? Unlike .NET the "common language runtime" WinRT is ACTUALLY more so a common language runtime than .NET by a thousand times. WinRT can be used in C#, VB, C++, Javascript/HTML, and pretty much anything else that can access COM. Another big factor is that ALL WinRT is XAML ready, sooooo... C++/XAML applications are possible where they were not really before.

There are a few downers to this though, WPF/Silverlight developers are not going to be as happy as they originally may have thought. WPF and Silverlight use XAML and so does Metro, but as one of the speakers at DevConnections stated XAML needs to be stated for what it really is and not associated directly with WPF/Silverlight.

XAML as he puts it is a language for object instantiation. Yes, this is true, but there is more to it than that. It defines data bindings as well and some extra syntax for handling event mapping as well. Basically think XML with a twist.

That being said, XAML in Metro communicates with WinRT, while WPF and Silverlight communicated with .NET or the Silverlight specific binaries. The objects and properties may be different... Actually you will find that only the MAJORLY used pieces of each were ported exactly as they are. So, just because you know WPF or Silverlight does not mean it's going to be easy, but it is NOT as bad as some are making out to be. General development should be rather simple, but most Metro apps will need a rewrite anyway to comply with the proper paradigm for Metro.

HTML 5

I really didn't take as much from these sessions. This is mainly because all they were really saying was it's an idea, you can use polyfills to add support to browsers that don't have it. There was some mention of more specifics, but there was nothing too glamorous or interesting to me aside from a little more realization to what the big fuss is. Oh and Opera is the most feature filled browser at this time. HTML 5 is very friendly will cell phones.

LOOK UP MONDERNIZR, this site has a ton of useful polyfills and scripts for making HTML5 work everywhere, even IE6.

Aside from that, I really don't have a whole lot here.

Conclusion

Ultimately, this article is not nearly as glamorous as I wanted it to be, but it's hard to get too deep into the specifics without losing majority of the people who may read this. Either way, I hope there was something here that sparked interest enough to do some of your own research.

ThreadPool vs Custom Pool : MultiPipeThread

This is actually something I did research on a while back (6 months or so). The ThreadPool object in .NET is a fairly powerful concept which reuses existing threads registered to the pool and allocates new threads as necessary dependent on your machine specs. Like all High Level objects this has some overhead to it, but probably less than what most could create on their own with exactly the same behavior.

But, is all of that truly necessary? My concept was to build a threading system which allocates and starts threads at initialization. It stops them when they're not necessary, but doesn't destroy them. It works with a piping concept system. Basically, in my example I describe 4 pipes associated with the MultiPipeThread object. This object receives information from the 4 threads and handles them appropriately.

Okay, so unlike the .NET ThreadPool, these threads are instantiated in the beginning. They are started immediately and block themselves when their inactive. How do we use them though? Ultimately we create an interface which defines the process method of an item that our threading will handle. Of course this could potentially be a simple delegate, but in my implementation it is not.

When we push items towards the thread piping system, it collections them and places them in a queue. It can potentially be expanded to collection them in buckets to gain more control over order etc. Once there are threads available to accept the information of a given type, if that type is prepared and ready to be added, it pulls the bucket or range of collection matching this type. Adds them all at once. This is where we gain our advantage over the ThreadPool. In the thread pool, every delegate is pushed towards and individual thread, existing or not. In our implementation items are pushed in bulk collections. The thread stays active until that thread pipe is complete. When it is complete it is marked ready for information, and as long as the type this pipe is set to accept is available the process starts over.

Does the tests prove the concept? Well yeah, the concept defines creating something very similar but pushing items in larger collections so there is less thread synchronization. Aside from this they are closely the same object, but we have a lot more control with a custom concept. We know exactly what is being pushed where and have opportunities to control order.

The above diagram shows the classes that are implemented. I know there is little you can see from this vantage point, but these objects are not that complex and easily implemented with some knowledge of synchronizing thread logic in .NET.

My favorite example of this object in use, is like so:

I have different phases of logic that need to be processed, some which can be overlapping some which cannot. Lets say 3 types, 2 of which can overlap and one that cannot. But the one which does not overlap must be first. So we mark it for priority and type ALONE. All pipes are marked to availability to accept ALONE, and we also add types GROUP1 and GROUP2, GROUP1 is allocated to pipes 1 and 2, where GROUP2 is allocated to pipes 3 and 4. Now we push a collection of items containing a mix of all. All objects of type ALONE fill up all 4 pipes immediately. Even though a pipe may open up before completion of ALONE, we restrict groups 1 and 2 from pushing in until all items currently in the pipe system complete. Once all is complete items in GROUP1 push to pipes 1 and 2 equally, and GROUP2 pushes to pipes 3 and 4. So they process simultaneously. Let's say we had a GROUP3 which could process at the same time as GROUP2, but not GROUP1. If GROUP2 finishes first it will still wait for GROUP1 to finish, however if GROUP1 finished first GROUP3 would allocated into the available pipes. We could even mark pipes 1-3 for GROUP3, so if pipe 3 is completed even though it's used by GROUP2, items from GROUP3 will go there.

This control is something we don't have with the ThreadPool, and even if we built a wrapper, we would not gain the same prowess we gain from implementing our own similar but different system.

Polymorphism: Structs, Interfaces, and Huh?

Ok, I know odd title, but I promise this will all get explained.

First, let’s start with some definitions of these concepts.

Polymorphism-

I like the Wiki version of this definition so here’s the link: http://en.wikipedia.org/wiki/Polymorphism_in_object-oriented_programming

For those who prefer not to click links and stay here, I will explain in a 1000 foot overview way. Basically in object-oriented programming, this is the ability to create a value, variable, function, or object that has more than one form. Shapeshifting so to speak, just as it would mean if not referring to coding. WOW! Simple enough right?

Structs-

MSDN does a fairly good job describing this:

http://msdn.microsoft.com/en-us/library/saxz13w4.aspx

Here, a struct is essentially the same as a class but it’s a value type. Just a little more limited, but can be less expensive, especially when using smaller objects that may be contained in large collections for example.

To get deep with it, a class is allocated in memory and allocates pointers (IntPtr) to the object wherever you place a field for reference type objects, classes.

However, structs, or value types, are allocated where you place them and do not generate pointers. This does mean they will copy and become unique allocations moving through the call stack and around the different scopes of your application, this can make them both easier and more difficult to control.

Interfaces-

MSDN also does their job well here:

http://msdn.microsoft.com/en-us/library/ms173156.aspx

Interfaces can be a complex topic for some, but to try to sum it up as simple as possible. They contain a reference to an object which implements the same methods, properties events that the interface defines. If you’ve dealt with C++ these will look a lot like class definitions in header files, and ALMOST behave like them. Interfaces cannot be instantiated and cannot define code implementation for it’s members.

Alright, Polymorphism in general is a huge topic of coding, so we’re not getting too deep, just showing one AMAZING example, though it may be hard to see all the benefits without trying it yourself.

If you’ve dealt with structs before you’ll know a few things about them, such as the inability to use inheritance, and the frustrating ref keyword when trying to pass them into methods as a reference type instead. So, their benefits typically get outweighed by the aggravations of these things. Here’s a tidbit that a lot of people forget, though structs cannot inherit they can implement an interface. It gets better though. Interfaces can inherit other interfaces….

Due to the two above facts, we can use polymorphism techniques to build a collection of structs which inherit each other, and can be passed as reference types without the ref keyword.

My example, which I’ve included the class diagram for, uses the following structs and interfaces:

• IVector
• IVector2
• IVector3
• IVector4
• Vector2
• Vector3
• Vector4

Since the structs, the one’s without the ‘I’, cannot inherit each other all the inheritance tree is in the IVector-IVector4, and it just goes incrementally.

Ok, got that part, but what does this allow me to do?

void RandomMethod( IVector aV1 ) { }

The above method declaration accepts IVector, so what can be passed into this method? This method can actually accept ANY Vector type as it’s interface description. It could use IVector.AxisCount to determine which one was actually passed and handle it appropriately.

That’s one use, which is pretty cool. What about this one?

Vector4 v = new Vector4( 1,1,1,1 );

IVector2 v2 = v;

Will this even work? YES, this just gave me a reference to the Vecto4, the object is obviously still the same object, but rather than converting the value, we made it accessible as a Vector2 interface. Of course you would need methods that accept the interface instead of the struct to use it. Hence why IVector2.ToVector2() exists. This is a simple convert, where if the internal object is not a Vector2, it converts it before returning it.

I know this is a lot to take in so I will stop here. Have fun!!!

Introduction

Hello All,

I just wanted to take a moment to explain why I’m even creating this blog before making my first real posting. This blog I am going to use to pull all of my development activity into a single location.

Those who know me already and have been watching some of my other posts on the RuneEngine v2 Facebook Page should know I post almost everything there as it is. However, there are things I reserve posting there as they are unrelated to the RuneEngine project or any of it’s inner workings.

So, basically anything I post there or anywhere else, will likely end up here as well.

A little about me though. My name is Danny Helms and I am a full time support agent for an Imaging SDK provider in Charlotte, NC and a part time faculty for CPCC SGD Department, as a programming instructor. I have been coding since I was 8 years old, so been a pretty long time now. One thing you may see as I write is that no matter how long I’ve been developing or how far I go, I’m always learning new things. It’s not only because the field changes and advances on a daily basis, but also because it’s already so vast it’s hard not to come across new things all the time.

You may see me mention use of languages in:

• -C#
• C++
• WPF
• Silverlight
• SOME VB.NET
• Javascript
• May start some Java soon
• ASP.NET is also something I will likely piddle with in near future.

Common SDKs I use:

• Microsoft’s XNA Framework
• WeiFenLou Docking API (May not use this as much as I am moving further into WPF development)
• BEPU Physics Engine

Current Projects:

RuneEngine V2 Projects----

• RuneEditor.API (Simple API for addons to the RuneEditor application)
• RuneEngine (2D)
• RuneEngine (Native)
• RuneEngine (Reach, Mobile friendly)
• RuneEngine (HiDef, the core project and most of my efforts go here)

Other Projects---

• BitbucketReporting.API ( Utility API for working with Bitbucket's WCF API, included in RuneEditor.API )