RuneEngine V2's Flow System

I thought it would be kind of me to write a larger post about what this system is, how it will be used, and why it is a crucial part of the new RuneEngine V2 feature list.

If you're at all familiar with DirectShow basically imagine that, but re purposed for something... different.  For those who don't know, essentially we're working in Blocks that I call the IFlowBlock interface.  A block defines a procedure that handles a certain type of data.  This part is somewhat different from what DirectShow does.  The data is generic, where in DirectShow it's always a type of media passed by a buffer of bytes.  RuneEngine's Flow system can pass more or less whatever you see fit.  There are just a few rules. So, in DirectShow a sample or data is passed downstream all the way to the end, in every case.  Well... Flow is not going to do the same necessarily.  It can, but generally speaking the end block is going to be a renderer or finalizing state, so it should only be hit when the rest is complete.  Flow data starts upstream and passes downstream until it hits a breaking condition.  Data can be flagged for passthrough transit meaning, it's going all the way to the end or to a certain point before processing itself.

I could go on and on about how data moves and the different possibilities, but the key thing to understand about Flow vs DirectShow.  DirectShow has rules, while Flow is more user defined.  The both have Pins and "Filters" or "Blocks", even "Graphs", but ultimately they are completely different beasts.

Flow is not designed for processing video/image data, it is designed for processing some data in a specified sequence using a system of interchangeable blocks.  Within the upcoming particle system this will be used by the ParticleController.

The blocks though will control a state of a particle.  A particle graph may look rather complicated, but it houses some insane possibilities compared to other particle systems.  And it does this without creating 20 emitters and controllers.  Below is an example of what a simple particle system graph may look like:

This may not be the final example, but in this the idea is despite the appearance, when in one of the velocity blocks the object actually does not receive wind, it just uses velocity to update.  Wind control happens after the velocity state is complete.  So, for a certain time, a particle will be moving in a specified velocity, when reaching it's final output, it moves to wind control.  Particles internally hold a velocity value, so the old velocity should be there still.  Wind control is going to manipulate this.  So, you would see the particle slow down and react to the wind.  Imagine and explosion where the particles move out quickly then get carried away in the wind after.  That is what this somewhat the idea of what this effect would accomplish.

This demonstrates that this system is more sequence driven.  One block does not start til the following is complete, though that is not a forced behavior.  It was the design.  However, particles are small instances so the overall will almost seem like streaming update, it's not exactly.  Now, in theory could you create a loop back effect to handling particles, without changing the underlying system?  Absolutely!!! You could easily create a block that has a loop back output pin which send's items back to another block if the parameter to pass forward has not been met.

Given that this system is newish, and purpose is different than similar systems these are only speculations of ideas.  The actual implementation and design of graphs will come after some testing.  If I find some cool tricks to do with the FlowGraphs they will likely get shared as examples. =)

That's all I got for now though.

New API objects for RuneEngine V2

Ok, the title of this post is almost misleading, this is more about the correction of some features that were slightly flawed in their design which have now been corrected.  So I'm going to talk a little about it.

RuneEngine V2 has had a feature since the beginning referred to as a content reference system.  The idea and purpose of this system is to be able to dynamically load content and handle such implementation as transparently as possible.  Sadly, it only truly came to my attention last night it was not serving it's full purpose.  It wasn't exactly wrong.. but it was defeating it's purpose in existing with the existing implementation.

Now some MAJOR changes were made, everything from the outside looks almost the same, except it is a little easier to use now.  I should be taking that another notch further, but that's a completely different talk.  So, what was changed?  Here's a basic list from the far back perspective:

Removed--
IMaterialReference
IPostShaderReference
MaterialReferenceObject
PostShaderReferenceObject
MultimapTextureReferenceObject (rebuilt to interact with the TextureReferenceObject instead of separate)

Added--
IShaderReference (new material reference, but also serves as post shader reference)
RenderTargetReferenceObject( it was missing )
IContentContainer (this guy... does magic)
ShaderReferenceObject

Ok, that's about it.  As you can see it's a pretty even exchange, but the added were completely missing or are more of a rename.  The multimap texture reference and texture reference objects were completely unnecessary to be separate, I may have to do some wiggling to make it work, but it is done.  Material references and post shading references were extremely redundant hence the spark to make these changes.  In the process is how I found the issues.

IContentContainer was added to resolve the biggest of the issues as well as add a few functionalities that really make this feature sing.  The issue was, the reference objects were calling load methods on instantiation in most cases, and if they didn't.  The components were.  Why not use a normal load call if I'm not going to use these for what they were added for right?  So, first I stripped all calls that forced loading of the content references, then looked closely at what was missing.  IContentContainer was what I came up with.  This interface defines a collection of callbacks that the IContentReference objects call when they are loaded or unloaded.  It has a method for making sure the reference is associated.  That sort of thing.

One interface all it really took?  Not a chance! Now, the IContentReference interface also has some added methods for working with the IContentContainer.  Also, RuneContentManager was extended and improved to work more closely with this new interface as well.

So, rather than being super abstract about what happens let me just explain a few scenarios and how these changes have impacted them.

Scenario #1:
I'm building a game where the scene is rather simple, but the Content takes a long time to load.  I do not want to make a super long load time when a level is loaded.  But, I'm afraid to preload the content bogging down the CPU in the main menu.  In this case I can preload the Scene files but not the content.  Once the level is selected we can prioritize required content loading it first allowing the level to start with partial content and potentially even a partial scene.  The reference objects allow this separation of load times, but not in a way that is forced by the engine.  You can load it whenever you please.

Scenario #2:
I have a large collection of global resources that need to be loaded.  I can put them in a global RuneContentManager and link it to my SceneGraphs allow the components to search through it's content as well.  I can load all these global content items at startup, background or load screen.  And pull them using the RuneContentManager linking systems.  The amazing part here is that the contentReferences will manage unloading and reloading of those assets individually if they're not really being used.  Also, the ContentReferences will pull more directly from the contentManagers now than before.

Scenario #3:
I'm building an Elder Scrolls Clone.... GG!  The newest implementations actually allow full user control over when content is loaded and the IContentReference objects do all the tricky stuff.  You can load a resource in anyway RuneEngine allows and the components and the IContentReferences will receive that callback letting all places that need to know, know that the resource is available and can be used now.  No periodic or frame by frame update required.  All of this leading to a fully seamless platform capability in a very simple way at that.

TextureContentComponent.AssetName = "assetName";
RuneContentManager.Load( "assetName" );

Done.. This like much of RuneEngine, is order independent.  RuneEngine is unique in this element that I've spent a lot of time working around pre-requisite type issues.  The game will not fail when resources are missing it'll just respond to it internally or with user defined code.  Components can be added in any order.  SceneNodes can be activated before or after components are added, it doesn't matter.  The only real prequisite to any of RuneEngine is this line:

RuneManagementService.Startup();

I'm extremely enthusiastic lately watching all of these architectural designs in the engine being realized. So pardon me if I seem ranty.

Anyways, happy coding.

Thoughts after the RuneEngine V2 Code Review

Last night I spent a lot of time going through the RuneEngine V2 source code, updating documentation and reviewing the source for poor organization and garbage code left behind.  This is a good practice to act on with some regularity, but isn't always necessary.

Going through this I had quite a few thoughts.  Absolutely none of them involved considering change.  I found that for 210 code files currently they all seem to be in pretty good order and have great readability.  But that is aside from the point.  The real point is an issue of mentality gained from reviewing the source.

I have spent a lot of time over the past few months procrastinating about not working on the engine.  I am getting heavy back in development with it so I decided a cleanup pass was in my best interest.  I had been procrastinating because of "how much is left."  Well, this review showed me quite a it in the direction of how inaccurate that really is.  There are still plenty of unfinished areas and new features that need implementation, but reminding myself that there are systems and features already implemented that push into professional grade quality affected my opinion of where I am at.  It would appear that by the end of the week I could have any loose ends taken care of and begin development of new features next week.  By new features I specifically mean building the editor.

Now, I've spoken here about the goals of the engine as far as features, but here's a list of what I found that gives me such a confident outlook on the status.

- Deferred Rendering
- Post Processing System
- Dynamic Content system
- Custom material format
- User defined geometry system capable of working to the extent of a simple modeling utility
- Extendable file IO system
- Extendable XNA content resource extensibility (This is small, but huge at the same time, specifically it is    built that shaders with custom structs used can be applied via the material file format with some extension to this area)
- Input System, with plugin capabilities
- Updateless and per-frame draw call removed system (huge optimization)
- Custom SpriteBatch which renders into a deferred scene with textures that support the multiple channels.
- Architecture in a way that almost every part of the engine is 100% self reliant.
- Low-Level and High-Level implementations for most features

As a foundation, RuneEngine V2 has all the necessary pieces to be something really impressive.  Nearly the entire engine is designed for extendability.  This design is going to make moving forward significantly easier than it was in past iterations.

Very soon, after the loose ends are tied up I will be making a post here and on the RuneEngine V2 facebook page.  This post is going to be a demonstration of development with RuneEngine V2.  I want to showcase, how some of this stuff is compartmentalized and how, as a coder using it, things are going to be done.  In many cases a test project only consists of a few RuneEngine lines of code.  If it's much more then it is due to setting up parameters of objects for a less "defaulted" test.

Anyways, happy coding.