How do you implement a message queuing system?

[Leslie Sanford]

smesser wrote:

Are you just banging this out for me or will you have an example to use the EventQueue class?

I may devote an article to it at some point. I'll definitely put it in the next version of my state machine toolkit for others to use. Let's see if I can give you a quick example of using the EventQueue.

public class MySystem
{
private EventQueue eventQueue = new EventQueue();

private MusicPlugin musicPlugin;

public MySystem()
{
    // Create events.
    eventQueue.CreateEvent("Play");
    eventQueue.CreateEvent("PlayingStopped");

    // Create music plugin and give it the event queue.
    musicPlugin = new MusicPlugin(eventQueue);

    // Subscribe to the playing stop event.
    eventQueue.Subscribe("PlayingStopped", new EventQueueEventHandler(HandlePlayingStopped);
}

private void HandlePlayingStopped(object sender, EventQueueEventArgs e)
{
    // Logic for handling playing stopped event.
}

}

Here, the system object handles an event sent to it from the music plugin telling the system that it has stopped playing. Also, the system takes on the responsibility for creating all of the events before passing the event queue on to the plugin(s).

public class MusicPlugin
{
private EventQueue eventQueue;

public MusicPlugin(EventQueue eventQueue)
{
    this.eventQueue = eventQueue;

    eventQueue.Subscribe("Play", new EventQueueEventHandler(HandlePlayEvent));
}

private void HandlePlayEvent(object sender, EventQueueEventArgs e)
{
    // Logic for starting playback.
}

public void StopPlaying()
{
    eventQueue.Send("PlayingStopped", this, null);
}

}

Now with an example this small, there's not much of an advantage here over using C#'s built in events. However, if you have a lot of plugins that are sending events to each other as well as to the system, this approach could help keep the plugins decoupled. You would have one central event queue for handling all of the event notification.

[Steve Messer]

Thanks for the example. This is exaclty what I needed to get my project back on track again. I have been mulling over this issue for too long.Having the plugins loosely coupled has been one of my goals since the beginning of my project. I have many parts completely coded and now finally I have a way to tie it all together. Once again, I am very greatful for your discussion and your help.

[Leslie Sanford]

I've been testing out my EventQueue class, and I've noticed a bug:

delegateQueue.Invoke(new SendOrPostCallback(delegate(object state)
{
List subscribers = (List)events[eventName];

if(subscribers == null)
{
    throw new InvalidOperationException("Event does not exist.");
}
else
{
    subscribers.Add(new Subscriber(handler, context));
}

}), null);

Get rid of the null that's getting passed to the annonymous method. It confuses the runtime when it starts to invoke the delegate. So it should look like this:

delegateQueue.Invoke(new SendOrPostCallback(delegate(object state)
{
List subscribers = (List)events[eventName];

if(subscribers == null)
{
    throw new InvalidOperationException("Event does not exist.");
}
else
{
    subscribers.Add(new Subscriber(handler, context));
}

}), new object());

[EDIT] I will have a new and improved version up later today as I test it more thoroughly. Also, that should be "new object()" getting passed to the anonymous method. [/EDIT] -- modified at 11:32 Monday 26th June, 2006

[Steve Messer]

Thanks for the bug fix. I look forward to the new and improved version.

[Leslie Sanford]

Here[^] is the updated version. I made a few changes, cleaned up the code, and added comments. I think this is a fairly stable version.

[Steve Messer]

Thanks you are most kind.

[Steve Messer]

I finally had a chance to look at your code, I was on vacation. I really like it but I want to use it a little differently than your example. I don't want to expose the eventQueue directly to plugins. I don't want them to have to have that code in order to compile a plugin. Maybe you can help me out. I want my system object to be responsible for creating events and subscribing plugins to them. If you remember my setup Plugins have an bool OnMessage( ISystem sys, IMessage msg ); The plugin can create a message using: IMessage m = sys.NewMessage( "music.play" ); // and then send it. m.Send(); My problem is that the message object gets created over and over again and therefore not persistent. The Send method in the Message class needs to be able to create and send the events. How can I accomplish this? Thanks -- modified at 14:44 Tuesday 4th July, 2006

[Leslie Sanford]

For some reason, I keep getting an error when trying to reply to your latest message, so I'm trying here instead. The System object could have a hash table where it keeps its messages. When a message is created, the name of the message is stored as a key in the hash table and the message object is stored as its value. Next time someone calls the NewMessage method asking for the same message, instead of creating a new message, the original message is retrieved from the hash table. So only one message of each type is created. When a message object is created, it can be passed the EventQueue along with the name of the message. When its Send method is called, the Message sends the message using the EventQueue. So the EventQueue is hidden inside the Message class. The plugins don't have to know anything about it.

[Steve Messer]

This part I understand

Leslie Sanford wrote:

The System object could have a hash table where it keeps its messages. When a message is created, the name of the message is stored as a key in the hash table and the message object is stored as its value. Next time someone calls the NewMessage method asking for the same message, instead of creating a new message, the original message is retrieved from the hash table. So only one message of each type is created.

This part I don't understand

Leslie Sanford wrote:

When a message object is created, it can be passed the EventQueue along with the name of the message. When its Send method is called, the Message sends the message using the EventQueue. So the EventQueue is hidden inside the Message class. The plugins don't have to know anything about it.

class System
    {
        Hashtable hashList = new Hashtable();
        EventQueue eventQueue = new EventQueue();  
        
        public Message NewMessage( string sub )
        {
            Message m = null; // = new Message( "music.play" );
            
            if( !hashList.ContainsKey( sub ) )
            {
                m = new Message( "music.play" );
                hashList.Add( sub, m );
                eventQueue.CreateEvent( sub );     
            }
            else
            {
               m = (Message)hashList[sub];
            }
            
            return m;
        }     
    }
    
    class Message
    {
        string subject;
        string data;

        public Message(string s)
        {
            this.subject = s;
        }
        
        public void Send()
        {
            string sub = this.subject;
            string datium = this.data;

            // what should happen here?
        }
    }

[Leslie Sanford]

This is what I was getting at:

public class Message
{
string subject;
EventQueue queue;
string data = string.Empty;

public Message(string s, EventQueue queue)
{
    this.subject = s;
    this.queue = queue;
}

public Send()
{
    queue.Send(this.subject, this, data);
}

}

[Steve Messer]

I thought that the eventQueue was part of the system object. Using your example the eventQueue will get disposed of each time a message is created and thereby losing any subscriptions. Also your still passing in the eventQueue which means the plugin will need that code to compile. Am I misunderstanding still? -- modified at 17:10 Tuesday 4th July, 2006

[Leslie Sanford]

smesser wrote:

I thought that the eventQueue was part of the system object.

The System object owns an EventQueue object. It's the System's job to dispose of it. It's simply passing the EventQueue object along to the Message objects so that they can use it to send messages. Make sense?

[Steve Messer]

It makes perfect sense but it violates the coupling I mentioned several responses back when we were talking about hiding the eventqueue completely from the plugin. My intention is that the System object creates the eventqueue and any new messages and sends them to all plugins which have subscribed. The plugin would send reponses that the system object would turn into events and add them to the queue or send them. Thats what I thought we were trying to accomplish a few posts back.

[Leslie Sanford]

Hmm, I may not have a clear understanding of the architecture you're trying to implement. As far as coupling goes, if you have an IMessage interface that all message classes implement, then all the plugins have to know about is the interface. The concrete implementation will be hidden from them, and thus the EventQueue.

[Steve Messer]

Yes, but then that brings me back to one of my orginal questions. If the System object creates and disposes of the event queue. Then where does the event queue inside of the message class fit it? Each instance of message would have it's own eventqueue that would get disposed of when the message was disposed of taking with it any subscriptions that may have been made to it. Simple overview: 1. System object creates messages and manages the global eventqueue 2. System object creates and sends events from plugins via the Message send method. This way there is one global event queue. 3. Plugin has no concept of event queue nor cares about it. Hope this makes sense.

[Leslie Sanford]

smesser wrote:

Each instance of message would have it's own eventqueue that would get disposed of when the message was disposed of taking with it any subscriptions that may have been made to it.

But the messages wouldn't dispose of the EventQueue given to them. It's all in how you implement the message class's Dispose method. Whatever the message does when it is disposed of, it knows not to dispose of the EventQueue because it doesn't belong to it.

smesser wrote:

Simple overview: 1. System object creates messages and manages the global eventqueue 2. System object creates and sends events from plugins via the Message send method. This way there is one global event queue. 3. Plugin has no concept of event queue nor cares about it.

I think it's number 2 that's giving me trouble. Let's run through a scenario to see if I understand: 1. The System sends a message to a plugin via the OnMessage method. 2. The plugin does something with the message. 3. The plugin wants to send a message telling the System and any other plugins listening that something has happened. 4. The plugin asks the System for a message object representing a specific message, e.g. "PlayCompleted". 5. The plugin gets the message back from the System and sends it calling the message's Send method. Is this close?

[Steve Messer]

Leslie Sanford wrote:

I think it's number 2 that's giving me trouble. Let's run through a scenario to see if I understand: 1. The System sends a message to a plugin via the OnMessage method. 2. The plugin does something with the message. 3. The plugin wants to send a message telling the System and any other plugins listening that something has happened. 4. The plugin asks the System for a message object representing a specific message, e.g. "PlayCompleted". 5. The plugin gets the message back from the System and sends it calling the message's Send method. Is this close?

This is very close 1, 2, and 3 spot on. 4. The plugin would subscribe to messages that it is interested in. The plugin never explicitly asks the system for a message. It could by timer send messages via a stored reference to the system object or via a subscribed event the plugin could get it's OnMessage function called thus exposing the System object and the current message that is being sent. The plugin could either do the prescribed action and/or generate it's own event/message. I am trying to have the System object bare the burdon of the event queue. The plugins only exposure to the system is via it's OnMessage function which only exposed the system object which would do the actual message creation and then the message retrieved fromt the system object could then be sent. 5. Close, the plugin would get a the message from the System/EventQueue and either ignore it, respond to it by using the system object to create a new message and then sending that new message. The plugin's OnMessage function would return true if the message was consumed. You wouldn't resend a message sent via the OnMessage function but create a new one if a response it warranted.

[Leslie Sanford]

smesser wrote:

5. Close, the plugin would get a the message from the System/EventQueue and either ignore it, respond to it by using the system object to create a new message and then sending that new message. The plugin's OnMessage function would return true if the message was consumed. You wouldn't resend a message sent via the OnMessage function but create a new one if a response it warranted.

So you could have something like this in a plugin?

public bool OnMessage(ISystem system, IMessage message)
{
// Do something with message.
if(message...)
{
}

// Send a message to system and other plugins:
IMessage msg = system.NewMessage("Completed");

msg.Send();

return true;    

}

One thing you may want to consider is to use the System class as a wrapper around the EventQueue class. So instead of using message objects, a plugin would call a method on the system object itself. So...

public bool OnMessage(ISystem system, IMessage message)
{
// Do something with message.
if(message...)
{
}

// Send a message to system and other plugins:
system.Send("Completed", this, someData);

return true;    

}

One thing that I think needs clearing up is the event handlers in the plugins. If a plugin sends an event to the other plugins, would you require those events to be handled in the OnMessage method, or would you have other methods for handling events from other plugins? I'm trying to find out if a plugin distinquishes between events sent from the System and from other plugins.

[Steve Messer]

Leslie Sanford wrote:

One thing you may want to consider is to use the System class as a wrapper around the EventQueue class. So instead of using message objects, a plugin would call a method on the system object itself. So...

The only problem for me with this approach is that my message object inherits from a Data item and also can cantain a list of this data items. It is very flexible and can be used to send lists of any type of data.

Leslie Sanford wrote:

One thing that I think needs clearing up is the event handlers in the plugins. If a plugin sends an event to the other plugins, would you require those events to be handled in the OnMessage method, or would you have other methods for handling events from other plugins? I'm trying to find out if a plugin distinquishes between events sent from the System and from other plugins.

Yes, all plugins would receive events via the OnMessage function whether it be from the system or another plugin. I haven't mentioned it but there is a separate send function for sending a message to another plugin. bool SendTo( plugin handle ) Something to that effect, I haven't worked out the detail yet. So, a message is a message regardless of who send it. It could be from the system or any number of plugins. If you are subscribed to it you will get your OnMessage function called.

[Leslie Sanford]

smesser wrote:

The only problem for me with this approach is that my message object inherits from a Data item and also can cantain a list of this data items. It is very flexible and can be used to send lists of any type of data.

Ok. Well, you could keep the message classes around but still move the event sending to the System class:

public bool OnMessage(ISystem system, IMessage message)
{
// Do something with message.
if(message...)
{
}

// Send a message to system and other plugins:    
system.Send("Completed", this, new SomeMessage()); 

return true;    

}