Synchronous message transmission
-
Is there a protocol that guarantees the order of messages? Not packets, I mean messages (consisting of multiple packets). For example, I want to be able to send msg1, msg2, msg3 and receive them in that order. But here's the catch, the sender can't (yes, really, it can't) send them synchronously, waiting for a message acknowledgement. So, is there a messaging protocol that guarantees delivery in a particular order for messages? Maybe named pipes (a pipe seems like the right concept)? Marc
I don't get it. Doesn't TCP/IP sockets get you this? Messages between a given sender and receiver are guaranteed to be in order.
Software Zen:
delete this; -
I don't get it. Doesn't TCP/IP sockets get you this? Messages between a given sender and receiver are guaranteed to be in order.
Software Zen:
delete this;Gary R. Wheeler wrote:
Messages between a given sender and receiver are guaranteed to be in order.
I'm still puzzling over whether that is true, especially if each message is sent within the context of its own socket connection. Marc
-
Gary R. Wheeler wrote:
Messages between a given sender and receiver are guaranteed to be in order.
I'm still puzzling over whether that is true, especially if each message is sent within the context of its own socket connection. Marc
I think Gary's right, considering the way TCP works (sending whole messages and not returning until the whole thing has been ACK'd) it seems to me that you have a pretty good guarantee right there. If you had to use UDP for whatever reason you'd have to do some thinking involving maybe embedding serial numbers into your messages but since UDP doesn't guarantee delivery it would be pretty useless I think. So to sum up: Yeah TCP should guarantee you get the messages in the right order. I took a computer networking course last semester so this stuff is still pretty fresh in my head :cool: hth
-
Is there a protocol that guarantees the order of messages? Not packets, I mean messages (consisting of multiple packets). For example, I want to be able to send msg1, msg2, msg3 and receive them in that order. But here's the catch, the sender can't (yes, really, it can't) send them synchronously, waiting for a message acknowledgement. So, is there a messaging protocol that guarantees delivery in a particular order for messages? Maybe named pipes (a pipe seems like the right concept)? Marc
It seems to me that if each message is being send async over different sockets you are going to need some way of identifying which sets of messages are related, the order in which they are meant to arrive and the total number to expect. The simplest solution I can think of is to create a message header that details this information. The receiving system should then be able store incomplete message sets and wait until the total number of messages has arrived before sorting them by message order. The receiver will need to be designed to be able to handle multiple sets of messages being sent at the same time to avoid data clashes. I would also add some sort of check sum if the data is going over a public network to ensure that no one has tried to inject messages into the system.
-
Never heard of anything like that, but one never knows... Anyway, I think that it shouldn't be very difficult to design a protocol like that. I think it's sufficient to add a timestamp on each message, and the receiver just needs to sort them by time. Or am I missing something?
If you truly believe you need to pick a mobile phone that "says something" about your personality, don't bother. You don't have a personality. A mental illness, maybe - but not a personality. - Charlie Brooker My Blog - My Photos - ScrewTurn Wiki
Dario Solera wrote:
I think it's sufficient to add a timestamp on each message, and the receiver just needs to sort them by time.
Alas, you're missing one small point: althought the time stamps would indeed sort the messages in the order sent - what's to guarantee that no message is missing from the sequence? They'd still be sorted temporally. Maybe the solution is to send one great big message!
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
-
Is there a protocol that guarantees the order of messages? Not packets, I mean messages (consisting of multiple packets). For example, I want to be able to send msg1, msg2, msg3 and receive them in that order. But here's the catch, the sender can't (yes, really, it can't) send them synchronously, waiting for a message acknowledgement. So, is there a messaging protocol that guarantees delivery in a particular order for messages? Maybe named pipes (a pipe seems like the right concept)? Marc
What Yortw said: It may be too heavy a solution for you, but MSMQ (Microsoft Message Queue) using transactional queues should provide this functionality. That functionality is built into MSMQ.
Tom Garth Developer R. L. Nelson and Associates, Inc., Virginia
-
Dario Solera wrote:
I think it's sufficient to add a timestamp on each message, and the receiver just needs to sort them by time.
Alas, you're missing one small point: althought the time stamps would indeed sort the messages in the order sent - what's to guarantee that no message is missing from the sequence? They'd still be sorted temporally. Maybe the solution is to send one great big message!
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
Normally just add an ID to the protocol. Each message starts at 1 and counts up. What layer 2 communication are you using? Ethernet, RS232, CAN, USB? TCP/IP does this for you. If you use UDP/IP then it's up to you. A typical protocol will have a header consisting of the senders ID, amount of bytes in the packet,Time stamp,receivers ID, packet counter, ect.. Followed by bytes of data and ending with a CRC checksum to guarantee that the data was not corrupted during transmission. Steve
-
Dario Solera wrote:
I think it's sufficient to add a timestamp on each message, and the receiver just needs to sort them by time.
Alas, you're missing one small point: althought the time stamps would indeed sort the messages in the order sent - what's to guarantee that no message is missing from the sequence? They'd still be sorted temporally. Maybe the solution is to send one great big message!
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
-
Gary R. Wheeler wrote:
Messages between a given sender and receiver are guaranteed to be in order.
I'm still puzzling over whether that is true, especially if each message is sent within the context of its own socket connection. Marc
Let me put it this way. The product I work on sends megabytes of print data and thousands of control and status messages per second around a multiprocessor/multicomputer high-speed ink jet printing system, all using TCP/IP over gigabit Ethernet. Message order is A Big Thing. If messages get out of order we screw up the print data, spray ink on the floor, or kill the operator with the world's worst paper cut (we move paper at 17 feet per second).
Marc Clifton wrote:
each message is sent within the context of its own socket connection
There you go, changing the rules. If each message gets its own connection (even though the sender and the receiver are the same), then all bets are off, since the message order is managed by (or at least a behavior of) the connection. Truth is, they will still probably be sequential, given all the overhead required to establish the connection, send the message, break the connection, repeat. This pattern is much more like UDP, which doesn't guarantee message delivery or order.
Software Zen:
delete this; -
It may be too heavy a solution for you, but MSMQ (Microsoft Message Queue) using transactional queues should provide this functionality.
And the advantage of this is that if you are using Microsoft servers and .Net then there is no further financial outlay to use MSMQ. But if you are in a large organisation you may find they have standardised on another similar product such as IBM's MQ series or Tibco Rendezvous. Nick
-
Normally just add an ID to the protocol. Each message starts at 1 and counts up. What layer 2 communication are you using? Ethernet, RS232, CAN, USB? TCP/IP does this for you. If you use UDP/IP then it's up to you. A typical protocol will have a header consisting of the senders ID, amount of bytes in the packet,Time stamp,receivers ID, packet counter, ect.. Followed by bytes of data and ending with a CRC checksum to guarantee that the data was not corrupted during transmission. Steve
Which am I using? None of the above (in this context). All I was pointing out (in my reply) was that sorting by time (alone) doesn't guarentee that all messages will be read in order. Only those received. w.r.t. you post: Is there more than one time stamp. By this, I mean, does the user system create the time stamp or the server. And, really in either case, can one be sure they are all synchronized (I think this was to come from more than one possible sending unit). A tag seems to be a good idea (so long as the tags coordinate, but then, the clocks could be synched). I don't know much about the communication protocols, so will defer to your expertise. Steve (too).
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
-
aha, you can add some fiedds in your message to index the order of the pkt1, pk2,..., and the totle number of the packets. If you use TCP protocl, then you don't worry the miss the packet.
I believe he was referring to entire messages (that's the context I was answering to) - not packets. What I hoped I had expressed was that sorting incoming messages by time sent doesn't guarantee that you've not missed one or messages in the sequence. I didn't even address synchonization of all the clocks from the sending units (if there is more than one).
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
-
It seems to me that if each message is being send async over different sockets you are going to need some way of identifying which sets of messages are related, the order in which they are meant to arrive and the total number to expect. The simplest solution I can think of is to create a message header that details this information. The receiving system should then be able store incomplete message sets and wait until the total number of messages has arrived before sorting them by message order. The receiver will need to be designed to be able to handle multiple sets of messages being sent at the same time to avoid data clashes. I would also add some sort of check sum if the data is going over a public network to ensure that no one has tried to inject messages into the system.