No REST for the weary
-
So a demi-rant... Why don't people get that REST sucks for a wide variety of applications. Being in the automation world I run into this problem a lot. People make a device, and probably make no effort to ask anyone like me what are needs are, they put a web server in it and design a simple REST API. The problem is that automation systems are fundamentally dependent on low latency awareness of changes in the state of devices, because users set up systems like ours to react to those changes. If it doesn't happen for a second or two after the actual change, then it's often useless or at best annoying. So now you have a a bunch of devices out that that you just have to pound relentlessly with GET commands in order to try to catch any changes in a reasonable amount of time. And then of course they get the bright idea to make it 'easy' by only allowing the transfer of the entire state of the device (like the Hue), so you have to transfer the entire state of the device multiple times a second in order to catch a change that might not happen for many hours or even days at a time, but has to be caught quickly when it does. And, then the customer has some other thing that also needs to do the same, so it's pounding the same device relentlessly and now the device is overloaded and starts responding more slowly in general, or crapping out periodically. It's like so many people just don't even think beyond a REST API or HTTP, and throw it into everything, whether it's remotely appropriate or not.
Explorans limites defectum
I find REST to be carp... but i'm in the minority... so i have to use it :( I once wrote something, but did not focus on latency. Its a good point that i had not considered. GitHub - maxsnts/TARAPI-An-API-Rant: “TARAPI” An API Rant![^] (I end up not finish the Rant because by that point i had released some steam. There's even lots of spelling mistakes)
-
So a demi-rant... Why don't people get that REST sucks for a wide variety of applications. Being in the automation world I run into this problem a lot. People make a device, and probably make no effort to ask anyone like me what are needs are, they put a web server in it and design a simple REST API. The problem is that automation systems are fundamentally dependent on low latency awareness of changes in the state of devices, because users set up systems like ours to react to those changes. If it doesn't happen for a second or two after the actual change, then it's often useless or at best annoying. So now you have a a bunch of devices out that that you just have to pound relentlessly with GET commands in order to try to catch any changes in a reasonable amount of time. And then of course they get the bright idea to make it 'easy' by only allowing the transfer of the entire state of the device (like the Hue), so you have to transfer the entire state of the device multiple times a second in order to catch a change that might not happen for many hours or even days at a time, but has to be caught quickly when it does. And, then the customer has some other thing that also needs to do the same, so it's pounding the same device relentlessly and now the device is overloaded and starts responding more slowly in general, or crapping out periodically. It's like so many people just don't even think beyond a REST API or HTTP, and throw it into everything, whether it's remotely appropriate or not.
Explorans limites defectum
<sarcasm> Well duh, you need to use npm to install AngularJS so you can use Node.js for your WebSockets that you should connect using Webpack. If you're not using at least 100 libraries you're not putting in enough effort and YOU're the problem :D </sarcasm>
Best, Sander sanderrossel.com Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
-
In the so-called 'internet of things' world, which is basically what has become of automation in the popular press and hence in the great unwashed masses for the most part, it's way too common. And of course it's even worse in that half the time it's a cloud based API, so not only REST but you have to talk to a server half-way around the world to talk to a device 20 feet from the computer.
Explorans limites defectum
Dean Roddey wrote:
you have to talk to a server half-way around the world to talk to a device 20 feet from the computer
Same deal where I work. They removed (or refused to replace) all of our attached desktop printers. We now have department printers, all managed from the home office. When I print a code fragment to mark up by hand(*), the data goes 800 miles to a server and then comes back 800 miles to the printer 30 feet away from me. This process typically takes 3-5 minutes. (*) Yes, I'm an old fart. I still do this occasionally. Don't worry, though - my annual stack of recycled paper is around one inch tall.
Software Zen:
delete this; -
You generally don't really want to go that direction for security purposes. The automation controller should always be the one that makes the connection if at all possible.
Explorans limites defectum
So, what's the solution? An x second timeout on a GET that only returns when the underlying data has changed? (Almost like a remote event handler?)
-
So a demi-rant... Why don't people get that REST sucks for a wide variety of applications. Being in the automation world I run into this problem a lot. People make a device, and probably make no effort to ask anyone like me what are needs are, they put a web server in it and design a simple REST API. The problem is that automation systems are fundamentally dependent on low latency awareness of changes in the state of devices, because users set up systems like ours to react to those changes. If it doesn't happen for a second or two after the actual change, then it's often useless or at best annoying. So now you have a a bunch of devices out that that you just have to pound relentlessly with GET commands in order to try to catch any changes in a reasonable amount of time. And then of course they get the bright idea to make it 'easy' by only allowing the transfer of the entire state of the device (like the Hue), so you have to transfer the entire state of the device multiple times a second in order to catch a change that might not happen for many hours or even days at a time, but has to be caught quickly when it does. And, then the customer has some other thing that also needs to do the same, so it's pounding the same device relentlessly and now the device is overloaded and starts responding more slowly in general, or crapping out periodically. It's like so many people just don't even think beyond a REST API or HTTP, and throw it into everything, whether it's remotely appropriate or not.
Explorans limites defectum
-
So, what's the solution? An x second timeout on a GET that only returns when the underlying data has changed? (Almost like a remote event handler?)
Don't use HTTP is one obvious one. Or, if you do, follow the ISY example and send async responses over a persistent connection.
Explorans limites defectum
-
Is your problem with REST per se, or is it just a problem with slow devices transferring too much state?
The problem is as I originally outlined. It's a completely synchronous call/response system being used in a situation where 99.9999% of the time there's nothing to respond with. But, when there is, it needs to be seen very quickly. So it's just the completely wrong tool for the job. On the automation system side, you could be dealing with hundreds of devices, so it's doing huge amounts of I/O and network traffic for essentially no reason, and it has to parse all that data, convert to binary, and check it against what it saw last time to see if it has changed, and each of those devices might have quite a few values to deal with. If the device can only transfer a single big chunk of state, it just makes it that much worse. If multiple clients needs to do the same on those devices, it compounds, and sometimes overwhelms the device.
Explorans limites defectum
-
<sarcasm> Well duh, you need to use npm to install AngularJS so you can use Node.js for your WebSockets that you should connect using Webpack. If you're not using at least 100 libraries you're not putting in enough effort and YOU're the problem :D </sarcasm>
Best, Sander sanderrossel.com Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
Does one of those libraries use the Retroflexive, Inter-secular Fractal Pipe Massage pattern?
Explorans limites defectum
-
The problem is as I originally outlined. It's a completely synchronous call/response system being used in a situation where 99.9999% of the time there's nothing to respond with. But, when there is, it needs to be seen very quickly. So it's just the completely wrong tool for the job. On the automation system side, you could be dealing with hundreds of devices, so it's doing huge amounts of I/O and network traffic for essentially no reason, and it has to parse all that data, convert to binary, and check it against what it saw last time to see if it has changed, and each of those devices might have quite a few values to deal with. If the device can only transfer a single big chunk of state, it just makes it that much worse. If multiple clients needs to do the same on those devices, it compounds, and sometimes overwhelms the device.
Explorans limites defectum
-
Sorry, I had my HTML backward. Your computer should be the server and all the IoT thingamabobs should be clients. Then they could PUT or POST asynchronously. If that's not the way the world works, get another world.
That's generally not desirable for security reasons. The automation system should be the one making connections, as configured by the administrator. In some cases it's unavailable but typically it's something that's not done unless really necessary.
Explorans limites defectum
-
So a demi-rant... Why don't people get that REST sucks for a wide variety of applications. Being in the automation world I run into this problem a lot. People make a device, and probably make no effort to ask anyone like me what are needs are, they put a web server in it and design a simple REST API. The problem is that automation systems are fundamentally dependent on low latency awareness of changes in the state of devices, because users set up systems like ours to react to those changes. If it doesn't happen for a second or two after the actual change, then it's often useless or at best annoying. So now you have a a bunch of devices out that that you just have to pound relentlessly with GET commands in order to try to catch any changes in a reasonable amount of time. And then of course they get the bright idea to make it 'easy' by only allowing the transfer of the entire state of the device (like the Hue), so you have to transfer the entire state of the device multiple times a second in order to catch a change that might not happen for many hours or even days at a time, but has to be caught quickly when it does. And, then the customer has some other thing that also needs to do the same, so it's pounding the same device relentlessly and now the device is overloaded and starts responding more slowly in general, or crapping out periodically. It's like so many people just don't even think beyond a REST API or HTTP, and throw it into everything, whether it's remotely appropriate or not.
Explorans limites defectum
Dean Roddey wrote:
Why don't people get that REST sucks for a wide variety of applications
Or from my point of view why don't people get that Agile doesn't work for architecture?
Dean Roddey wrote:
People make a device, and probably make no effort to ask anyone like me what are needs are,
They decide that they can deliver the API in 6 weeks or worse 2 weeks, and the "product owner" is more interested in delivery than usability. Not to mention of course that when the implementation phase starts there is in fact no actual requirements beyond "make an API" but certainly no willingness to back off on the delivery.
Dean Roddey wrote:
So now you have a a bunch of devices out that that you just have to pound relentlessly with GET commands ...make it 'easy' by only allowing the transfer of the entire state of the device
It was designed based on what info the implementer had. They didn't have time to make 120 different routines, and QA certainly would not have had time to test it. And probably when asked they asserted they were not even sure anyone would use the API. So you get one API call.
Dean Roddey wrote:
It's like so many people just don't even think beyond a REST API or HTTP, and throw it into everything, whether it's remotely appropriate or not.
Its like people think that 2 week sprints are more than sufficient to plan for an architecture that is supposed to be robust and last for years. An an off hand comment in a meeting several weeks ago leads to a requirement that must be implemented right now. And it certainly doesn't have anything to do with actually determining how it will be used and very definitely there is no time to actually figure that out.
