Network communication in the raw
-
Actually, this is the very reason I got a Computer Engineering degree instead of a Computer Science degree. I had enough experience with the software side of things on my own as a hobbyist I wanted to understand how the hardware level worked. I can remember in one class early on we couldn't get our circuit to work just right. Turns out it was because of the wires we were using were too long and so the signals weren't getting to the right places at the right times. It IS amazing how much stuff we can do with a bunch of changes in voltage.
Everyone is born right handed. Only the strongest overcome it. Fight for left-handed rights and hand equality.
ZurdoDev wrote:
It IS amazing how much stuff we can do with a bunch of changes in voltage.
True enough. Voltage and polarity; that's all there is to it, really. (And that's essentially a complete computer manual in under 25 words.) (And it's about as useful as some of the cr@ppy manuals I've had to wade through.)
I wanna be a eunuchs developer! Pass me a bread knife!
-
You know what they say: knowledge is dangerous! ;P
Chris Maunder wrote:
Which implies some kind of state management going on.
What I found fascinating when simulating TCP/IP over satellite and dealing with rain fade (packet loss due to rain) is that the hardware (or more precisely, the software in the hardware) handles all the protocols for requesting retransmittal of packets. Something we take for granted is that packets will be received in the correct order, which is achieved by the hardware layer, packet numbering, and ACK's. By default, TCP/IP over satellite is darn slow, as every ~1400 byte packet requires an ACK, which is a 500ms round trip (250ms from ground to satellite to ground, 250ms back for the ACK.) We were simulating buffering packets on the ground and simulating the ACK so we could get a continuous stream of packets, and using a separate "Error ACK" to resend only the garbled packets. It worked quite well in the simulations, achieving near hardwired transmittal rates. That was the closest I got to diving into the bowels of how things talk to each other.
Latest Article - A Concise Overview of Threads Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
Now add a torrent client on top of that, add meatballs, and serve.
I wanna be a eunuchs developer! Pass me a bread knife!
-
Something I take for granted is that my browser will connect to my servers which will connect to other servers in my hosting setup and that there will be a form of communication that will allow data to be requested and received. But I have no idea how it all works at a low level. Zero. I know that my code will make use of the .NET TCP/IP libraries and that they will handle the work of talking to the OS, which will invoke the drivers, which will use my WiFi hardware, which will talk to my ISP, which will route the communication to the server I requested. The .NET libraries will take care of handling the handshaking to establish a connection, and the sending and receiving the data itself. But I've always wondering how this works. At low level. As in: what does the OS say to Broadcom driver to make the antennae do stuff. What commands does the driver send to the firmware? What (and how?) does the firmware do to make the antenna emit an electromagnetic signal and how does the firmware handle all the electromagnetic radiation it picks up to enable it to listen to the right signal? Another thing that's always bugged me is the nature of the (necessarily) discontinuous communication. It's not a continuous sine wave with a continuous stream of data going to and from. The data is broken into packets and there is acknowledgement and error correction in the packets being transmitted. Which implies some kind of state management going on. Which then raises the question of how does something like SignalR work when there are 10,000 connections (and what is a "connection"?) to a server and the data travels back and forth on a connection that's long term, not short term like a HTTP request (small furrowed brow thinking about HTTP Keep-alive) None of this is important to a developer, and yet it kind of is important to a developer in the same way garbage collection, disk access patterns and thread scheduling. You don't need to know them, but knowing them gives you the knowledge to make informed decisions. Does anyone know of a write-up of this stuff anywhere? Failing that: would anyone be interested in taking a stab at an article that walks through the life cycle of a bit of data travelling from code through the libraries to the OS to the drivers to the firmware to the hardware, via the ISP, across the routers, to the other and and back? I think it would be fascinating.
cheers Chris Maunder
-
That should be a good start - a thousand pages for TCP/IP alone ... (Well, take it with a grain of salt - the TOC seems to suggest that there is some coverage of e.g. X.25, FR and ATM, ... in the part labeled "Core TCP/IP protocols" :-)) Note that the book is (C)2006, twelve years old. Now, IPv4 is 37 years old, so for the basic protocols it doesn't matter. But if you today made the statement opening the chapter on ATM, "ATM-based networks are of increasing interest for both local and wide area applications", half of your audience would dive straight into their smarphones to google what the h "ATM" is, and the rest would try to make sense of the role of banking machines in networking. I guess that the IPv6 parts may have other statements about state of deployment an completeness of the protocol suite that may not be quite up to date. Thanks for the reference to the book - but honestly, I think that for a completely blank beginner, it will serve more as a sledgehammer to knock the reader to the ground than as a simple-to-read introduction to networking!
-
Something I take for granted is that my browser will connect to my servers which will connect to other servers in my hosting setup and that there will be a form of communication that will allow data to be requested and received. But I have no idea how it all works at a low level. Zero. I know that my code will make use of the .NET TCP/IP libraries and that they will handle the work of talking to the OS, which will invoke the drivers, which will use my WiFi hardware, which will talk to my ISP, which will route the communication to the server I requested. The .NET libraries will take care of handling the handshaking to establish a connection, and the sending and receiving the data itself. But I've always wondering how this works. At low level. As in: what does the OS say to Broadcom driver to make the antennae do stuff. What commands does the driver send to the firmware? What (and how?) does the firmware do to make the antenna emit an electromagnetic signal and how does the firmware handle all the electromagnetic radiation it picks up to enable it to listen to the right signal? Another thing that's always bugged me is the nature of the (necessarily) discontinuous communication. It's not a continuous sine wave with a continuous stream of data going to and from. The data is broken into packets and there is acknowledgement and error correction in the packets being transmitted. Which implies some kind of state management going on. Which then raises the question of how does something like SignalR work when there are 10,000 connections (and what is a "connection"?) to a server and the data travels back and forth on a connection that's long term, not short term like a HTTP request (small furrowed brow thinking about HTTP Keep-alive) None of this is important to a developer, and yet it kind of is important to a developer in the same way garbage collection, disk access patterns and thread scheduling. You don't need to know them, but knowing them gives you the knowledge to make informed decisions. Does anyone know of a write-up of this stuff anywhere? Failing that: would anyone be interested in taking a stab at an article that walks through the life cycle of a bit of data travelling from code through the libraries to the OS to the drivers to the firmware to the hardware, via the ISP, across the routers, to the other and and back? I think it would be fascinating.
cheers Chris Maunder
Main page: Computer Networking — CNP3www 2014 documentation[^] Book: Computer Networking : Principles, Protocols and Practice, 2nd edition — Computer Networking : Principles, Protocols and Practice[^] 2019-01-02: just came across this, coincidentally. not sure how well it applies: Network Protocols – Programmer's Compendium[^]
-
Something I take for granted is that my browser will connect to my servers which will connect to other servers in my hosting setup and that there will be a form of communication that will allow data to be requested and received. But I have no idea how it all works at a low level. Zero. I know that my code will make use of the .NET TCP/IP libraries and that they will handle the work of talking to the OS, which will invoke the drivers, which will use my WiFi hardware, which will talk to my ISP, which will route the communication to the server I requested. The .NET libraries will take care of handling the handshaking to establish a connection, and the sending and receiving the data itself. But I've always wondering how this works. At low level. As in: what does the OS say to Broadcom driver to make the antennae do stuff. What commands does the driver send to the firmware? What (and how?) does the firmware do to make the antenna emit an electromagnetic signal and how does the firmware handle all the electromagnetic radiation it picks up to enable it to listen to the right signal? Another thing that's always bugged me is the nature of the (necessarily) discontinuous communication. It's not a continuous sine wave with a continuous stream of data going to and from. The data is broken into packets and there is acknowledgement and error correction in the packets being transmitted. Which implies some kind of state management going on. Which then raises the question of how does something like SignalR work when there are 10,000 connections (and what is a "connection"?) to a server and the data travels back and forth on a connection that's long term, not short term like a HTTP request (small furrowed brow thinking about HTTP Keep-alive) None of this is important to a developer, and yet it kind of is important to a developer in the same way garbage collection, disk access patterns and thread scheduling. You don't need to know them, but knowing them gives you the knowledge to make informed decisions. Does anyone know of a write-up of this stuff anywhere? Failing that: would anyone be interested in taking a stab at an article that walks through the life cycle of a bit of data travelling from code through the libraries to the OS to the drivers to the firmware to the hardware, via the ISP, across the routers, to the other and and back? I think it would be fascinating.
cheers Chris Maunder
I was teaching such stuff (and even wrote parts of the text forthe course, but in Norwegian), in an age when we still were hoping for networks to be orderly and well designed. (That is, in the 1990s.) I see other commenters mentioning the OSI model, which was an essential part of the orderlyness. Well, those were the days... Today, networking is more or less a total mess. Lots of stuff isn't what it appears to be: Maybe the name is the same, but under the hood there is a hodge-podge of different, and largely incompatible, technologies that must be explicitly bridged between. Take "Ethernet" as an example: A multitude of cable types, a dozen different physical connectors, different topologies, different allocation strategies, ... But to the user, it is presented as the eternal, single Ethernet-techology (just with a more convenient RJ45 plug). ooo oo00oo ooo Along another axis: One ting you ask explicitly about is "discontinous communication" (often called "asynchronous" communication). First: In a protocol stack, it may vary from one layer to another. The cable may carry a continous stream of bits at well defined steady rate. The user of this cable may insert some packets now, some then, in a varying rate. If these packets carry some real time data, such as streaming sound or video, the packets may be receied "now and then" at irregular speed, but are buffered so that they can be forwarded to the sound/video units as a contstant speed, continuous bit stream. Second: On the fly, I can list a whole series of alternatives for handling asynchronous traffic in the physical layer alone. It much depends on whether you have the line to yourself or share it with other: POTS, i.e. analog modem. When you have nothing to say, the line is quiet. When you want to send a byte, you sound a buzzer ("start bits") to wake up the receiver, followed by your eight data bits, and a final buzz ("stop bit") to call it off. In later modem protocols, more than eight bits were sent at a time, to reduce the percentage-wise red tape in start and stop bits. Bits were encoded as a beeping tone that in the oldest versions switched between two frequencies ("FSK"), one for binary 0, the other for binary 1. Later, fairly complex schemes were introduced: Each wave peak could vary both in its amplitude and delay (i.e. phase), peak hitting (or being close to) one of, say, 32 defined points, conveying 5 bits per full wave (i.e. per Hz analog channel bandwidth). HDLC is somewhat similar (binary, on/off, no analog), but th
-
Computer Networks 5th (International Economy Edition) Paperback – January 9, 2010 by Andrew S. Tanenbaum (Author), David J. Wetherall (Author) Internetworking with TCP/IP Vol.1: Principles, Protocols, and Architecture - Douglas E. Comer Internetworking with TCP/IP Vol.2: Internals and Implementation Internetworking With TCP/IP Vol.3: Client-Server Programming And Applications Versions
Caveat Emptor. "Progress doesn't come from early risers – progress is made by lazy men looking for easier ways to do things." Lazarus Long
abmv wrote:
Computer Networks 5th (International Economy Edition) Paperback – January 9, 2010 by Andrew S. Tanenbaum (Author), David J. Wetherall (Author)
I got my basic introduction to networking from the 1st ed of this book - the original source of the quote "The nice thing about standards is that you have so many to choose from." The remark comes after a discussion of the tiny little details in which HDLC and SDLC differs, making them incompatible. Tanenbaum continues something like (this is quoted from memory): "And if you don't like this year's crop, just wait until next year, and you will have a few more to choose from."
-
Something I take for granted is that my browser will connect to my servers which will connect to other servers in my hosting setup and that there will be a form of communication that will allow data to be requested and received. But I have no idea how it all works at a low level. Zero. I know that my code will make use of the .NET TCP/IP libraries and that they will handle the work of talking to the OS, which will invoke the drivers, which will use my WiFi hardware, which will talk to my ISP, which will route the communication to the server I requested. The .NET libraries will take care of handling the handshaking to establish a connection, and the sending and receiving the data itself. But I've always wondering how this works. At low level. As in: what does the OS say to Broadcom driver to make the antennae do stuff. What commands does the driver send to the firmware? What (and how?) does the firmware do to make the antenna emit an electromagnetic signal and how does the firmware handle all the electromagnetic radiation it picks up to enable it to listen to the right signal? Another thing that's always bugged me is the nature of the (necessarily) discontinuous communication. It's not a continuous sine wave with a continuous stream of data going to and from. The data is broken into packets and there is acknowledgement and error correction in the packets being transmitted. Which implies some kind of state management going on. Which then raises the question of how does something like SignalR work when there are 10,000 connections (and what is a "connection"?) to a server and the data travels back and forth on a connection that's long term, not short term like a HTTP request (small furrowed brow thinking about HTTP Keep-alive) None of this is important to a developer, and yet it kind of is important to a developer in the same way garbage collection, disk access patterns and thread scheduling. You don't need to know them, but knowing them gives you the knowledge to make informed decisions. Does anyone know of a write-up of this stuff anywhere? Failing that: would anyone be interested in taking a stab at an article that walks through the life cycle of a bit of data travelling from code through the libraries to the OS to the drivers to the firmware to the hardware, via the ISP, across the routers, to the other and and back? I think it would be fascinating.
cheers Chris Maunder
Without reading the entire thread, try this: Internetworking with TCP/IP[^] by Douglas Comer, et al. I have the 2nd edition. Volume 1 is a great introduction to low-level TCP/IP and other base network protocols. Sadly, I have had to learn this stuff in the course of debugging the TCP/IP 'stack' in some embedded software of ours.
Software Zen:
delete this; -
Something I take for granted is that my browser will connect to my servers which will connect to other servers in my hosting setup and that there will be a form of communication that will allow data to be requested and received. But I have no idea how it all works at a low level. Zero. I know that my code will make use of the .NET TCP/IP libraries and that they will handle the work of talking to the OS, which will invoke the drivers, which will use my WiFi hardware, which will talk to my ISP, which will route the communication to the server I requested. The .NET libraries will take care of handling the handshaking to establish a connection, and the sending and receiving the data itself. But I've always wondering how this works. At low level. As in: what does the OS say to Broadcom driver to make the antennae do stuff. What commands does the driver send to the firmware? What (and how?) does the firmware do to make the antenna emit an electromagnetic signal and how does the firmware handle all the electromagnetic radiation it picks up to enable it to listen to the right signal? Another thing that's always bugged me is the nature of the (necessarily) discontinuous communication. It's not a continuous sine wave with a continuous stream of data going to and from. The data is broken into packets and there is acknowledgement and error correction in the packets being transmitted. Which implies some kind of state management going on. Which then raises the question of how does something like SignalR work when there are 10,000 connections (and what is a "connection"?) to a server and the data travels back and forth on a connection that's long term, not short term like a HTTP request (small furrowed brow thinking about HTTP Keep-alive) None of this is important to a developer, and yet it kind of is important to a developer in the same way garbage collection, disk access patterns and thread scheduling. You don't need to know them, but knowing them gives you the knowledge to make informed decisions. Does anyone know of a write-up of this stuff anywhere? Failing that: would anyone be interested in taking a stab at an article that walks through the life cycle of a bit of data travelling from code through the libraries to the OS to the drivers to the firmware to the hardware, via the ISP, across the routers, to the other and and back? I think it would be fascinating.
cheers Chris Maunder
-
Something I take for granted is that my browser will connect to my servers which will connect to other servers in my hosting setup and that there will be a form of communication that will allow data to be requested and received. But I have no idea how it all works at a low level. Zero. I know that my code will make use of the .NET TCP/IP libraries and that they will handle the work of talking to the OS, which will invoke the drivers, which will use my WiFi hardware, which will talk to my ISP, which will route the communication to the server I requested. The .NET libraries will take care of handling the handshaking to establish a connection, and the sending and receiving the data itself. But I've always wondering how this works. At low level. As in: what does the OS say to Broadcom driver to make the antennae do stuff. What commands does the driver send to the firmware? What (and how?) does the firmware do to make the antenna emit an electromagnetic signal and how does the firmware handle all the electromagnetic radiation it picks up to enable it to listen to the right signal? Another thing that's always bugged me is the nature of the (necessarily) discontinuous communication. It's not a continuous sine wave with a continuous stream of data going to and from. The data is broken into packets and there is acknowledgement and error correction in the packets being transmitted. Which implies some kind of state management going on. Which then raises the question of how does something like SignalR work when there are 10,000 connections (and what is a "connection"?) to a server and the data travels back and forth on a connection that's long term, not short term like a HTTP request (small furrowed brow thinking about HTTP Keep-alive) None of this is important to a developer, and yet it kind of is important to a developer in the same way garbage collection, disk access patterns and thread scheduling. You don't need to know them, but knowing them gives you the knowledge to make informed decisions. Does anyone know of a write-up of this stuff anywhere? Failing that: would anyone be interested in taking a stab at an article that walks through the life cycle of a bit of data travelling from code through the libraries to the OS to the drivers to the firmware to the hardware, via the ISP, across the routers, to the other and and back? I think it would be fascinating.
cheers Chris Maunder
Buy a pair of 2$ micro controllers off ebay. Buy a pair of radio ICs (doesn't really matter much, as long as its not BLE or WiFi). Configure radio IC. Send a packet. Be amazed. Spend next 6 months working on a low level network for micros. Worked for me :-\ Sincerely, an EE who skipped all tele-communications classes because they sucked.
