What they mostly should know: TCP provides a bidirectional stream of bytes on th... (2024)

> What they mostly should know: TCP provides a bidirectional stream of bytes on the application level. It does NOT provide a stream of packets.

> That means whatever you pass to a send() call is not necessarily the same amount of data the receiver will observe in a single read() call.

Yes, this. For god's sake, listen to them.

I had to fight a coworker on this. I had quickly created some client code just to validate that the server was working. Due to some quirk, all the messages were arriving in full in every read call. He told me to ship it.

I said no! "I need to check if there's more data and if so add a loop to read again" "But it is working, release it". That went on for a while, to no avail. Wouldn't look at documentation either.

Eventually he head to leave for the day, and I took the time to implement it correctly.

I started including basic TCP questions on interviews. Not many people even get past the TCP handshake (if they even know about that).

No, it does not, by design. When you use any standard TCP implementation the abstraction provided at both ends is just a stream of bytes. The guarantee TCP makes is that the bytes received at one end will be in the same order as the bytes that were sent at the other end.

If you want to use TCP to implement some (higher-level) message-based protocol, you need to parse those out for yourself.

The original question didn't refer to such issues of parsing, just checking for whether the full message was received, which, I would think, would be handled by the TCP library read() (or whatever) call. It sounded like the OP (outworlder) was delving into lower-level TCP details that should have been abstracted away -- my thinking is that the TCP caller shouldn't have to concern itself with details about checking whether the full message has been received, at least not as a separate step. That is, it just wouldn't return anything until the full message is received, or would include some data structure that indicates it's not complete. Does that make sense?

Edit: On second thought, I guess OP meant that all of the results were coming back "complete" which doesn't obviate the issue of needing to do a check that handles the "not done" case.

> the TCP caller shouldn't have to concern itself with details about checking whether the full message has been received

The caller absolutely must concern itself with this crucial “detail”. If you do otherwise, your code is broken, full stop.

You can implement a higher-level protocol on top which handles this kind of thing internally and presents a higher-level interface (e.g. not passing any partial data along to its caller until a full message has been received), but if you are just working with TCP directly, what you get is just a stream of bytes. The guarantee you get is that the bytes will be in order and without any gaps.

If you e.g. send UTF-8 encoded text, you must be prepared on the read side to have your stream of bytes cut off arbitrarily in the middle of a character.

Let function A call TCP read() and pass back a struct that includes the bytes it's received and a flag that indicates whether it's read to the end of the message.

Let function B call TCP read() and never return anything until it's received all the bytes of the message.

Both of those seem (IMHO) like functions you could have in a TCP library. Neither seems (IMHO) like a higher level protocol.

The reason that this function doesn't exist is that even if the sender sent a "message" in a single API call, there is no guarantee that the networking code will send it in one IP packet (or one layer 2 frame). We don't want to couple the message size with the lower network protocol, the type of network equipment and so on, and also want to allow merging of messages for more efficient use of the network (If we have a large window size it allows better throughput).

So the job of separating the stream into messages is left to the application layer. (unlike for example in UDP, but then you have to worry about dropped messages)

What is that responding to? Of course TCP can split a message across multiple packets of the lower level protocol; that has no bearing on the concepts TCP works in and whether it can indicate end-of-message.

If your point is just that TCP doesn't have a concept of a "message" (a bytestream with a clear beginning and ending), then that's fair, but, as I said elsewhere [1] the original comment took for granted that TCP does have a well defined notion of "you've reached the end of the message", or at least, "there is no further data to receive". No one seemed to have a problem with that there, and I was just working off that assumption.

As before, I haven't checked whether this is true (I can't quickly verify from descriptions of TCP).

And, interestingly enough, there's this comment [2], which says that what I described does exist, but isn't the default. So ... I'm at least not getting a consistent answer to my question, and people who think they know what they're talking about are inconsistent with each other.

[1] https://news.ycombinator.com/item?id=23200293

[2] https://news.ycombinator.com/item?id=23200906

> the original comment took for granted that TCP does have a well defined notion of "you've reached the end of the message"

No, the original comment was complaining about a coworker who didn’t understand (and refused to listen when told otherwise) that there is no such notion in TCP. It was a response to another comment complaining about people on the internet (e.g. Stack Overflow) too often making the same mistake.

You’re more or less playing the part of that coworker here. It’s unclear why.

> Both of those seem (IMHO) like functions you could have in a TCP library. Neither seems (IMHO) like a higher level protocol.

It's responding to the last part. It is a higher level of protocol. In the traditional TCP/IP model, it's in the application level. There are many libraries with an API like you asked, they are just in a higher level.

(and TCP_WAITALL is a partial solution, applicable only if you know in advance the exact size of the message you are about to receive)

> which, I would think, would be handled by the TCP library read()

You would be wrong.

Sorry, the rest of the conversation seemed to be assuming that "whether you have received the full message" is well-defined at the level of TCP, as suggested by the original comment where I joined[1]:

>I had to fight a coworker on this. I had quickly created some client code just to validate that the server was working. Due to some quirk, all the messages were arriving in full in every read call. He told me to ship it.

>I said no! "I need to check if there's more data and if so add a loop to read again" "But it is working, release it". That went on for a while, to no avail. Wouldn't look at documentation either.

I was just going along with that.

I didn't check up on TCP further to verify whether this was actually true; if you are saying that's not a well-defined concept, you might want to reply to that comment to say so.

[1] https://news.ycombinator.com/item?id=23195230

Of course, but they do indicate that "there is still more of the message to receive" is a well-defined concept (or at least "there is more data to receive" is), which is all my point requires.

(Edit: also, it would help if you said what the correct interpretation would be, since it has the phrase "messages were arriving in full in every read call".)

>but they do indicate that "there is still more of the message to receive"

Not at the TCP level they don't.

They just give you some bytes. It's up to you to decide whether you have the full message or not and if you want to try to read more. The TCP read functions just give you the data they have. There is no concept of one write at the sender end translating to a complete message at the other end. It's just a stream of bytes.

They do... with an option called MSG_WAITALL.

That naturally leads to the question of why it's not the default, which can be answered by understanding the history of TCP and computer networking; and more interestingly, how might things have been different if MSG_WAITALL was the default from the beginning.

If you're using a library, then sure. But if you're just reading from a raw TCP socket, it's just a stream of bytes. It's up to your application to parse those bytes (e.g. into a http request).

The OS will buffer bytes received from TCP packets for you until you read from the socket again to drain the buffer. Your application needs to determine how to semantically chop those bytes up into the protocol it's expecting (e.g. http request).

My low-level networking chops are a little rusty so please correct my understanding if I'm off-base somewhere.

"But it is working, release it".

Famous last words :-)

> the rest of the conversation seemed to be assuming that "whether you have received the full message" is well-defined at the level of TCP

I don't see anything in the comment you linked that implies that.

Sorry you got confused, but there's no need for anyone to go reply to that comment to say so.

In the fall of 2016 I had a lengthy email exchange with an industrial automation vendor who didn't understand this issue. I even mailed them a short Python proof-of-concept snippet that slept a few milliseconds between the write() calls and in response got back my code "fixed" with the sleep removed.

In between the emails I googled a bit and found the changelogs for the RTOS they were using. Turned out that it was a bug in the upstream HTTP server. This also meant that the platform they were using had all the security holes from those five-plus years. The bug was later silently fixed when they acquired a newer release from upstream.

Currently I'm having a similar issue with the very same vendor. This time they don't understand why client-side authentication means no authentication at all and why passwords must not be stored in plain text in the database that can be remotely backed up from the device.

But in the meantime, won't the vendor keep adding more broken devices to the public internet, making the problem worse?

The longer it takes for this problem to become public, won't the more harm be caused when it does become public?

> This time they don't understand why client-side authentication means no authentication at all

I've seen this... with an intern! I can't imagine dealing with a whole team like that.

I would assume the “free market” here is that these companies will over-extend themselves so much that they will no longer be able to hide the bugs from the malicious parties and their devices will start getting hacked en masse.

I would assume, however, that there is no law forcing minimal security so you can class A them, can you?

In inverse order:

Because nobody gives a sh*t about quality unless it hits their paycheck.

"Onnnngggg. They pay me hourly. Onnnngggg."

They cut lots of checks.

Yeah. Fun problem for beginners, because 1) your incorrect code may work for a while when reads/writes are small or it's only run on a local network or such, 2) you might design a broken protocol if you don't understand fragmentation, etc., which will tend to be harder than (say) an isolated client bug to fix, 3) the implementation-dependent nature of fragmentation can make it look like you hit a language/library/OS issue, 4) your language/library may or may not offer tools to help a beginner to implement a delimited or framed wire format properly (ideally with things like record-size limits and timeouts).

Not sure it says anything you haven't, but a StackOverflow answer on fragmentation (framed by asker as Go not behaving like C) is one of the more-read ones I've written: https://stackoverflow.com/questions/26999615/go-tcp-read-is-...

A version of Microsoft Exchange had a bug in its SMTP implementation that was tickled when lines crossed packet boundaries. (EDIT: The issue was more likely a bug in Exchange's TLS record processing, breaking when a logical line crossed TLS records.) My async SMTP library used a simple fifo for buffering outbound data which didn't realign the write pointer to 0 except when it was completely drained, so when reading slices (iovec's) from the fifo for write-out it would occasionally call write/send with an incomplete line (i.e. part of a line that wrapped around from the end of the fifo buffer array to the front) even if the application had only written full lines. (At the time it didn't support writev/sendmsg, though I'm not sure it would have helped as the TLS record layer might still have been prone to splitting logical lines across packets.) There was no bug here on my end--everything would be sent correctly--but you can't tell the customer that he can't send e-mail to some third-party because that third-party is using a broken version of Exchange.

The first quick fix was to unconditionally realign the fifo contents after every write (the fifo had a realign method), but that ran into a computational complexity problem when you had lots of small lines (e.g. the application caller dumped a huge message into the buffer and then flushed it out in one go) and a high-latency connection that resulted in many short writes; you were constantly memmove'ing the megabytes of remaining contents in the buffer for every tiny write you did. So then I ended up having to add a new interface to the fifo that returned a slice up to a limit but always ending with a specified delimiter (e.g. "\n") if the delimiter was within the maximum chunk size.

Of course, none of these fixes would have completely remedied the issue as lower layers (the TLS stack, the kernel TCP stack) could have still potentially split logical lines, and I'm sure did on occasion. But it at least seemed to put us on equal footing with everybody else in terms of how often it happened, which is really the best anybody could have done. Complaints did die down.

I have often wished for an "unhelpful testing environment" of sorts, to deal with these things before they get out of hand. It would feature a compiler that had creatively different interpretations of undefined behaviors, randomly compile against glibc and musl, have a base OS lovingly crafted from Ubuntu, but with most coreutils replaced with busybox and/or BSD versions. And, now, I suppose, it would have a customized network stack (kernel module?) that would randomly reorder/drop/duplicate packets, randomly reselect MTU on every boot, or maybe just randomly fragments things regardless of MTU. Ideally it would come with a FAQ of "my program broke on X; what did I do wrong?".

The idea being that if your software is actually written to relevant standards, and actually handles things properly outside the golden path, then it should still work fine. If, however, you accidentally did something implementation-defined, or that only worked by coincidence, this system will break it.

There are tools that intentionally insert failures into the network streams of applications. A few of them are described here: https://medium.com/@docler/network-issues-simulation-how-to-...

The other linking/OS problems can probably be automated with some simple integration tests and a bunch of different docker containers to compile the code in. Should be possible to squeeze it into a CI/CD flow somewhere with some clever tricks.

I created such an environment for my unit-tests: Wrapping TCP sockets in a stream which only accepts 1 byte at a time in both directions and returns EAGAIN on every second read provides an easy way to make sure the code on top of the socket does perform all the correct retries.

That will most likely not help newcomers which directly write their code agains the OS socket. But once you get a better understanding of the topic and start adding tests to your codebase it's rather easy to add.

> stream of bytes

I've always wondered: What's the best/defacto way to delimit this back into packets at the application level on the receiving end?

I would think the obvious approach would be to insert some magic word into the stream so that you can re-sync.

Or is this not an issue since you know that once you're connected, you'll never drop a single byte, therefore, the only way to get out of sync would be a program error?

You will never drop a single byte.

If you need some packet-oriented messaging, you could use something like http://jsonlines.org/ (i.e. JSON messages separated by newline characters), or https://github.com/protocolbuffers/protobuf if it's more performance-critical.

Protobuf isn't self delimiting so you still have to have some extra packet wrapper around it to say the length.

I like zeromq to get to a packet based system.

The standard way is to include explicit information on the length of the message that is following.

For example if the message is x bytes long then you first send 'x' then you send the x bytes of the message.

Or your messages have a defined header that contains the length of the message payload.

It will never get out-of-sync because TCP guarantees that the bytes will be delivered in the same order they've arrived.

The best approach is typically put a length in front of every message. The good things about that approach are:

1. The receiver can allocate buffer that is exactly the size it needs to fit the message.2. The receiver can check whether the message is too long before seeing the entire message.

The only disadvantage is that you have to know the length of all messages in advance.

I mean, at this point you're effectively defining a new network protocol (or, you will be shortly once you implement ways to work around all the other issues you're going to run into). I'd go all-in from the start and start every packet with a magic string/byte sequence of your own, a length, and probably a version code just to make it extensible.

Or see if there's an existing protocol you can abuse for what you want. If it's transactional, you get a pretty big ecosystem of battle-tested clients/servers/proxies/etc if you use HTTP.

Good advice. I was actually referring to this: https://rachelbythebay.com/w/2016/02/21/malloc/ I read this article a long time ago, and yet I made a similar mistake in my own code.

Netstrings[1] :)

[1]: https://en.wikipedia.org/wiki/Netstring

preamble of chunk length and 1 bit for end-of-message indicator.. if you only do chunk length you will eventually find you can't stream but want to.

or just use http.

I suppose I should say something.

The thing to turn off is delayed ACKs. See "TCP_QUICKACK". Delayed ACKs were a feature which is only useful for things like Telnet, where the payload in each packet is one character when the user is typing. The fixed timer for delayed ACKs is for keyboard typing speeds, and for networks so slow that human typing could congest them.There's a reasonably good explanation here.[1]

As others said above, TCP is not a message protocol. It's a stream protocol. If you're sending messages over a stream, you need something that's reading data from the stream, and when it has a full message, it send that off to be processed. There is no set of TCP options which will reliably cause one write at the sending end to result in one read at the receiving end. If there were, it would be inefficient for small messages and would fail for large ones.

[1] https://www.extrahop.com/company/blog/2016/tcp-nodelay-nagle...

Your terminology is a little off. TCP does not provide anything for the application layer as it is transport layer. The application layer rides on top of that. Examples of transport protocols are TCP and UDP while application protocols are things like http, ssh, irc, and all those things your applications use.

The network layer on which the transport layer rides is packet switched. The TCP uses segments with each segment having its own header and sequence numbers. Streams are just a series of segments populating across a single established handshake without a prior defined termination segment.

> It was more about: [user-space] applications which use the TCP/IP stack do not observe packet boundaries

That is still a bit imprecise. Userland applications won't directly see TCP as they are just looking at an application protocol. Typically it's the OS that packages and unpacks the application protocol data into a TCP segment, so of course the userland application won't see it since its not managing that part of the communication.

https://en.wikipedia.org/wiki/Transmission_Control_Protocol#...

There are some exceptions where some application platforms allow developers to write custom TCP protocols, such as Node.js, but these exceptions generally apply to network services and don't commonly apply to the end user application experiance.

https://nodejs.org/dist/latest-v14.x/docs/api/net.html#net_n...