Skip to content

Joseph Mansfield

Don't condition input on eof()

New C++ developers often learn the following incorrect pattern for reading from a file:

while (!file.eof()) {
  // Read data from file
  // Process data

At first glance, it seems to make sense — if I haven't reached the end of the file, I can keep reading from it — but this logic is wrong. The fundamental issue is simple: the EOF bit not being set does not necessarily imply that reading from the file will succeed. This often occurs when looping over lines (or other delimited elements) in a file, in which case an extra iteration occurs and one of two common symptoms is observed:

This problem raises an interesting question. If we know that we've read the last line of a file with std::getline, why does the EOF bit not get set and prevent us from iterating again? Surely we have reached the end of the file. Why would it continue only to fail at the next attempt to read?

The EOF bit is only set if you actually attempt to read anything beyond the end of the file. That is, if you were to call file.get(), which extracts only a single character, reading the last character would not set the EOF bit, but the next call would. This doesn't fully answer the question, however, because some functions actually read one character beyond the data they extract. For example, std::cin >> x, where x is an int, discards any initial whitespace, and then will read up to and including a non-digit. The non-digit is not extracted, but it is read in order to determine that it should stop. Similarly, std::getline will read up to and including a newline character. These functions will also stop if they attempt to read beyond the end of the file. So the question remains — if we're reading the last line of a file with std::getline, why does it not attempt to read past the end of the file and therefore set the EOF bit?

The answer is simple, but not immediately apparent: in general, all lines in a text file should end with a newline character — even the last line (in fact, the POSIX standards require it). In most text editors, this is not obvious because the final newline isn't visible in any way, but it's there (Sublime Text is one exception). There are many reasons to have this newline character, but it has unfortunate consequences for our !file.eof() loop. Consider the following file:

Today I pass the time reading
a favorite haiku,
saying the few words over and over.

If we render the newline characters, it looks something like this:

Today I pass the time reading\n
a favorite haiku,\n
saying the few words over and over.\n

Now, if we're iterating through this file, reading each line with std::getline, what happens when we read the last line? std::getline will read up to and including the final newline character and then return. It won't attempt to read beyond the end of the file. The EOF bit won't be set. If we are conditioned on eof(), we will start another iteration with nothing left to read.

The same is true if we're extracting formatted data with >>. The last extraction will stop at the newline character and the EOF bit will not be set. There is no more data left to extract. In this case, it often appears that the last set of data is duplicated because the objects we are extracting into retain their values from the previous iteration.

Of course, these operations are not specific to files. Any kind of input stream will exhibit the same behavior. The newline character at the end of files is frequently the cause of this problem, but conditioning on eof() is flawed regardless. It simply cannot tell if the following input will succeed or not. Any other I/O problems will also be ignored, since eof() only tells you if the EOF bit is set and not any of the other failure bits. This can lead to infinite loops.

The correct and idiomatic approach to read from a stream is to condition on the success of the read itself. In this way, we attempt to extract the input and handle it appropriately if it fails. This means that you can stop at the right time and report problems to the user if necessary (if the stream data was incorrect). To loop over all lines in a file, for example, you should use the following:

while (std::getline(file, line)) {
  // Process the line

Now we know for certain that the read succeeded before starting a new iteration of the loop. This also protects us against any other I/O problems that might cause it to fail. In fact, this pattern works everywhere, and should be used to validate any kind of input, even if you're not looping over it. For example, if you want to get some details about the user:

if (std::cin >> first_name &&
    std::cin >> family_name &&
    std::cin >> age) {
  // Success
} else {
  // Failure

The &&s ensure that each input operation succeeds before attempting the next one and that all must succeed in order to enter the success branch.

In practice, user input and data files tend to require a combination of these approaches. For example, they are often organised in structured lines. in which case I recommend looping conditioned on std::getline and then parsing each line individually. Inside the loop, you can put the line into a std::stringstream and extract the individual items of data from that. You can validate those extractions with an if statement as in the previous example.

The !file.eof() anti-pattern is still seen often, even in C++ tutorials and other teaching materials. As we have seen, eof() only tells if you've already attempted to read past the end of a file, which is rarely what you really want to check for. Instead, we can easily condition on the success of the extraction, which ensures that we only continue processing if it succeeds. This pattern helps us write robust input handling for all kinds of streams and should be used even when !file.eof() appears to work.