Infinite loop with unbounded memory on a 4-byte UTF-16 file that ends in an identifier character

[CrJyyy]

Summary

A 4-byte file consisting of a UTF-16 BOM plus one identifier character, with no newline after
it, makes simplecpp loop forever at ~100 % CPU while allocating without bound.

$ printf '\xFF\xFE\x41\x00' > poc.c    # UTF-16 LE BOM + 'A'
$ ./simplecpp poc.c
  (never returns)

Measured on the standalone simplecpp binary, sampling /proc/<pid>/status:

t= 20s  rss=0.31 GB  cpu=100%
t= 60s  rss=0.78 GB  cpu= 95%
t=120s  rss=1.63 GB  cpu= 97%
t=180s  rss=2.39 GB  cpu= 98%     (still climbing linearly, ~13 MB/s)

This is adjacent to #637 / #636 but is a different path: #637's exact bytes
(\xFF\xFE\x00\x3B) are fixed and complete instantly on current master; the case above does not.

Version

Reproduced on master at f420e61 ("Fix #688: Support #elifdef and #elifndef (#691)",
2026-08-03), built exactly as the README describes, with no sanitizers and no fuzzing engine:

$ g++ -std=c++11 -O2 -o simplecpp main.cpp simplecpp.cpp

Host: Linux x86-64, GCC.

Trigger matrix

bytes	meaning	result
FF FE 41 00	UTF-16 LE BOM + A	hang
FE FF 00 41	UTF-16 BE BOM + A	hang
FF FE 24 00	UTF-16 LE BOM + $	hang
FE FF 00 3B	UTF-16 BE BOM + ;	ok
FF FE 41 00 00 00	LE BOM + A + NUL	ok
FF FE 41 00 0A 00	LE BOM + A + newline	ok
41	plain UTF-8 A, no BOM	ok
FF FE 00 3B	the bytes from #637	ok (fixed)
FF FE	BOM only	ok

So it needs all three of: a UTF-16 BOM, a final character that continues an identifier, and
nothing after it. A trailing newline avoids it, which is presumably why ordinary files do not
hit this.

Cause

FileStream::unget_internal() rewinds unconditionally in the UTF-16 branch:

void unget_internal(int ch) {
    if (isUtf16) {
        // TODO: use ungetc() as well
        // UTF-16 has subsequent unget() calls
        fseek(file, -1, SEEK_CUR);
    } else {
        ungetc(ch, file);
    }
}

FileStream::peek() calls fgetc() and then hands the result straight to unget_internal():

int peek() override {
    // keep lastCh intact
    const int ch = fgetc(file);
    unget_internal(ch);
    return ch;
}

At end of file that fgetc() consumes nothing and returns EOF, but the fseek(-1) still runs,
so every peek() past the end moves the read position one byte backwards. The reader can then
never reach the end, and readfile() keeps appending tokens — hence both the spin and the memory
growth. The ungetc() branch does not have the problem because ungetc(EOF, …) is a no-op.

Possible fix

Skipping the rewind when nothing was consumed fixes every case in the table above:

--- a/simplecpp.cpp
+++ b/simplecpp.cpp
@@ -463,7 +463,8 @@
             if (isUtf16) {
                 // TODO: use ungetc() as well
                 // UTF-16 has subsequent unget() calls
-                fseek(file, -1, SEEK_CUR);
+                if (ch != EOF)
+                    fseek(file, -1, SEEK_CUR);
             } else {
                 ungetc(ch, file);
             }

With that applied, make testrunner && ./testrunner still exits 0, and all nine inputs in the
table behave as "ok".

I have not checked whether this is the fix you would prefer — you may want to handle it at the
call site instead. While reading that code I noticed something adjacent that looks unintended,
in Stream::peekChar():

if (isUtf16) {
    (void)get();
    const auto ch2 = static_cast<unsigned char>(peek());   // EOF (-1) becomes 0xFF here
    unget();

the second peek() is cast to unsigned char without an EOF check, so a truncated final
UTF-16 unit is silently read as 0xFF. That is not what makes the loop above spin — the
unget_internal change alone is enough — but it may be worth a look.

Also affects cppcheck

cppcheck vendors this file in externals/simplecpp, and the same input reaches it through
CppCheck::check(FileWithDetails) → simplecpp::TokenList::TokenList(filename, …), which is the
path cppcheck <file> uses. I observed the identical hang there (via the OSS-Fuzz cppcheck
target, ASan build): 100 % CPU with RSS at 4.9 GB after 288 s on a 68-byte UTF-16 file that ends
in M. The standalone reproducer above is self-contained, so nothing in this report depends on
that build.

How it was found

Automated fuzzing of cppcheck; the input was then minimised by hand to the 4 bytes above and
re-verified against upstream simplecpp master, so the reproducer involves no fuzzing harness.