C Strings and my slow descent to madness

https://www.deusinmachina.net/p/c-strings-and-my-slow-descent-to

by Decabytes • 3 years ago

145 322 3 years ago

bluetomcat2• 3 years ago

In well-written C, you don't work with strings the way you do in other HLLs. For example, extracting and copying substrings is something unnecessary, unless you want to modify the parent string. Otherwise, a substring is represented by a pointer and a size_t length, and can easily be printed that way via the "%.*s" printf specifier:

    const char *s = "Hello World!";
    const char *world = s + 6;
    size_t world_len = 5;
    printf("%.*s\n", world_len, world);

gpderetta3• 3 years ago

On other HLLs it is easy to have subviews on other strings. C makes is needlessly hard by requiring null termination in half the APIs.

tom_3• 3 years ago

* consumes an int, not a size_t: https://port70.net/~nsz/c/c11/n1570.html#7.21.6.1p5

TremendousJudge• 3 years ago

I love how in every C code snippet on every comment on this thread, somebody got something wrong. I take it as a sign that it's probably best to avoid C as much as possible.

lolcatuser5• 3 years ago

Maybe my least favorite "feature" of C. I can manage most aspects of zero-terminated strings well enough, but when I have to specify the length of them, is it an 'int', 'size_t', 'ssize_t', or something else? (Answer: All of the above!)

coldpie2• 3 years ago

It's unfortunate the author put the arrays-are-pointers thing so early in the doc, as that's a very beginner-to-C mixup and really nothing at all to do with strings. Otherwise, yep. It's pretty bad. C is a great language, but its string handling is definitely garbage. You get used to it pretty quick, and it's not hard to write a handful of sane wrappers or a simple string library for your own use, but the standard library's terrible string functions are an unending source of bugs.

gpderetta3• 3 years ago

I don't see any mention or insinuations of arrays-are-pointers anywhere in the article. Am I missing something?

coldpie2• 3 years ago

This bit:

    But you might be asking. “Why can’t I just assign the source variable directly to the destination variable?”

    int main() {
      char source[] = "Hello, world!";
      char* destination = source;
    
      strcpy(destination, source); // Copy the source string to the destination string
    
      printf("Source: %s\n", source);
      printf("Destination: %s\n", destination);
    
      return 0;
    }
    You can. It’s just that destination now becomes a char* and exists as a pointer to the source character array. If that isn’t what you want them this will almost certainly cause issues.

unwind• 3 years ago

This is almost a cliche among many C language lawyers and/or Stack Overflow answer-rich people and I know you mean well, but: arrays are not pointers.

In some contexts, the name of an array decays to a pointer to its first element. That is a better way of putting it, and it's a (much) weaker statement.

Edit: if they were the same, this code:

    int foo[] = {1, 2, 3};
    int *bar = foo;
    printf("%zu and %zu\n", sizeof foo, sizeof bar);

Would print the same valde twice, but it doesn't. On Ideone [1] I got 12 and 8.

[1] https://ideone.com/CP7WTu

int_19h6• 3 years ago

This also makes a big difference once we start talking about pointers to arrays.

    int a[] = {1, 2, 3}
    int (*p1)[3] = &a; // ok
    int (*p2)[3] = &a[0]; // not ok
    int *p3 = &a; // not ok

(It should be noted that these will compile with warnings in C due to implicit conversions via void*, but you're still risking UB if you actually use the resulting value. They are all errors in C++ because it doesn't have implicit conversion from void*.)

marcodiego• 3 years ago

If we try to print out some Japanese characters… [] The output isn’t what we expect.

Yes it is. And I bet on a modern windows version it is too. The terminal has been (probably intentionally) neglected by ms for a long time, but as far as I know this has mostly been fixed on modern windows versions.

EDIT: Author admits it later in the text "will be fixed in Windows 11 and Windows Server 2022"

Also it says "strlen("有り難う")); [...] and the output is… The length of the string is 12 characters". But according to "man strlen": "RETURN VALUE: The strlen() function returns the number of bytes in the string pointed to by s.". It says nothing about "number of characters".

kevingadd• 3 years ago

It makes sense to point it out even if it's fixed in win11, lots of people (myself included) are still on 10.

dspillett• 3 years ago

The terminal has been (probably intentionally) neglected by ms for a long time,

I don't think it is an intentional lack of care, just a lack of care. Internally MS devs affected by the appalling state of the console just did what the rest of us did and installed an alternative.

but as far as I know this has mostly been fixed on modern windows versions.

Ish. The default console for powershell is better, but a lot of improvements you might be thinking are in there are in fact only in Windows Terminal (https://en.wikipedia.org/wiki/Windows_Terminal) which is not currently included by default.

cmovq• 3 years ago

you might be thinking are in there are in fact only in Windows Terminal

A lot of those changes are in ConsoleHost, so Windows 10 and 11 get those improvements (like VT100 sequences) in cmd.exe as well

nordsieck• 3 years ago

Also it says "strlen("有り難う")); [...] and the output is… The length of the string is 12 characters". But according to "man strlen": "RETURN VALUE: The strlen() function returns the number of bytes in the string pointed to by s.". It says nothing about "number of characters".

Yeah - when dealing with Unicode, you have to be very clear about whether you're dealing with bytes, runes or glyphs.

riceart• 3 years ago

Runes are not a Unicode concept - that’s a Golangism. Basically a code point.

Also in terms of Unicode, graphemes are even more relevant to the programming side than glyphs - unless you’re writing a renderer.

flohofwoe6• 3 years ago

And I bet on a modern windows version it is too.

It's still broken unfortunately, you need to switch the console to a special UTF-8 codepage in your own code:

    SetConsoleOutputCP(CP_UTF8);

...and before exit restore it to the original code page.

st_goliath• 3 years ago

... except that that is also subtly broken.

It works if you write multiple UTF-8 code-units in one go, but breaks if you send them in several writes (and by that, I mean direct writes to the HANDLE). It also breaks if you try to use the ANSI API (with the A suffix), as it internally tries to convert the bytes from codepage-random to UTF-8.

You run into both issues if you try to use the MS implementation of stdio (printf and friends).

And we didn't even discuss command line argument passing yet :-)

I had a lot of fun with this (more explanation in the issue comments): https://github.com/AgentD/squashfs-tools-ng/issues/96#issuec...

I tried to test it with the only other two languages I know besides English: German and Mandarin. Specifically also, because the later requires multi-byte characters to work. Getting Chinese text I/O to work at all in a Windows DOS box, on an existing, German Windows 7 installation was an adventure on it's own and ended up breaking things in different ways than German text.

Turns out, trying to write language agnostic command line applications on Windows is a PITA.

jandrese3• 3 years ago

Honestly if you are expecting a sane and modern text console on Windows you're just begging for disappointment. I note that even the author briefly tries it on Bash and finds it to be undramatic.

_benj4• 3 years ago

With the woes of string.h being known, why not just use an alternative like https://github.com/antirez/sds ?

I’ve also been having a blast with C because writing C feels like being a god! But the biggest thing that I like about C is that the world is sort of written on it!

Just yesterday I needed to parse a JSON… found a bunch of libraries that do that and just picked one that I liked the API.

felipemnoa2• 3 years ago

>I’ve also been having a blast with C because writing C feels like being a god

Not trying to be a troll but as someone who has also written a lot of C in the past why do you feel like this?

t435623• 3 years ago

It's not doing as many things behind your back as the dynamic languages and C++ do. More things are your responsibility.

_benj4• 3 years ago

It’s the access and control that it gives me!

As when I’d pick Go because I was doing some concurrency, I can now explore a bunch of concurrency libraries, including some implementations that look a lot like Go channels. Want to watch a file for changes? I can do that all the way from taking to the kernel to picking a multi-platform library. I guess, I haven’t really found anything that I can’t do in C, and if I’m lazy, I can just embed and scripting language to handle things in a higher level. Macros are also very powerful! I’ve been writing code that writes code for me, export the thing to a .h file and import it using #include.

pkg-config —list-all has become my friend and I keep discovering that the world is written in C and the access to libraries is huge!

Also, idiomatic C is whatever you make it. There are a bunch of ways to skin a cat. Want a different platform? Build tool? Compiler? Debugger? Wanna write you own debugger? C is chill with that.

It’s also such a simple language that without much effort you can know everything about it (I don’t care that much about anything over C99). I don’t know the whole ecosystem, or standard libraries, or data structures and algorithms and whatnot, but the language itself is quite trivial.

With that said, I’m not using C in project teams. In that setting some strong conventions would likely be necessary or even better, something enforced by tools or the compiler (like Go), but yeah, I’ve been quite enjoying working with C and being kind of annoyed at other langs that I need to use for work because they keep doing all this stuff behind my back that is supposed to help me, but instead is a pain trying to debug and understand what is actually happening

t435623• 3 years ago

Yes, I do think that you cannot fairly dismiss C because of strcpy_l only being available on Windows when it's quite possible to implement it yourself or use a library like the one you mention.

gavinhoward2• 3 years ago

I encourage people to use sds if that's the best option.

However, I don't think that's the best option if you can roll your own.

I personally think that there should be two different types of strings: static and dynamic ones. The static ones should not be able to be changed, but the dynamic ones can serve as a "string builder" type of sorts.

Second, I don't see sds's first advantage (in the README) to be an advantage. Sure, you may have to explicitly pass the buffer in to C functions, but that tells you that you're calling a function that takes a char array rather than your string. It makes it more explicit.

Second, if you use my method of splitting static strings from dynamic strings, then sds's second advantage doesn't apply because the pointer will never change.

But the disadvantages of sds still apply, and both disadvantages are big since they easily lead to bugs. Hence, I think sds is not the best option if you can make your own.

Oh, another advantage of the static/dynamic split: I can implement small string semantics. For small enough strings, I use a union to put the array into the same bytes as the pointer, so on 64-bit machines, I can have 8-byte strings (including nul) before needing an actual allocation.

kerkeslager• 3 years ago

With the woes of string.h being known, why not just use an alternative like https://github.com/antirez/sds ?

That library really doesn't address any of the Unicode issues.

UncleEntity• 3 years ago

Was going to say the same thing.

If you want Unicode in C a wrapper library is pretty much a given.

When I was adding Unicode support to the small scheme interpreter I like playing with I found a super simple string library, a bunch of generated code (who doesn’t love 1000 line switch statements) for dealing with utf-8 code points and bob’s your uncle. Could have probably found a library that did it all but the goal was learning and yak shaving.

Haven’t ever messed with the wide strings, seems like more of a hassle than they’re worth.

kelnos3• 3 years ago

I’ve also been having a blast with C because writing C feels like being a god!

That's funny, because I look at it in the opposite way: it makes me feel like a super-fallible human because it's so easy for me to break things in horrible ways, something a hypothetical god would not do.

Something like Rust &str or String would make me feel more like a god, as I can do whatever I want (more or less) without worrying about safety.

lolcatuser5• 3 years ago

A hypothetical god would not destroy the world, but absolutely could.

I think that's the idea - they're not reveling in the fact that they can do anything anybody could reasonably want to do, they're reveling in the fact that they can also do everything else, too.

jmclnx• 3 years ago

Yes this is something to get use to. The BSDs created strlcpy(3) and wcslcpy(3)

https://man.openbsd.org/strlcpy.3

https://man.openbsd.org/wcslcpy.3

which to me will help with some of these issues. Too bad other Operating Systems do not have these. On Linux there is libbsd to get these, but I would like to see these to be added to the stdc.

Instead the c23 standard is messing with realloc(3) which could break some old programs. I have not looked at that in detail yet, so maybe it is a non-issue :)

torstenvl3• 3 years ago

I don't know of a compiler that forces you to use the newest version of the standard, which is why I've always kind of thought "don't break old code" was treated too much like dogma. So from that perspective, a non issue.

However, there is a problem that has nothing to do with old code: they increased the number of situations that constitute undefined behavior, with no public discussion and no justification. It's frankly dangerous behavior.

alecco• 3 years ago

Ushering out strlcpy() https://lwn.net/Articles/905777/

GabrielTFS• 3 years ago

These functions are in the current POSIX draft - though not published, it's quite unlikely to be removed (someone actually specifically filed an issue against POSIX to try and get it removed, basically on the basis of "it's not perfect so it should be removed", and the issue got rejected on the basis that there's no consensus for removal, and it seems unlikely this will change), and as a result, the functions are getting added to glibc: https://sourceware.org/pipermail/libc-alpha/2023-April/14696...

anthomtb3• 3 years ago

strlcpy is nice due to the guaranteed NUL termination.

strlcpy is not so nice due to the strange (IMO) return value of the number of characters in the source string. Which could be the number of characters copied or much, much larger than the number of characters copied. snprintf does the same thing.

So using strlcpy is safe (by C's low bar) but using the return value may be highly unsafe.

jandrese3• 3 years ago

The thing that annoys me the most about strlcpy is that it is supposed to be safer, but what happens in the case where the source string is not properly NULL terminated? You might think that it will stop at the character limit you specified, but that's not what it does. It just blows on past the end of the buffer looking for a \0 until it either finds one or causes a segmentation violation.

IMHO I would like it much more if the return values were:

  0: string copied
  1: string partially copied but truncated
  -1: Error, errno set.  This can occur when src or dst are NULL.

hgs33• 3 years ago

Yup, there is also Linux's strscpy which doesn't require reading memory from the source string beyond the specified "count" bytes and the return value is idiot proof.

userbinator2• 3 years ago

Well-written C tends to minimise string usage in general, preferring to convert to another format as soon as possible. Allocating, copying, and passing around strings in large quantities is not a good idea for efficiency, but of course some people coming from other HLLs seem to try to do it anyway, which causes many other problems.

bell-cot• 3 years ago

THIS.

And programming, engineering, and life in general have so, SO many other situations where "X is not very good at doing Y". Yet (my experience) guys seem extremely resistant to the common-sense strategy of "then try to minimize how much Y you do with X".

agumonkey• 3 years ago

to the point I often wonder if strings should exist.. buffers -> symbols | structs.

gaws2• 3 years ago

preferring to convert to another format as soon as possible.

Like what?

userbinator2• 3 years ago

A structure of binary fields.

bobajeff• 3 years ago

I've been wondering lately why many people write c in c++ rather than just c. I think this might be the reason.

qsort2• 3 years ago

People write C in C++ because they don't actually know C++ and think it's "basically C with classes and strings".

There are legitimate reasons why someone would rather write C, but "I don't understand RAII" is not one of them.

owlbite• 3 years ago

C with basic templates is normally what I want. Occasionally other C++-isms drift in, but normally to the harm of the code quality.

wruza2• 3 years ago

C++ doesn’t require you to commit to all of its features and/or paradigms. Using it as you see fit is valid. Just don’t advertise yourself as a C++ programmer to the job market, as it’s not what most people expect.

There’s nothing wrong with “C with classes and strings” idea by itself, if that is your choice or a consciously sufficient level of competence.

tjoff• 3 years ago

For a long time on windows the official MS recommendations was to use the c++ compiler for c projects.

Which was exerbated by an obsolete c compiler that only supported c89.

zabzonk5• 3 years ago

the usual accusation is that many people write c++ code as if it were c. also, the the code in 2nd ed of K&R was all compiled with stroustrup's c++ compiler, as there wasn't a c compiler that could handle it.

infradig• 3 years ago

I stopped when I read strcmp returns 0 if two strings are equal and 1 if they aren't.

tom_3• 3 years ago

A much better description of strcmp's behaviour: https://en.cppreference.com/w/c/string/byte/strcmp

t-3• 3 years ago

It's actually 0 if equal, positive if greater, negative if less than.

  > The strcmp() and strncmp() functions return an integer greater than, equal to, or less than 0, according to whether the string s1 is greater than, equal to, or less than the string s2.  The comparison is done using unsigned characters, so that ‘\200’ is greater than ‘\0’.

jstimpfle2• 3 years ago

cmp stands for compare, so the behaviour (returns <0, 0, or >0) is completely reasonable. With three possible outcomes, the function is suitable to be used for sorting.

jhatemyjob• 3 years ago

It's sad how often you find the truth at the very bottom of a HN thread these days

gavinhoward2• 3 years ago

Okay, I agree that by default, C strings are bad.

But it doesn't have to stay that way. Someone else in the comments mentioned antirez's sds library for dynamic strings. This works, but you could also easily roll your own. All you need is an init function, and perhaps an assert or other check at the end of it that the string has a nul terminator.

At that point, type checking will let you blindly pass those strings (or their char arrays) to any of those C functions without worry.

Edit: I'll also add that I think a string library should have a difference between static strings and string builders (dynamic strings). It makes everything easier.

magicalhippo4• 3 years ago

by default, C strings are bad.

C strings aren't bad. They can't be, because they don't exist. C doesn't have strings. And that is the issue.

As you say, things get a lot better when you actually introduce strings as a concrete concept rather than a set of lose conventions.

junon3• 3 years ago

There is no "loose convention". A C string is a null terminated string of non-null bytes. That's the definition. Working with them in memory-unconstrained environments is unnecessarily hard.

flohofwoe6• 3 years ago

C doesn't have strings.

C has string literals though, and those bake a specific string representation into the language (of course libraries can use their own string representation, but those then need at least some conversion function from string literals).

tialaramex3• 3 years ago

I don't think it's useful to pretend C doesn't have strings when it has string literals.

WUFFS doesn't have strings. That's what a language which doesn't have strings looks like, you can't write "Hello, world" in WUFFS because it involves Strings, which WUFFS doesn't have, and I/O, which WUFFS also doesn't have.

A pretence that C doesn't have strings because it lacks a concrete string type in the language itself also seems like you'd be claiming C++ doesn't have strings, Zig doesn't have strings, and Rust came pretty close to not having strings (for a while it was mooted to make Rust's str just a slice [u8] but today Rust does bless str as a distinct type even though e.g. &str and &[u8] aren't very different)

junon3• 3 years ago

There's also RapidString, though the original author seems to have disappeared off of the internet. Would be curious to see benchmarks against sds.

bluetomcat2• 3 years ago

The best way to write C is to treat strings as memory locations with characters and nothing more. Every such memory location has an allocated size, either statically at compile-time (with literals and arrays), or dynamically with malloc and friends. Treat string operations as mere memory operations and don't imagine them to be something else. The str* functions from the standard library are just convenience helpers which are not adding "string" functionality whatsoever.

nerdponx2• 3 years ago

Maybe you could summarize this by saying that C strings are "strings of bytes", not "strings of characters".

FpUser2• 3 years ago

C strings are bad for sure. Consider those raw assembly. Instead of using it directly get some decent string library ASAP and use it exclusively.

jstimpfle2• 3 years ago

They are just arrays, like everything else in the language. If you don't want to manage plain arrays, better look for a different language.

FpUser2• 3 years ago

You are free to make good use of strings as arrays. I've written tons of code including firmware for MCU so I think I'll keep to my own practices.

throwawaymaths• 3 years ago

they are not "just" arrays, they are arrays with a null-terminated expectation. Most of the time. The lack of consistency and difficulty communicating the expectations is what is hair-pulling in C, and that's on top of the difficulty of communicating the difference between an array and a pointer in C.

habibur2• 3 years ago

I don't use null terminated strings. ptr+len struct everywhere. And when I need to call an API, like fopen, I make a temporary copy of that string + the null termination, do my work and then free it.

You can printf non-null terminated strings too. Check printf("%.*s", length, strptr).

torstenvl3• 3 years ago

You can printf non-null terminated strings too. Check printf("%.s", length, strptr).*

I haven't checked yet, but I'm about 90% confident that's UB. Is printf() guaranteed not to read to the end of the string when you give it a length?

nwellnhof• 3 years ago

Yes, it is. From the C11 spec:

Characters from the array are written up to (but not including) the terminating null character. If the precision is specified, no more than that many bytes are written. If the precision is not specified or is greater than the size of the array, the array shall contain a null character.

habibur2• 3 years ago

Won't read a single byte beyond the length you give it.

And this is a standard practice in libraries for printing pre-determined lengthed string.

t435623• 3 years ago

A long time ago my solution was ptr+len but I allocated 1 more byte so that if a string had to be given to libc, I could terminate it at that time. No need for a copy then.

wvenable• 3 years ago

BASIC strings in Windows -- you store the length in the 4 bytes before the pointer to the string and put a null terminator at the end.

https://learn.microsoft.com/en-us/previous-versions/windows/...

stephc_int1316• 3 years ago

If you are using C and do some non-trivial work with strings you should either use a good library to handle strings or build your own.

It is not that difficult in practice.

The old C std lib is, in my opinion, outdated, obsolete and a very bad fit for complex string handling, especially on the memory management side.

In my own framework, the string management module is using a dedicated memory allocator and a "high level" string API with full UTF8 support from the start.

As a general rule, I think that the C std lib is the weakest part of the C language and it should only be used as a fallback.

tyingq• 3 years ago

Fair, though there's always the crossover point where you need to interact with the OS, 3rd party libraries, protocols, etc. It's not difficult to miss a spot where your utf8 string gets mangled, truncated, etc.

stephc_int1316• 3 years ago

Yes, do not trust the OS, use the minimal API surface, and build your own toolkit, you won't have to do it often.

maxloh• 3 years ago

Is your framework open sourced?

stephc_int1316• 3 years ago

Not 100% decided yet, but it is very probable that I will open source it.

attractivechaos• 3 years ago

especially on the memory management side.

Libc string functions don't manage memory. They can be used no matter where your strings are stored. It is more of a choice between generality vs convenience in common cases.

nicoburns• 3 years ago

A lot of them require the string to be null terminated rather than taking a length.

stephc_int1316• 3 years ago

I think this is the main culprit of the libc string functions, you have to provide buffers to store results, and the responsibility of managing those individually can be annoying, and bug prone, resulting in vulnerabilities.

Passing an allocator (like Zig) or a container (like in my framework) to anything that needs to allocate some memory to store a result is both explicit, low overhead and quite convenient in practice.

skitter• 3 years ago

You cannot e.g. store a string as a slice of another string (unless the slice reaches the end).

marssaxman3• 3 years ago

the C std lib is the weakest part of the C language and it should only be used as a fallback.

I've been musing for a while now: what would it look like if we were to discard the C library and design a new one, leaving the language itself intact?

stephc_int1316• 3 years ago

I think it could be very nice.

C is not perfect, there are some parts of the syntax that I strongly dislike, like casting or function pointers declaration...

But it is overall a good enough syntax, much simpler than C++.

klodolph3• 3 years ago

There are several libraries or projects where people have done exactly that.

You often end up with some kind of structure, or variations of structures, for strings:

    struct string {
      size_t length;
      char data[];
    };

    struct string {
      size_t length;
      size_t alloc;
      char *data;
    };

Those are just examples. The tricky part is figuring out the different ownership use cases you want to solve. Because C gives you so much freedom and very little in the standard library, you end up with a lot of variations. You might use reference-counted strings, owned buffers, or string slices, etc. You might want certain types to be distinguished at compile-time and other types to be distinguished at run-time.

An example can be found in the Git source code.

https://github.com/git/git/blob/master/strbuf.h

The history of changes to this file is interesting as well. This is a relatively nice general-purpose string type—you can easily append to it or truncate it.

oneshtein• 3 years ago

You can backport Rust standard library to C using https://github.com/eqrion/cbindgen .

athenot• 3 years ago

The old MacOS (pre-X) did just that. Strings were all "Pascal strings", ie. with the first byte containing the length of the actual string.

Building blocks for memory were also very different from stdlib, notably the use of Handles, which were pointers of pointers, so that the OS could move a block of data around to defragment the heap behind your back without breaking the memory addressing.

epr• 3 years ago

The creator of this library (antirez) is a regular here on hn.

I believe this is used by Redis.

https://github.com/antirez/sds

WorldMaker2• 3 years ago

Glib answer (but also relevant because mentioned in the article, too): it would look a lot like how a lot of people write C++.

scythe2• 3 years ago

The problem is that it's just too tempting to write something like

    my_function(my_var, 3.6, "bzarflo", my_other_var, false);

The string handling functions are part of the story, but the null-terminated char * is produced when the compiler reaches a string literal, and writing code without being allowed to just use string literals when it's convenient tends to feel like coding with oven mitts on.

rychco• 3 years ago

This caught my eye the other day & looks quite promising, though I haven't spent much time looking at it so I can't comment on it's memory safety:

https://github.com/tylov/STC

quelsolaar4• 3 years ago

I'm in the WG14 and my opinion is that there isn't one good way to do strings it all depends on what you value (performance/ memory use) and the usage pattern. C in general only deal with data types, north their semantic meaning. (IOW We say what a float is not what it is used for). The two main deviations from that are text and time and both of them are causing us a lot of issues. My opinion is that writing your own text code is the best solution and the most "C" solution. Te one proposal i have heard that i like is for C to get versions of functions that use strings that take an array and a length, so as to not force the convention of null termination in order to use things like fopen.

tristan957• 3 years ago

GLib is an alternative to the standard library.

m463• 3 years ago

The old C std lib is, in my opinion, outdated, obsolete

...and has been since most of us ever used C.

I think one of the major failings of C was the lack of a good standard library that updated with the times.

Actually, I believe a rich standard toolbox was one of the best features of python, and helped with its success.

pjmlp2• 3 years ago

Which is why most applications ping back into POSIX when available, not that fixes the security issues with the standard library.

1vuio0pswjnm76• 3 years ago

"If you are using C and do some non-trivial work with strings you should either use a good library to handle strings or build your own."

   unsigned int str_len(const char *s)
   {
     register const char *t;
   
     t = s;
     for (;;) {
       if (!*t) return t - s; ++t;
       if (!*t) return t - s; ++t;
       if (!*t) return t - s; ++t;
       if (!*t) return t - s; ++t;
     }
   }

I still use this instead of stdlib strlen. Of course I also use software everyday that I know uses stdlib strlen. For most C programs dealing with strings I just use flex and yyleng, which in turn uses the stdlib strlen. Using flex for small jobs is overkill but it's quick and convenient. I am a hobbyist programmer; I write so-called "trivial" programs.

That said, this exact function is used in some "non-trivial" software written by someone else and that person is IMHO a better C programmer than any HN commenter I have seen, most of whom do not let the public see the code they write anyway. Go figure.

NB. I am not the author; this is in the public domain. The author is djb.

stephc_int1316• 3 years ago

I think this naming style should be considered obsolete.

- This function will return the number of bytes, not of characters or codepoints. - str and len are both abbreviations, we should use full words when possible - We can also be more explicit about what the function does, it does not simply returns the string length, it counts characters (or bytes in this case)

Here is how I would name it:

u32 CountBytesInString(char* string); u32 CountCharactersInString(char* string);

And on the implementation side, this work can be done with SIMD instructions, and be really freaking fast, but still, it should be explicit for the user that the work is O(n) complexity, not exactly free.

kelnos3• 3 years ago

Out of curiosity, why do you use this? I expect the builtin strlen() to be even more optimized than this. There's a lot more you can do than a simple loop unrolling.

asveikau4• 3 years ago

Don't do this. libc string functions are usually done with hand tweaked assembly or as a builtin by the compiler. They will be faster than this.

This sort of practice is very outdated. The last time it made sense for performance reasons was probably the early 90s or earlier.

Additionally, strlen is not one of the C string functions you want to replace due to defects, the same way you would want to do with crusty old strcpy. If you're working with C strings there is nothing wrong with strlen. (Just don't call it redundantly in a loop body ...)

1vuio0pswjnm76• 3 years ago

Here is the musl stdlib strlen. I do not use glibc.

https://git.musl-libc.org/cgit/musl/plain/src/string/strlen....

   #include <string.h>
   #include <stdint.h>
   #include <limits.h>
   
   #define ALIGN (sizeof(size_t))
   #define ONES ((size_t)-1/UCHAR_MAX)
   #define HIGHS (ONES * (UCHAR_MAX/2+1))
   #define HASZERO(x) ((x)-ONES & ~(x) & HIGHS)
   
   size_t strlen(const char *s)
   {
   const char *a = s;
   #ifdef __GNUC__
   typedef size_t __attribute__((__may_alias__)) word;
   const word *w;
   for (; (uintptr_t)s % ALIGN; s++) if (!*s) return s-a;
   for (w = (const void *)s; !HASZERO(*w); w++);
   s = (const void *)w;
   #endif
   for (; *s; s++);
   return s-a;
   }

1vuio0pswjnm76• 3 years ago

There are HN readers besides myself, not to mention hundreds if not thousands of others, who use various programs that use this function, such as

https://cr.yp.to/ucspi-tcp/install.html

https://cr.yp.to/daemontools/install.html

https://cr.yp.to/djbdns/install.html

https://cr.yp.to/clockspeed.html

https://cr.yp.to/software/qmail-1.03.tar.gz

https://smarden.org/runit/install.html

1vuio0pswjnm76• 3 years ago

glibc strlen:

https://sourceware.org/git/?p=glibc.git;a=blob_plain;f=strin...

   #include <libc-pointer-arith.h>
   #include <string-fzb.h>
   #include <string-fzc.h>
   #include <string-fzi.h>
   #include <string-shift.h>
   #include <string.h>
   
   #ifdef STRLEN
   # define __strlen STRLEN
   #endif
   
   /* Return the length of the null-terminated string STR.  Scan for
      the null terminator quickly by testing four bytes at a time.  */
   size_t
   __strlen (const char *str)
   {
     /* Align pointer to sizeof op_t.  */
     const uintptr_t s_int = (uintptr_t) str;
     const op_t *word_ptr = (const op_t*) PTR_ALIGN_DOWN (str, sizeof (op_t));
   
     op_t word = *word_ptr;
     find_t mask = shift_find (find_zero_all (word), s_int);
     if (mask != 0)
       return index_first (mask);
   
     do
       word = *++word_ptr;
     while (! has_zero (word));
   
     return ((const char *) word_ptr) + index_first_zero (word) - str;
   }
   #ifndef STRLEN
   weak_alias (__strlen, strlen)
   libc_hidden_builtin_def (strlen)
   #endif

NetBSD common strlen:

https://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/common/...

   size_t
   strlen(const char *str)
   {
   const char *s;
   
   for (s = str; *s; ++s)
   continue;
   return(s - str);
   }

Apple strlen:

https://opensource.apple.com/source/Libc/Libc-1244.50.9/stri...

Apple strlen comes from FreeBSD. Until 2009, FreeBSD used an unoptimised strlen.

https://svnweb.FreeBSD.org/base/head/lib/libc/string/strlen....

   size_t
   strlen(str)
           const char *str;
   {
           const char *s;
   
           for (s = str; *s; ++s);
           return(s - str);
   }

FreeBSD eventually copied^1 NetBSD's x86_64 strlen.

1. "modeled after", "inspired by", etc.

https://svnweb.FreeBSD.org/base?view=revision&revision=18770...

   #include <sys/cdefs.h>
   __FBSDID("$FreeBSD$");
   
   #include <sys/limits.h>
   #include <sys/types.h>
   #include <string.h>
   
   /*
    * Portable strlen() for 32-bit and 64-bit systems.
    *
    * Rationale: it is generally much more efficient to do word length
    * operations and avoid branches on modern computer systems, as
    * compared to byte-length operations with a lot of branches.
    *
    * The expression:
    *
    *      ((x - 0x01....01) & ~x & 0x80....80)
    *
    * would evaluate to a non-zero value iff any of the bytes in the
    * original word is zero.  However, we can further reduce ~1/3 of
    * time if we consider that strlen() usually operate on 7-bit ASCII
    * by employing the following expression, which allows false positive
    * when high bit of 1 and use the tail case to catch these case:
    *
    *      ((x - 0x01....01) & 0x80....80)
    *
    * This is more than 5.2 times as compared to the raw implementation
    * on Intel T7300 under EM64T mode for strings longer than word length.
    */
   
   /* Magic numbers for the algorithm */
   #if LONG_BIT == 32
   static const unsigned long mask01 = 0x01010101;
   static const unsigned long mask80 = 0x80808080;
   #elif LONG_BIT == 64
   static const unsigned long mask01 = 0x0101010101010101;
   static const unsigned long mask80 = 0x8080808080808080;
   #else
   #error Unsupported word size
   #endif
   
   #define LONGPTR_MASK (sizeof(long) - 1)
   
   /*
    * Helper macro to return string length if we caught the zero
    * byte.
    */
   #define testbyte(x)                             \
           do {                                    \
                   if (p[x] == '\0')               \
                       return (p - str + x);       \
           } while (0)
   
   size_t
   strlen(const char *str)
   {
           const char *p;
           const unsigned long *lp;
   
           /* Skip the first few bytes until we have an aligned p */
           for (p = str; (uintptr_t)p & LONGPTR_MASK; p++)
               if (*p == '\0')
                   return (p - str);
   
           /* Scan the rest of the string using word sized operation */
           for (lp = (const unsigned long *)p; ; lp++)
               if ((*lp - mask01) & mask80) {
                   p = (const char *)(lp);
                   testbyte(0);
                   testbyte(1);
                   testbyte(2);
                   testbyte(3);
   #if (LONG_BIT >= 64)
                   testbyte(4);
                   testbyte(5);
                   testbyte(6);
                   testbyte(7);
   #endif
               }
   
           /* NOTREACHED */
           return 0;
   }

1vuio0pswjnm76• 3 years ago

Something worth considering: Sometimes people may use older versions of compilers, e.g., older versions of GCC, for compiling older programs for older hardware. These GCC versions are smaller in size and written to run on less powerful hardware. For example, the gzip'd source tarball for GCC 2.95 from 2001 is 12M while the one for GCC 12.2 from 2022 is 143M, an 11x size increase.

faxmeyourcode• 3 years ago

Do you have any recommendations for good open source C standard library replacements instead of rolling your own string manipulation functions?

gaws2• 3 years ago

use a good library

Such as?

nathell• 3 years ago

Our last function is strcmp. It looks at two strings and determines whether they are equal to each other or not. If they are it returns 0. If they aren’t it returns 1.

No it doesn’t.

    RETURN VALUES
         The strcmp() and strncmp() functions return an integer greater than, equal
         to, or less than 0, according as the string s1 is greater than, equal to,
         or less than the string s2.  The comparison is done using unsigned
         characters, so that ‘\200’ is greater than ‘\0’.

Decabytes2• 3 years ago

I've added a footnote to my incorrect explanation and credited you. I'm still a C noob so thank you for pointing this out!

andoma• 3 years ago

Reminds me of the incorrect cast of memcmp() return value that resulted in this bug: https://bugs.mysql.com/bug.php?id=64884

cassepipe• 3 years ago

I don't have MacOs to prove it but I believe `strcmp` on MacOs returns either 0, 1 or -1

skywal_l• 3 years ago

https://developer.apple.com/library/archive/documentation/Sy...

saagarjha14• 3 years ago

It currently returns a character difference. Don’t rely on it, though!

mahoho• 3 years ago

Just a pedantic comment, but 有り難う is arigatou or roughly "thanks", not "hello". Hello would usually be こんにちは or, more confusingly, 今日は

teddyh2• 3 years ago

もしもし

commandlinefan• 3 years ago

Sort of unfortunate, because there's really no good translation for "hello" into Japanese - you'd say こんにちわ in the morning, in the afternoon こんばんわ and もしもし when answering the phone...

scrame• 3 years ago

you'd say ohayogozaimasu in the morning, konnichiwa in the afternoon and konbanwa in the evening.

I don't know how to type japanese on my phone, but the first is literally "early" surrounded by honorifics. The characters for konnichiwa means "this/now", "day" and the "wa" at the end is an article making the previous phrase the subject of the sentence. Same with konbanwa, but for evening instead of day.

no idea on the etymology of moshimoshi for answering the phone, though.

gpderetta3• 3 years ago

The worst part of C strings is that they tend to show up in APIs (especially system calls). This make interoperability with other languages harder than it should m

ziml77• 3 years ago

This is why I hate them too. You can use a custom length + pointer type for representing strings in your own code, but interfacing with other libraries and the OS almost always requires having a null-terminated string. It forces you to make copies just to tack on the null terminator.

benmmurphy• 3 years ago

`strlcpy` is the function you probably want. but again it is not standard. https://lwn.net/Articles/507319/

I think the reason people don't want to standardise this kind of function is it often gives wrong behaviour. for example if you are trying to copy a string into a fixed buffer and its too long then often it is an error or potentially even a security bug to truncate it. so these functions generally do the 'wrong' thing even though they are 'safer'. if you are dealing with static buffers then I think you should be explicitly checking the source fits in the target and then handling the error case. you could even have a function like `strlcpy` that does `strlen` then checks if it fits, then does the copy or return an error code. alternatively, if the string should always fit and you don't want to handle the error case then the safe thing to do is check at runtime that it fits then abort the program if it doesn't fit.

Night_Thastus4• 3 years ago

strlcpy is not needed, strcpy_s (not strncpy_s) is safe and is part of the C11 standard.|

In fact, strlcpy is worse:

* strlcpy truncates the source string to fit in the destination (which is a security risk)

* strlcpy does not perform all the runtime checks that strcpy_s does

* strlcpy does not make failures obvious by setting the destination to a null string or calling a handler if the call fails.

zabzonk5• 3 years ago

strcpy_s is part of the C11 standard

an optional part, which makes it pretty worthless, if it were not so already.

kelnos3• 3 years ago

On systems that aren't memory constrained, we just shouldn't be using static buffers at all. Just always use something like asprintf() and free() the result when you're done. No, it's not in the C or POSIX standards, and that's a shame, but it's at least available on Linux and the BSDs.

I end up working on a lot of code that uses Glib, so I tend to use g_strdup_printf() a lot, which works the same as asprintf().

Ultimately the cost of allocations is usually not a big deal, and you gain a lot of safety. Sure, you then have to remember to free(), but I'll take a memory leak over a segfault (and its possible security consequences) any day.

And if allocation cost is a problem, you can always go back and optimize with static buffers later. That shouldn't be the default that people reach for, though.

saagarjha14• 3 years ago

No, it’s not. The return value it provides is generally unwanted.

photochemsyn• 3 years ago

For initial string input, i.e. from a network/file/terminal stream, using fgetc and/or fgets plus code to verify and sanitize makes the most sense IMO.

This does mean you have to write a lot of C code for what would be simple tasks in other languages, e.g. a correct file open, read-to-dynamically-allocated-memory, and file close with good error checking is a full page (at least) of dense code in C and just two lines in Python.

If you've done a good job sanitizing and verifying all the input to your program, only then does it becomes relatively safe to use the standard string functions, with caveats for multithreading.

Asking ChatGPT to compare and contrast fgetc and fgets is a good place to start, and then ask how to use fgets to handle errors during stream I/O, and what can go wrong with multithreading etc. Then take a look at the sqlite source code for in-house C-string handling, here's the take-away comment:

"Because there is no consistency, we will define our own."

https://github.com/sqlite/sqlite/blob/master/src/util.c

Night_Thastus4• 3 years ago

It's important to mention that strncpy (and also strncpy_s) are really not a strcpy replacement, it's not intended for the same usages. The name is a total misnomer. Do not use strncpy that way!

In any case, strcpy_s (which is a good replacement for strcpy) is part of the C11 standard. I'm confused how that isn't considered portable.

zabzonk5• 3 years ago

it's an _optional_ part of the standard, and so can't be relied on. also, the idea behind it is pretty poor.

Night_Thastus4• 3 years ago

Don't GCC, Clang and MSVC provide it? It may be optional in practice but if the major compilers support it, it's not really an issue.

The idea may not be perfect, but for C which is intended to be low overhead, strcpy_s is about as good as it gets. If you want something more user friendly, that is what C++ is for with std::string, or library implementations like Boost or QT string.

cellularmitosis• 3 years ago

Yeah, I was a little confused by the strncpy "gotcha": "The answer is that the destination gets filled with all the characters of the source string with no room left for the null terminator."

Well, I mean, the docs specifically call that out: "If src is less than len characters long, the remainder of dst is filled with ‘\0’ characters. Otherwise, dst is not terminated."

So if you want null termination in all cases, you need to pass len-1, not len.

js2• 3 years ago

Programs which have to deal with C strings beyond the bare minimum that libc provides will generally have a set of routines for making it more ergonomic. e.g.:

https://github.com/git/git/blob/master/strbuf.h

flohofwoe6• 3 years ago

This is from a C fan: If you are going to do any string heavy work, please use anything else than C (Python is pretty nice for this sort of stuff for instance).

And if you need to use C anyway, then please use anything else than the string functions from the standard library. The C stdlib is (mostly) a leftover from the K&R era when opinions about what makes a good API were very different from today, and C was a much 'harsher' language.

C is pretty nice for a lot of things, but working with strings definitely isn't one of them.

cozzyd• 3 years ago

I would say, unless there's a performance reason not to, always use asprintf for every string operation.

axilmar• 3 years ago

For string heavy workload, C is ideal, provided that you don't use C string functions.

You can always allocate a very large buffer and do your string operations there, using memncpy and the assorted functions which can be inlined in many architectures and be really fast.

Then you can dispose of the buffer really quickly with one call or reuse it for later operations by simply setting a few pointers to initial status...

flohofwoe6• 3 years ago

If you have large data sets and need maximum performance I agree, but a lot of day to day string processing works on small data sets and isn't very performance-sensitive.

BananaaRepublik• 3 years ago

As a newcomer to C, why is it that the C standard library doesn't get updated? Newer languages seem to place a lot of emphasis on getting their standard libraries as useful as possible. It's odd to be told not to use the standard library functions but to write my own instead. I'm really doubting I can just sit down and hammer out string functions superior to string.h.

flohofwoe6• 3 years ago

Just my guess: C doesn't depend as much on its standard library as other languages (which has its good and bad sides), and this also means that fixing the standard library isn't very high on the priority list of the C committee, because everybody knows that for any serious work the stdlib isn't suitable anyway.

The C stdlib is basically the lowest common denominator which enables writing very simple UNIX-style mostly-cross-platform command line tools, but not much else. For anything serious you either call OS API functions directly, or resort to specialized third-party libs.

Attempting to 'fix' the stdlib would first mean agreement on what such a stdlib should actually contain and look like, and this has a real risk of ending up in C++ Commitee style busywork (e.g. lots of activity with little to show for).

pixelbeat__• 3 years ago

String handling in C has many gotchas indeed. Here are some of my notes on the subtleties:

https://www.pixelbeat.org/programming/gcc/string_buffers.htm...

teddyh2• 3 years ago

strcmp takes two strings and returns 0 when they are true.

ITYM “equal”, not “true”.

Tarragon• 3 years ago

So how can we handle this case safely? There are a few ways I can think of.

strdup:

The strdup() function returns a pointer to a new string which is a duplicate of the string s. Memory for the new string is obtained with malloc(3), and can be freed with free(3).

kevin_thibedeau• 3 years ago

wchar_t is a massive landmine that should never be used since its size varies by platform. The locale of the compiler has to match the end user for L prefixed strings to work correctly. Likewise char16_t and char32_t are just swimming against the easy path at this point. You're much better off sticking to UTF-8 and using the C11 u8 prefix on literals so you can use the regular string API and never have to worry about locale settings.

Decabytes2• 3 years ago

This is great advice! I wasn't aware of this and I will keep that in mind. When I first came across Unicode literals I was unsure when exactly you would use them over wchar_t

zh3• 3 years ago

I once got called in to fix an SS7 stack suffering from poor performance. Pretty well written, and not obvious at first sight why it was going slow. Most of it was low-level bit fiddling, and some small strncpy's() - generally about 8 chars or so.

Didn't take that long to profile (well, printf's as no profiling available) and figure out it was the strncpy's causing the problem, but why? Well, there was a handy 8 megabyte buffer used for working memory that the strings were being copied into that for modification.

From the strncpy() man page:-

If the length of src is less than n, strncpy() pads the remainder of dest with null bytes.

Ah. So every little strncpy was essentially copying the string then zeroing out 7,999,992 bytes. And there were lots of little strncpy's...

PaulDavisThe1st5• 3 years ago

The appearance of strncpy() in any source code is an immediate panic attack for me. It should never be used, and if it is used, it should be removed.

Similar rule for sprintf(), all instances of which should be replaced by snprintf().

bluGill• 3 years ago

Unfortunately there often isn't a better replacement in your standard library (embedded systems are weird). I ended up using strncpy followed by automatically setting the last byte of the string to null.

saagarjha14• 3 years ago

snprintf has very similar performance pitfalls.

russellbeattie• 3 years ago

Literally 25 years ago I was a beginner programmer and tried writing a .dll for Microsoft's Internet Information Server, which was relatively new at the time. (I hadn't so much as seen a Unix-based OS at the time, let alone understood CGI). C strings were mind boggling and frustrated me so much I simply gave up. Happily around the same time, MS introduced Active Server Pages and I was able to use that and never messed with C again. It's amazing the same issues still exist decades later.

chihuahua• 3 years ago

That is the most mind-boggling part of this saga to me. People have been using C since the 1970s. It's now 2023, and there still isn't an obvious solution to this other than suggestions that every team should write their own string library from scratch.

And apparently it all started with some genius deciding that using a single 0-byte at the end is so deliciously efficient and therefore obviously the way to go. We can't waste 4 bytes for the string length, that's out of the question. I think only the Pascal solution of having a single byte for the string length is worse.

kloch• 3 years ago

At first this looks great, but there is a problem. What happens when the source string minus the null terminator is as long as the size of the destination string? The answer is that the destination gets filled with all the characters of the source string with no room left for the null terminator.

The 'n' in strncpy is mainly there to help you avoid overrunning the destination, it does not guarantee whatever makes in there is null-terminated.

This is why you should always explicitly set the last byte to zero after using strncpy (and never ever use strcpy).

  char dest[16];

  strncpy(dest, src, 15);
  dest[15]=0;

djha-skin• 3 years ago

Related and good read about strcpy in the kernel: https://lwn.net/Articles/905777

Dwedit• 3 years ago

"But for real if anyone knows how to get this to work on Windows 10 let me know!"

Since the May 2019 update, Windows 10 has supported declaring the code page in a manifest file.

In Visual Studio, you must add "/utf-8" to the compiler command line, this makes it parse the source code as a UTF-8 file, and makes it output UTF-8 string literals.

To make console output work, call the Win32 function "SetConsoleOutputCP(65001);"

To get support for opening files with names that aren't in your system codepage:

* Create a manifest file as shown in https://learn.microsoft.com/en-us/windows/apps/design /globalizing/use-utf8-code-page

* Add this as an "Additional Manifest File" in Visual Studio project settings for the manifest tool

Additionally, there is an undocumented NTDLL function "RtlInitNlsTables" that sets the code page for the process. It is difficult to use without a lot of example code, but some app locale type tools (used to change locale for a process) make use of this function.

jeffrallen• 3 years ago

Programming in Go made me a better C programmer, because now I no longer use C strings, only a buffer/length/capacity struct.

bruce343434• 3 years ago

Meh. The w_char stuff is barely C's fault. You use wide (constant width) characters then set the terminal encoding to utf8 (variable length encoding). What did you expect? It's a windows issue. I can copy paste all sorts of utf8 in "normal" string literals, printf and puts them, and it just works in my terminal.

RE counting characters: this is a whole can of worms. Do you want to count grapheme clusters? Code points? Anything other than just the amount of bytes? Use a unicode library.

The latter part of this article is a bit like those articles that make fun of javascript for having floating point numbers behave like, gasp, floating point numbers.

analog31• 3 years ago

As a cellist, I was about to sympathize when I read the title.

simonblack• 3 years ago

"We're not in Kansas any more, Toto"

Or to paraphrase that "We're not in Python any more, and C is not Python".

You know what sends me insane? Indentation and lack of fixed types in Python. But I don't have problems with C strings. Because I have grown to love and know C's string foibles just like the author will certainly not be driven insane by 'Python's shortcomings according to me'.

The world is full of people who complain that something or other is different from what they know, so that 'other' is wrong. That's just being isolationist. Everything has its own advantages, its own disadvantages. Let's accept that and move on, instead of making mountains out of mole-hills.

Sohcahtoa82• 3 years ago

Indentation and lack of fixed types in Python.

Whenever I see someone complain about Python's indentation, my brain internally translates it to "I poorly format my code."

If you code is properly formatted, then Python's indentation is never a problem. I praise Python's indentation-as-syntax because it prevents issues like a dangling else or a forgotten brace while also making proper formatting a requirement for your program to run.

smackeyacky• 3 years ago

Let's be clear about this: python indentation-as-syntax makes one particular style of formatting a requirement. Not everybody likes it. You might say I poorly format my code in other languages, but most of the time for readability you are somewhat stuck with the formatting of that code base which turns out to be a non-problem thanks to modern editors (sarcasm) like vi that can do brace matching and are older than python.

Python's rigidity on formatting should solve that problem, but it really doesn't and over time it relaxed the rules somewhat which made it better.

_a_a_a_• 3 years ago

then Python's indentation is never a problem

I'm sure you have some solid 3rd party research to cite to back up that absolutist claim. Or have some explanation that a it with my workmate's fault and not a git merge (which mixed up indenting between the two files) that introduced a bug that took hours to track down. Or why people like me loved the idea of meaningful indentation in python, then grew to not love it any more after experience.

Or the bitch that it can be when you have to generate code and have to do more than just slap braces around it to delimit blocks.

All in all, another one of those pure-opinion HN posts that are becoming too common.

klibertp• 3 years ago

You've missed the GP point by so wide a margin you could have as well shot in the opposite direction. You're basically doing exactly what they asked you not to.

People are different, work in different ways, are productive with different techniques, have their own habits formed over decades that they don't feel the need to change. None of that means they're worse than you, who, of course, always properly formats your code.

It's perfectly possible, and not that rare, to be careful with indentation and not liking significant whitespace (esp. if the rules are not overly consistent, like in Python). It's also possible to feel like the need to manually adjust indents after moving the code around is a distracting chore that braces largely eliminate. It's possible to work exclusively within an IDE that will make "dangling else" problem impossible to happen, therefore it's possible not to see that as a problem. And so on.

Keep an open mind. Try to get accustomed to various style of working with code. Stop your brain from being discriminatory, and don't assume too much about what other people consider "good" or "poor". If in doubt, ask politely.

BTW: I'm using Python since 2.5.2 professionally. Just a quick disclaimer.

cma• 3 years ago

The biggest issue is python's indentation stuff doesn't work in a function parameter list, so you get stuck with single line lambdas only or defining a temp named function before using then in an unnatural way.

eloff• 3 years ago

That’s a fair point, but of the over twenty programming languages I’ve used in my career, only C uses null terminated strings. All the others store the length. There’s good reasons for that. I think C strings are objectively bad and error prone.

lmm• 3 years ago

Indentation and lack of fixed types aren't responsible for over 50% of known security issues in software.

C's issue are not just harmless foibles. They cause real harm to the poor people actually using the software.

heywhatupboys2• 3 years ago

Indentation and lack of fixed types aren't responsible for over 50% of known security issues in software.

there is no way of fixing this with a "string type". The errors come from IPC/Internet, and there will always be just a sequence of bytes, and some length, maybe, given by the user. Somewhere some code will have to trust this length, or compute a length.

chlorion• 3 years ago

There is no practical advantage to null terminated strings though.

It's not that they are "different", it's that they are extremely error prone and have poor performance for certain operations such as getting the length.

But I don't have problems with C strings

Everyone thinks they are clever enough to use them and other parts of C without problems, and those people are the most dangerous.

vkou2• 3 years ago

You've grown to love the footguns and hundreds of thousands of security holes that null-terminated strings have introduced over the decades?

It's not so much a question of different is bad, it's that having one of the six positions for your car's stick shift be marked 'Self-destruct' is... Sub-optimal. I'm sure you're smart enough to operate that car safely, but the ditches seem to be filled with burnt-out husks.

Tab-based, versus curly-brace indentation, on the other hand, is a question of how you want the car painted. Purely personal taste.

unscaled• 3 years ago

It's not so much a question of different is bad, it's that having one of the six positions for your car's stick shift be marked 'Self-destruct' is... Sub-optimal. I'm sure you're smart enough to operate that car safely, but the ditches seem to be filled with burnt-out husks.

Good analogy, but I think you're being to forgiving to C. It's more like having all 6 out of the 6 positions of your car's stick shift marked as self-destruct. If you don't want the car to self-destruct while changing gears, you need to tune your FM radio to 99.0 Mhz and quickly set your turn signals to left, right and left again before shifting. And that only works safely when shifting into the 1st, 2nd and 4th gears, unless you modify your car engine sot it can only drive on Microsoft roads.

scoutt• 3 years ago

On a Python car, the car won't even start if you don't have your cosmetics right, and they have to be of a specific color.

tragomaskhalos2• 3 years ago

K&R contains this beautiful koan-like string copy code:

    while (*t++ = *s++)
        ;

Honestly the elegance of this thing was one of the hooks that made me fall in love with C. But this was from a now-forgotten age of innocence, as there are so many "nopes" around this line-and-a-half that one would, rightly, be tarred and feathered for ever putting it in a program today.

kovac2• 3 years ago

Could you explain why this line should be discouraged? I'm a beginner in C, so I really don't know. That's why I'm asking.

fatcow• 3 years ago

while (t++ = s++) ;

You're assigning a char to another, relying on the return value being 0 to detect end of string.

You're performing the copy while also increasing the pointers with ++ in the same expression.

You're using the cryptic ; empty statement to signify nop, thereby confusing newbies.

Etc

commandersaki• 3 years ago

Equivalent of a strcpy() there's no bounds checking.

tragomaskhalos2• 3 years ago

Another reason, in addition to other replies and separate from safety concerns, is that strcpy, memcpy etc are nowadays usually implemented via more efficient compiler intrinsics rather than an explicit loop.

ranger_danger2• 3 years ago

By default, Windows PowerShell .lnk shortcut is hardcoded to use the "Consolas" font

Surely this is not the case for Japanese versions of Windows (or users with Japanese set as their display language?)

lifthrasiir2• 3 years ago

Yes, you can actual look at the full list from `HLKM\Software\Microsoft\Windows NT\CurrentVersion\Console\TrueTypeFont` (taken from my copy of Windows 10, bracketed comments mine):

    0       Lucida Console
    00      Consolas
    932     *ＭＳ ゴシック [MS Gothic for Japanese]
    936     *新宋体 [Simsun for simplified Chinese]
    949     *굴림체 [Gulimche for Korean]
    950     *細明體 [Windows MingLiU for traditional Chinese]

Note that there are actually two global defaults which only are differentiated by leading zeros. This is intentional and can be used to enable additional fonts; it is a common tweak for Korean (and probably other CJK) users to add a preferred font with a name 0949 or 00949 etc.

ranger_danger2• 3 years ago

Japanese versions use CP932 though, which according to your list would use a font that's not Consolas (MS Gothic in this case).

TheRealPomax• 3 years ago

How to make C safe: by putting it back in the box and putting the box back on the shelf and then closing the door to the garage. Then using a safe-by-design language.

kens• 3 years ago

It's strange that computer programmers think of themselves as being on the cutting edge of technology, but then we use a language that is over 50 years old. Of course there are going to be lots of problems with C strings since they were designed for a totally different world (no Unicode, no security issues, memory was precious, etc). The hardware is a million times as powerful but the software environment improves at a glacial pace.

nneonneo4• 3 years ago

Pop quiz, which of these is safe, given "char buf[80]" and arbitrary user input in argv[1]?

    gets(buf);
    scanf("%s", buf);
    strcpy(buf, argv[1]);
    scanf("%80s", buf);
    strncpy(buf, argv[1], 80);
    snprintf(buf, 80, argv[1]);

----

The delightful answer is none of them. The first three have no bounds checking at all, meaning that they will happily overflow the buffer to an arbitrary extent (gets, at least, will usually trigger a warning on modern compilers). The next two have off-by-one errors: scanf will write a NUL byte out of bounds (and that's exploitable! https://googleprojectzero.blogspot.com/2014/08/the-poisoned-...) while strncpy will fail to NUL-terminate the string. The last one uses the right buffer length, but treats user input as a format string and can leak memory contents or produce arbitrary memory corruption with the %n format specifier.

C string handling practically invites off-by-one errors and horrible security practices out-of-the-box.

myrmidon2• 3 years ago

Are you sure that strncpy does an out-of-bound write here? I believe it doesnt, but would give you an unterminated string in buf which is also... less than ideal (if the input is 80 non-null characters or longer).

nneonneo4• 3 years ago

Ugh, I had that in my comment before I "refactored" it. Fixed now, thanks for pointing it out.

Lt_Riza_Hawkeye• 3 years ago

Yeah, and I wouldn't say it's definitely unsafe. You can memchr a '\0' out of it (or not) to determine if a null terminator got in there or not.

throwaway2037• 3 years ago

This is a very thoughtful post. Real question: Is the solution to avoid C strings entirely? Use something like a Pascal string that includes the length?

jiggawatts4• 3 years ago

Yes. For the love of god, a thousand times yes. Every other language, even some of the "C-compatible" ones uses explicit-length strings.

fractalb• 3 years ago

    snprintf(buf, 80, “%s”, argv[1]);

Should work.

LoganDark• 3 years ago

As long as you replace the 'smart'-quotes with actual quotes.

-Emily

alxmdev• 3 years ago

Bingo, you should never pass arbitrary strings where they could be used as format specifiers, it's like running arbitrary code. Some compilers even issue warnings when you pass non-literal format strings to the printf family.

robomartin10• 3 years ago

The delightful answer is none of them.

No. Sorry. This is bad programming. C'mon.

I started programming back in the 8080, 8085, 6502, etc. days. I had to program some prototype computers using a hex keypad while entering raw machine code (not even assembler). I still own a couple of these:

https://i.imgur.com/ZsIJj1p.png

In a couple of cases I had to take this approach to bootstrap Forth on a 6502, then write a full Forth code editor and finally write the robotics application from there.

Do not confuse bad programming or lack of knowledge with something attributable to a language, any language. A knowledgeable software developer, among other things, stays clear of these issues. This is also the value of experience and exposure to a wide range of technologies.

It's like blaming MicroPython for a machine getting destroyed because garbage collection interrupted a critical real time process. There's nothing wrong with MicroPython in that regard, the programmer/designer of the embedded system either lacked knowledge and understanding.

Part of the problem, as I see it, is that a good deal of modern university CS degrees don't even touch low level stuff. They start students on languages like Javascript and Python. These are fantastic, however, someone with deep-rooted experience in these languages who jumps into C is very likely to do some truly horrific things. The language isn't the problem, at all.

I mean, not to go too far, the Linux kernel is written in C. Right? It's about the person, not the language.

skribanto• 3 years ago

One could take a glance at these and easily believe that they do the right thing. I don’t think that no one would accidentally miss such a small error from time to time.

imtringued• 3 years ago

Do not confuse bad programming or lack of knowledge with something attributable to a language, any language.

In C everyone starts out as a "bad programmer". In other languages people are merely inexperienced.

however, someone with deep-rooted experience in these languages who jumps into C is very likely to do some truly horrific things. The language isn't the problem, at all.

You are contradicting yourself.

jiggawatts4• 3 years ago

A knowledgeable software developer, among other things, stays clear of these issues.

This is the "no true scotsman" fallacy.

Languages can be designed so that less than perfectly knowledgeable programmers fall into the pit of success, or they can be designed so that they fall into the pit of failure.

For people making your argument, I like to provide this challenge: Go take a flight on a 737 MAX that hasn't had its MCAS fixed/disabled. That should be fine, right? After all, no "true" pilot ought to disregard one sentence on page 437 of the flight manual that they weren't even given during a 1 hour training video. A true professional pilot memorises the engineering blueprints, the source code of the avionics, and the wiring schematics, surely. So you have nothing to fear! The plane is "safe", and pilots can be trusted to be knowledgeable.

Go buy that ticket.

KyeRussell2• 3 years ago

God, this attitude reeeallllyy grinds my gears.

This is precisely why C has outstayed its welcome in so many areas of software development.

Every time some kid looking for a self-confidence boost buys into the idea that using a language with a minefield of archaically-named string manipulation functions somehow makes them a ‘real’, ‘smart’, developer, we are all left a little worse off.

No, it’s not the fault of the language’s design. It’s not even the fault of history - the fact that C was conceived at a time when security wasn’t what it is. It’s these damn kids that only know Python and JavaScript! Why can’t they be as smart as us C developers!

This is all completely ignoring the fact that in 2023 we have no shortage of string manipulation-related vulnerabilities in widely popular and supposedly battle-tested C code. All some version of the typical list completely justifiable human errors that anyone is bound to make writing C.

A language that is so popular but that so few people seem to be able to write secure code with, is not a very good language.

I’m immediately skeptical of anyone that’s not of the view that the single best thing we as an industry can do for security is to drastically reduce the amount of C code in circulation. It always comes down to “I’m set in my ways and I think I’m superhuman”.

My hope is that these modern, sensible systems programming languages successfully eat the world faster than the pool of C developers thins out, as people slowly retire, and more greenhorns clue into the fact that C is being used in more places than it ought to be.

Signed, someone that did learn C in school, and has written it professionally.

Zababa2• 3 years ago

I did start with C at university, C and Scheme. We didn't go very in depth in the "defensive programming" part, especially with C. We talked a bit about safety when we were doing web and database stuff, but I think that's it. I am pretty lucky, one of my friend is in cybersecurity, another is very good with C, and there are lots of people online freely sharing their knowledge, so at least I kind of know what I don't know.

On the other hand, not everyone had that luck. I've seen a good number of people that are very good at what they do but lack more general culture. But it's hard to keep up with everything. Software is a huge world, I think already way too big for everyone to know everything. And it's not just software too, it's important to learn about business too, and maybe a bit of maths here and there isn't a bad idea, and there's also the hardware part, networking, and every day there is more and more and more.

What I mean by this is that I don't know how things were before, but today, for a lot of people that write code, it's not possible to know everything, have everything fit inside your head. In those cases, people usually start asking for more guardrails in their tools, because they're no longer manipulated only by experts. And sometimes the experts themselves ask for guardrails too. So some want tools to change, others don't want them to change, and both have a point.

On one hand, I understand that blaming the tool isn't a good attitude to have. On the other hand, my job consists in building tools for other professionals, and I feel like I have way higher standards for the tools that I produce compared to the tools that I use.

up2isomorphism2• 3 years ago

Dude, your last one is not even the correct way of calling snprintf, what the hell are you talking about?

nneonneo4• 3 years ago

I mean, none of these are correct. But they will all compile, often with no warnings at all. The format string doesn’t need to be a literal string: this is useful for situations like localization.

KyeRussell2• 3 years ago

No surprise that this completely unjustified vitriolic reaction comes from someone arguing about C.

heywhatupboys2• 3 years ago

snprintf(buf, 80, argv[1]);

yeaaa, this has much bigger problems than a null write...

cdelsolar• 3 years ago

So then what do you do?

Woodi• 3 years ago

C string handling practically invites off-by-one errors and horrible security practices out-of-the-box.

At that point it is beating dead horse - it is such well known fact how C strings works. And it's insanity that noone proposed new standard library with better implementation for strings domain. Boo hoo "old programs" bla bla...

And it's total insanity to blame powerfull language for allowing you to do almost anything in it. You don't even need to ask commitee for permission to roll your own _low level constructs_ - how insane is that ? ;)

But keep spitting on what gives you freedom...

dsvr• 3 years ago

There's a framework for C now at https://vely.dev which may help with C strings safety and memory management, among other things.

mantratome• 3 years ago

I found Vely on dev.to and did a test project with it. Pretty neat and solid.

pipeline_peak• 3 years ago

Handling strings in C was enough for me to choose C++…

hollowturtle• 3 years ago

Any good resource around with all the common C pitfalls and relative solutions?

tmsln• 3 years ago

Beej's guide to C programming is very helpful:

https://beej.us/guide/bgc/html/split/unicode-wide-characters...

C Strings and my slow descent to madness

Related Stories