I wrote one of those once! It was a nightmare morass of one-pixel errors. Maybe there was some clever simplification I missed that would have made it all easy, but these days I use the libraries. :P
I know that's how stb_truetype's renderer works, incidentally. Don't know if Freetype is the same, but I'd be mildly surprised to find it wasn't.