RTFC is the New RTFM

http://jakeradakovich.wordpress.com/2013/04/24/rtfc-is-the-new-rtfm/

by jradakov • 13 years ago

33 47 13 years ago

jakeradakovich.wordpress.com

  I nonchalantly told him to look at the source code of the library
  we were using. He looked at me like I had asked him to help me dig up a grave.

I can't imagine not examining the source code of the various frameworks I use on a daily basis.

I think the next time I interview a candidate I will ask them what major open-source libraries they use. Then I will ask them what they think of the quality of the source code, how it's implemented, what they find difficult to reason about, etc. Getting a blank stare at this point will be a kind of FizzBuzz moment.

Edit: I'm not suggesting that candidates should be an expert on the internals of the libraries they use (I'm certainly not). What I'm looking for is the ability to crack them open when you hit a wall.

tych0• 13 years ago

I think it also depends on the language too. In languages with more expressive type systems, you can usually make an educated guess about something based on the type signatures it has. For the dynamic languages, though, reading the code is an absolute must.

tterrace2• 13 years ago

It definitely depends on the language. The author mentions spring security so I'm assuming he's using java, where it's a massive PITA to manually hunt down and attach the sources to the 50 jars you have in lib/. Maven probably has something that will automatically do this but if your legacy project is just using ant then you're pretty much stuck.

kaffeinecoma5• 13 years ago

I agree, and sadly it's way harder to locate relevant bits of code in dynamic languages. (Hey Steve Yegge- can we have Grok now, please?) But even in static languages it's a must for me. I find that Javadoc becomes largely irrelevant if I can just click through to the actual source.

camus• 13 years ago

strictly typed languages like java are harder to write , but easier to work with (and document ) since one can know what is what , and what a method expects. It's easy to write a hash in JS but harder to understand what parameters are expected when you are using a library that passes hashes as parameters.

That's why i prefer languages with type inference rather than dynamic typing. You can be explicit when you need to and lazy when you want to.

(Excuse my french)

gyepi2• 13 years ago

>I think the next time I interview a candidate I will ask them what major open-source libraries they use. Then I will ask them what they think of the quality of the source code

Sometimes, it is eye opening to read the source code to some popular libraries/frameworks and discover how badly the code is structured. Some definitely belie the notion that code is written for humans.

jimmaswell2• 13 years ago

I don't look at my libraries' source unless I have a problem the documentation or google doesn't answer. that's really going to be your fizzbuzz?

A lot of the time I don't need the docs anyway, I just find what I'm looking for in the iltellisense/other suggestions because the OO model makes this work great.

kaffeinecoma5• 13 years ago

  unless I have a problem the documentation or google doesn't answer

Documentation/googling is definitely the right thing to do off the bat. But if they've never had a moment where they had to peek under the hood (or never thought to try, or didn't know how to) then yes, that's my FizzBuzz. I would perhaps not apply this to someone fresh out of school, but I would expect an experienced candidate to know how to "use the source".

lacksconfidence• 13 years ago

I feel the same. I don't know where i would be as a programmer if I didn't open up the code to libraries and look around. Reading through this thread I seem to be the odd one out, but generally after about a week of working with a library i skip the documentation and go straight to the source. Even more so on libraries that fake their documentation by auto-generating it from doc comments. Why would i want to read documentation generated from the source when I could just read the source and comments, having twice as much info? Additionally many projects documentation only gets to the level of 'why do i want to use this' and not the nitty gritty details. Those details are in the code, and thats where I tend to look first on any project I'm familiar with.

I should add that i work with interpreted languages, so going from my code to library code is literally just a key combo that looks at the token under the cursor, finds it in ctags, and goes there. It takes a fraction of a second to be looking at the code.

eterm• 13 years ago

Wow, I hope you never interview me.

FizzBuzz was a filter to filter out people who cannot code.

At all.

It was never designed as a competency measure, it was designed as a filter, to remove people who are mistaken into believing they can do something they cannot do at all.

What you have is a very precise measure, which would not even filter out the BSers who could probably happily talk for five minutes on the merits of a codebase they've never looked at (or perhaps even they have) but cannot despite this write a loop counting from 1 to 10.

Any blank stare after this question would not be me not understanding that libraries have code bases but would be me losing track wondering where my next interview would be.

kaffeinecoma5• 13 years ago

So you'd be unwilling to discuss a major library that you've used in your career (say Spring, or Rails) with a potential employer?

The "filter" I'm looking for is to weed out people who cannot crack open the source to something that is standing between them solving a problem. I'm not asking them to be an expert on the internals of everything they use. Apologies if my comment was unclear.

sophacles2• 13 years ago

So what would you do if you were unclear on how to do something based on the docs and googling? Have you genuinely never done anything even slightly out of spec with library author's expectations? Do you never need just part of the functionality of a library and wonder what the right way to get it is? Do you always just decide "it's not documented in a way I understand it, time to roll my own"?

mtrimpe• 13 years ago

My most valuable two questions have always been "What part of the stack you're working with do you understand best? Can you explain to me how it works?" and "What is the most complex modification/extension of the stack you've ever done? Can you explain to me how you did it?"

That will show you exactly how well they know their tools, how and how far they bent them to their will, how passionate they are about them and how well they can communicate on difficult technical topics.

I'd be hard-pressed to find other questions that consistently give me that much information about a candidate.

drivingmenuts• 13 years ago

Works or doesn't work, does what it says in the docs or doesn't. Those are the only true measures.

Everything else is just opinion.

ghshephard• 13 years ago

One organization that I worked at had a firmware team that moved quickly, and wrote the interface layer to the PHY/MAC/Network devices. After they defined the interface, later on enterprise software teams, and utility software teams, would write the "User Friendly" tools that would be used to configure the networking devices, but the firmware team did write a tiny "shim" utility that they released with firmware, that could be used to control the firmware at a very raw/native level.

Whenever you were wondering what parameters to pass to this low level (very useful) utility, the answer was almost always, "Just check out the Code" - and answer that was actually surprising straightforward once you got used to it.

nkuttler• 13 years ago

http://catb.org/jargon/html/R/RTFS.html

jradakov3• 13 years ago

I didn't realize an abbreviation already existed (RTFS) when I wrote the post. RTFS is probably better than RTFC.

wereHamster• 13 years ago

How far we've gone. First RTFB, then RTFM, now RTFS/C. What's next? RTF{Minds Of The People Who Wrote The Code}?

jradakov3• 13 years ago

I agree (hence the post), but at least with open source we have access to the source code. I realize this doesn't replace documentation, but could you imagine using a closed source project with sh*y documentation?

telent• 13 years ago

Compare http://catb.org/jargon/html/U/UTSL.html

dllthomas3• 13 years ago

Came to say this. Thanks for gettin' that for me.

gyepi2• 13 years ago

I have noticed the increasing paucity of documentation in many open source projects. In the past, this would annoy me when I had to context switch from a perfectly good flow state to a documentation reading state. However, I realized that I was doing it wrong; it meant that I was not spending enough time analyzing the code I was about to "adopt" and depend on for my project. Now, I read the documentation and source code for every open source program before I choose to use it. I also find correlations between the quality of the code and the documentation; good quality code sometimes lacks documentation but poor quality code rarely has good documentation.

Reading the code also provides the opportunity for learning beyond the documentation; there's a wealth of really good code out there just waiting to be read.

bttf• 13 years ago

I can agree with this. There have been times when I thought I understood the concept of something after reading the documentation, only to find myself gaining a deeper understanding when reading the code.

jerrya• 13 years ago

Yes, projects that claim to be infrastructure and come with no documentation, and presumably few comments, and who knows what unit tests.

It would be an exaggeration to refer to that as minimally viable code (mvc). It is really only a statement of the lack of engineering actually being practiced, and makes me wonder about the lack of robustness, the lack of security, the numbers of bugs in the systems produced.

I am not certain this is a winning strategy for open source.

astrodust• 13 years ago

Something needs to be done to address this enormous problem.

One of the worst examples I've come across lately is Zookeeper (http://zookeeper.apache.org/), which should win an award for how confusingly useless the documentation is. The Java source is probably refreshingly concise.

sixothree• 13 years ago

My number one annoyance about many open source projects is that they often don't compile out of the box. I can't even count the number of times I've downloaded a library only to have it fail miserably on compile because of some strange state the author left the source code in, or a library he wrote and failed to include. Etc etc.

spartango• 13 years ago

This worries me a little, because it cuts against the ideas of abstraction that are central to most software engineering.

Don't get me wrong, I'm not against reading the code if and when it is available.

I just worry that reliance on this MO goes against the idea that you can specify a programming interface for your system and people can develop against that without knowing the implementation. This is rather powerful in working with other people, complex systems, and systems that are changing, where reading the code may become inefficient.

I'd push back on the mindset of RTFC by saying that as a developers we should try not to link ourselves to infrastructure for which RTFM fails. I know it biases my search for libraries.

> I realize complaining about the lack of documentation in some open source projects is like complaining that there is no foie gras at a friend’s free dinner party

I'd argue that docs aren't foie gras, and not having them is like having a compile error/obvious segfault in your code. Just like stability, docs matter.

dllthomas3• 13 years ago

I agree with you. There should be solid documentation.

However, there will always be corner cases, and occasionally it can be worthwhile/necessary to dig in and figure it out, and the ability to do so can be a big win. Though one must be appropriately wary about relying on undocumented behavior.

sophacles2• 13 years ago

On the other hand, I out of hand reject libraries for which code is not available, unless forced by employers to do otherwise. Documentation is great, but at the end of the day, I still will throw a spike, no matter how good the documentation is. Similarly, I will look at the code to understand how the authors do an abstraction. It isn't to tie my code to something, it is to find out where the abstraction stops working. My applications aren't that special, and not tied to the code, but if I don't understand how something works, I can't wrap my head around how to utilize it properly.

Simply put: I'm going to write my own wrapper for just about any external code anyway - my internal API can then be changed on the backend to use new libraries, rather than changing all my code. I will do this anyway, because rarely is a new library a drop-in replacement for an old library. This, not avoiding an understanding of the code details, is how you keep yourself independent of implementation.

adventureloop• 13 years ago

I really dislike having to go into source code to figure out basic functionality. This is confounded by the amount of uncommented code. For a lot of the cases one or the other will do. No your shader is not self evident.

hamburglar• 13 years ago

The main problem with RTFC is that it tells you nothing about how the code is supposed to behave, which is important. Yes, I can see, by looking at the source, how this version of this code behaves, but unless I have an interface spec or some documentation, I have no idea whether this behavior is intended or not, and therefore I have no idea whether the behavior is something I can count on in v.next. This is one reason my current project uses a whole stable of ruby gems that we are absolutely terrified of ever upgrading: all we know about their behavior is what we can observe about one specific version of the source.

duwease• 13 years ago

I've been bitten by open-source AND closed-source applications recently, from different angles.

Once when using an Adobe application that was behaving erratically, and I was unable to see what was happening internally for myself (and obviously the behavior wasn't documented). Incredibly frustrating, and no recourse except to hack around it and hope it didn't mutate.

More recently, I've been bitten by a fairly popular open source application that I intended to use without modification (just configuration). But the documentation was so awful that I couldn't even figure out how to configure it. Ultimately, I had the recourse of stepping through the code to figure it out, but when you're trying to build adoption of your application, it would make sense to make it usable for its main purpose without setting up the actual source code in a debugger.

One interesting generalization I see is that the user-friendliness and documentation of the open-source app correlates quite often to the language used. PHP and Ruby projects seem to be built with an eye towards helping the eventual users use it. The Java projects, while often more competently written and complicated, seem to often be presented as code-only, take it or leave it, user-friendliness be damned.

stox• 13 years ago

I always preferred UTSL ( Use The Source, Luke ), instead of RTFC.

pothibo• 13 years ago

I think thoughtful, detailed guides on how to browse/use code is better than code documentation.

Too often (mostly in interpreted language), I see a method that takes a hash as an option and I see that option being passed down to 3, 4 or more different functions. So in order to know what options are available for a given method, you have to browse through all the other methods that takes the same hash as argument.

I haven't seen a framework dealing with that through documentation. Some are better than others, but they never cover everything.

jamieb2• 13 years ago

I'm using Spring 3.1, Jetty, various OWASP, guava. I am in the code of Spring and Jetty on a daily basis, and OWASP when I first used it. I learned guava by writing unit tests to verify my assumptions - assumptions I learned from the documentation - and which turned out to be wrong in key aspects.

Documentation is ambiguous, and even documentation which is unambiguous is often wrong (or perhaps more kindly, "out of date").

greenyoda• 13 years ago

"I realize complaining about the lack of documentation in some open source projects is like complaining that there is no foie gras at a friend’s free dinner party"

Actually, it's more like not having chairs at a dinner party. It's a significant discomfort, not a lack of a luxury.

"The Spring Security documentation says it best…

It is much easier to debug your application or to work out where a problem lies if you don’t treat the external code you are working with as a black box which you never look inside. The first thing you should do when an exception you don’t understand is thrown from an open source library is jump to the class and line number and take a look to figure out what the code was doing there. Otherwise you’re missing out on much of the benefit of using open source code."

Sorry, but it's a lot easier to debug your application if you have documentation that tells you under what conditions the library throws a particular exception.

Having to understand the source code of a library to be able to figure out its behavior eradicates much of the benefit of using a library in the first place. And the average product these days relies on so much open source software that you can't possibly have the time to dig through the sources to solve every little problem. Would the author of this statement be happy if he had to dig through Linux kernel code, or Apache web server code, or the code of the dozens of Java libraries that Spring is dependent on? I don't think so.

RTFC is the New RTFM

Related Stories