Efficiently Browsing Text or Code (esp. with Emacs)

http://www.kirubakaran.com/articles/efficiently-browsing-text-or-code.html

by kirubakaran • 14 years ago

104 49 14 years ago

kirubakaran.com

KC8ZKF• 14 years ago

(find-grep-dired DIR REGEXP)

Find files in DIR containing a regexp REGEXP and start Dired on output.

kirubakaran4• 14 years ago

That is definitely useful. However the scenarios I explored needed two or more keywords occurring on different lines in a file. This is so that I can significantly reduce the number of files that needed a closer look. Is that easily done with find-grep-dired?

walexander• 14 years ago

I'm an emacs coder myself, so I'm wondering.. why no mention of cscope?

kirubakaran4• 14 years ago

I haven't used cscope. I'll check it out. Thanks.

jbp3• 14 years ago

"cscope + ascope" is the best thing ever I found for groking the code(in emacs). It is great for getting your way around huge code base. I find myself using "ascope-find-this-symbol" all the time, which gives me all occurrences/usages of that symbol.

signa11• 14 years ago

yeah without cscope it would be impossible to follow huge code bases. though i am not sure how good is cscope at generating symbol database for non-c languages.

ideally, it would be really nice if cscope/xcscope provided some means of creating (and saving) bunch of call-stacks, which you can navigate in forwards/reverse direction (not sure if i am making much sense here...)

codemac2• 14 years ago

I know there is at least, cscope.el, xcscope.el, acscope.el, bcscope.el...

What's the advantage to acscope vs. the others? I personally use xcscope, I don't even remember why I initially chose it...

Peaker3• 14 years ago

cscope and ascope still cannot find particular struct fields.

decklin• 14 years ago

No mention of M-x grep?

zacharypinter• 14 years ago

I find M-x rgrep extremely useful as well.

kirubakaran4• 14 years ago

I left that out as I needed to filter for files with two or more keywords occurring on possibly different lines. I don't know of an easy way to make M-x grep do that.

wladimir6• 14 years ago

It's 2012 and we're still navigating source code, which is effectively a nested graph structure, using grep and simple keyword queries.

No (reverse) call trees, no high-level views, no help on how data propagates through the program. We could do so much better, especially in static languages. Sometimes it seems like reverse engineering tools that work without source code (such as IDA) do a better job of cross-referencing than tools that have the actual source code.

Is there any good/usable point and click code browser / code comprehension tool these days? Or is the state of the art of 1998 still that of today?

drv• 14 years ago

Visual Studio has some fairly decent tools ("Find all references", "Go to definition/declaration"), although they tend to fall over in macro-heavy C++.

A lot of my coworkers swear by Understand, but I haven't tried it myself.

Derbasti• 14 years ago

If only these tools would wirk reliably. I have seen too many cases when find all references did not find the reference I was looking for (even though it was in no way special or obfuscated). Goto definition works pretty well though.

tikhonj• 14 years ago

We're still doing this for two reasons: it's easy and it works surprisingly well. More complicated solutions would be difficult to implement and would probably be less flexible. It's just difficult to be significantly more efficient than somebody really good with Emacs using mostly code agnostic tools, except in real edge-cases (finding the right variable named x our of hundreds of xs is going to be difficult, but if your code looks like that you have bigger problems).

Also, I really wish the myth that "point and click" is inevitably superior would go away. It may be easier to learn, but my experience is that with some practice keyboard-based programming tools are almost invariably faster.

wladimir6• 14 years ago

I wasn't trying to claim that "point and click" is superior. It was just my question whether such tools are available. I'm also more of a command line monkey, but it gets old at times to do the same thing I was doing 10 years ago over and over.

My point is that much more intelligent, useful and efficient queries could be possible by making use of the fact that code is a data structure, and not just text. Various things could be inferred automatically, which would be especially handy in legacy C source code where it's really hard to make head or tail of (I see a lot of that in my job).

It should be able to answer questions like "where does this value come from", "how is it computed" and show a tree/chain of statements. Or "where is this structure accessed" ... and so on.

As clang makes these things easier, I hope there will be a new resurgence in intelligent code comprehension tools. Those don't necessarily have to be point and click, but could have different UIs...

jcheng• 14 years ago

All the good (heavyweight) Java IDEs have had excellent cross-referencing tools for well over a decade at least.

dkarl• 14 years ago

Is there any good/usable point and click code browser / code comprehension tool these days? Or is the state of the art of 1998 still that of today?

That's a lot to ask for code that the editor may not know the compile and link options for, which may not even compile using the right options, or which may only be understood by a compiler that isn't installed on the system where you're browsing the code.

Basically what you want is an IDE with a project set up telling it exactly how to compile the code or what the runtime setup will be for a dynamic language. Then you'll have decent code navigation, if the IDE is integrated with a compiler that can handle the code, and if there aren't any bugs! I can tell you that for Java the problem is thoroughly solved. The language is simple, and the IDEs are mature. For C++ you're SOL. Scala is getting there with Eclipse and IntelliJ. I don't know about other languages.

I for one enjoy knowing how to handle unstructured text, because you can never count on having tools that will understand everything you come across. XML configuration files, Java projects that have Ruby build scripts for unknown reasons, log files, there's an unlimited number of text formats that some tool somewhere knows how to parse, but it isn't worth your while to spend half an hour setting up an obscure dev environment for a one-off job that you can do in five minutes in emacs.

tptacek4• 14 years ago

It is very hard to do this for C/C++, because C/C++ code is customized at compile time by the build environment and usually can't be parsed and analyzed without that customization.

The tools therefore need a fully working build environment, and the ability to temporarily override aspects of that build environment.

turnersr2• 14 years ago

People at Mozilla have created a lot of neat tools for working with large C/C++ code bases.

Relavent Links: https://developer.mozilla.org/en/Dehydra https://developer.mozilla.org/en/Treehydra http://dxr.lanedo.com/ http://corp.galois.com/blog/2010/6/4/tech-talk-large-scale-s...

wladimir6• 14 years ago

It's a hard problem indeed. Hopefully that will motivate some people :)

morsch• 14 years ago

Eclipse gives you a wealth of tools to do that for Java. It's exceedingly rare for me to plain search (as in Ctrl-F) for anything in source code, I'm sure I go days without it[1].

It goes way beyond "find references" and "go to definition", too. For instance, marking a variable indicates all the usage points near the scroll bar, and ctrl-,/. moves between them. Ctrl-T opens an index of all classes. Ctrl-O has an index of the current file. Shift-Alt-Up/Down selects increasingly large expressions (across lines, as well).

I guess it all sounds trivial, but taken together I navigate within a Java project in an entirely different way than I navigate source code for other languages or unstructured files.

[1] Grep and its ilk are obviously still common tools for things like log files.

mschnell• 14 years ago

The best tool I have ever used for that is CodeSurfer: http://www.grammatech.com/products/codesurfer/overview.html.

It supports slicing operations: Which downstream changes happen if I change this variable? What upstream control structures influence the value of this variable?

Great stuff.

wladimir6• 14 years ago

That's a great list of features, and exactly the kind of intelligent tool that I'm looking for. Too bad it's horribly expensive :/

swah• 14 years ago

This would be much easier if compilers/interpreters would expose their parse trees in a friendly way. They have all this knowledge about the code and won't give us.

wladimir6• 14 years ago

I suggest taking a look clang/llvm. They have a friendly API for access to parse trees and intermediate metadata exactly for purposes like this. And it's open source too!

aristus• 14 years ago

There are many roads to Damascus.

(global-set-key "\M-/" 'tags-search) (global-set-key "\M-'" 'tags-apropos)

res0nat0r• 14 years ago

ack and ack.vim = bliss.

heretohelp• 14 years ago

Indexing is better, but ack is still nice to have for ad-hoc searches.

Ack plugins for Emacs:

http://www.emacswiki.org/emacs/Ack

http://rooijan.za.net/code/emacs-lisp/ack-el

https://github.com/jhelwig/ack-and-a-half

http://www.emacswiki.org/emacs/FullAck

docgnome• 14 years ago

Ack is great

tptacek4• 14 years ago

Highly, highly recommend hilock-mode for browsing code; hilock does ad-hoc syntax highlighting, so, as you read code and notice patterns, you write regexes for them and let Emacs spot them for you.

This turned out to be so useful for code reviews (we do a lot of those) that I "ported" it to a web app that lets us set hilocks that follow us from buffer to buffer (along with xrefs, class/method/fn definition sites, bookmarks, and a lot of other junk).

kaeluka• 14 years ago

wow, that's amazing! Thanks for that :)

dw5ight• 14 years ago

awesome article, thanks K!

kirubakaran4• 14 years ago

Thank you :-)

abc_lisper• 14 years ago

Um. I thought I was inefficient at browsing Emacs and this sucks more. Here's what I do.

1. Generate a file index, every now and then(read cron).

  > find . > fileList

2. Write a script(can also do this in emacs) in bash that takes two arguments viz., filename and search string. The script does a two level grep(regex), first on filenames and then in each of those files for the search string

3. If the previous routine was written in bash written a elisp function that calls the bash script.

That's mostly it.

I could use cscope but it sucks donkey balls with java. When I ask for definition it finds a declaration.

Peaker3• 14 years ago

Then you want to search all uses of the Linux kernel's struct scsi_device field "vendor". Unfortunately, there are hundreds of structs with a "vendor" field.

cscope, etags, etc can't really handle this task. Eclipse and Visual Studio can.

I am really disappointed that emacs and vim who spout productivity over learning curve allow important productivity tasks such as this to remain more than a decade behind the state of the art.

counterpunt• 14 years ago

if it's really that important to you, quit bitchin and start coding!

Peaker3• 14 years ago

I am coding, but it's actually a more ambitious project to get rid of text in the toolchain altogether, making searches such as this much simpler cross-reference searches.

Erwin• 14 years ago

There's an affordable (USD 299, free 7 day trial) commercial tool for C/C++/Javaallowing structured, context-sensitive code browsing and refactoring that actually integrates into Emacs: http://www.xref.sk/xrefactory/emacs.html

In fact it seems the pure C version is free now; it claims full indexing of 3 million lines of Linux source code in 15 minutes.

I used to keep track of it many years back back while it was being developed, but then C++ starting making up 1% of my projects rather than 99% (nowadays I use the JetBrains PyCharm for those 99% which does a reasonable effort).

It seems the tool hasn't seen updates in a while (forget about C++11 support), but I remember it working much better than just tags, cscope last time I tried it many years ago.

Efficiently Browsing Text or Code (esp. with Emacs)

Related Stories