Unix as IDE

http://blog.sanctum.geek.nz/series/unix-as-ide/

by dchest • 14 years ago

204 104 14 years ago

blog.sanctum.geek.nz

geophile2• 14 years ago

grepping for "foobar" is very different from finding invocations of the function named "foobar". Cross-file search and replace, which is doable from emacs, is different from refactoring. I love emacs and unix, but sorry, it's not an IDE. Intellij, now that's an IDE.

udp• 14 years ago

Yes, you're right. But a whole bunch of people don't need or use things like automatic refactoring, and more often than not write better code than the people who do.

kleiba• 14 years ago

I don't use IDE's and I love Emacs, but sorry, for "more often than not write better code than the people who do" I feel like I have to invoke "citation needed"...

beothorn3• 14 years ago

The automatic refactoring on IDEs are error proof (I'm thinking about eclipse and java here). Saying that you can do a line without a ruler is one thing, but saying your line will be straighter, I really doubt it. IDEs are important for people that do refactorings. If you don't, fancy IDEs will slow you down. I don't do refactorings that I'm not sure if they will break my code. It's the same reason I don't code with dinamically typed languages and avoid using reflection.

geophile2• 14 years ago

There are people who write bad code, with and without IDEs. There are people who write great code, with and without IDEs. I suspect that everyone who writes great code does refactoring. Some prefer to do it manually, and some prefer the automation.

truncate• 14 years ago

Problem with IDE is that they are often bloated and take lot of space. Also I think I don't use more than half the features they provide. You infact get slower there when you get used to something like Vim/Emacs.

skrebbel3• 14 years ago

If UNIX gets language-aware tools that allow me to search-replace methods of a particular name (and not other identifiers with the same name), and if Vim gets autocomplete* that understands which packages/libraries/headers/assemblies I'm importing, then I'll seriously consider moving to UNIX as an IDE.

Until then, I fully agree with you.

* To be ahead of the haters: autocomplete isn't a trick for avoiding having to type some extra characters. It's a way for exploring an API you're only vaguely familiar with. I guess you need to be at least somewhat familiar with statically typed languages in a decent IDE to really appreciate what this can do to your productivity. I seldom need to refer to the API docs of libraries I use.

vog• 14 years ago

Autocomplete is not tied to statically types languages. For instance, in the Python command line prompt you have autocomplete directly on the live objects.

gizzlon2• 14 years ago

Get what your saying about autocomplete, but the "avoid having to type extra characters" kind of autocomplete is very useful as well. I use it to complete things like long variable names and English text. Its main strength is that you can use it to autocomplete everything, not just the stuff your IDE is aware of.

tikhonj3• 14 years ago

The thing is that UNIX already has those tools--they just naturally depend on language. So, for example, Emacs has js2-mode and js2-refactor which are about as good as you can expect for JavaScript. There are also similarly capable modes for Lisp.

There are also editor-independent tools like Ensime for Scala which is basically a server that provides IDE-like tools that can be used from Emacs or Vim. Scion provides something similar for Haskell.

There are also things like CEDET for C/C++ which add very IDE-like capability to Emacs. In fact, CEDET is basically just a framework for parsing languages and using that information for things like auto-complete.

Of course, there are some languages conspicuously missing--Java and C# for example. I tried using JDEE for Java a while back and it was really bad; I've never used C# but I don't think it's well-supported either. Ironically, these are two of the languages that need an IDE the most in my experience. (And the language that needs it least--Haskell--has some support.)

So if you're doing serious Java/C# development, then you have a valid point. If you're using another language, chances are that Emacs or maybe several editors can actually support it intelligently. (Also, maybe there are options for C#/Java that I am not aware of. If so, I would appreciate being enlightened.)

Figs• 14 years ago

You can get some of that functionality out of ctags. I mostly use it for the navigational aspect, but I've seen it used for autocomplete too.

captobvious• 14 years ago

IDE's: When things feel only vaguely familiar, and you know just enough to be dangerous.

johncoltrane• 14 years ago

Is someone begging you to drop your IDE for the UNIX toolchain?

IDEs have their use, Vim and Emacs have their uses, too. I'm perfectly satisfied by Vim (with omnicomplete and tags) for my JavaScript/HTML/CSS coding but I have to turn to Eclipse for AS3.

The "right" tool for the job.

muks• 14 years ago

Try cscope and xrefactory for emacs.

cscope is very good for searching through C codebases. You can use this with vim too. It has emacs integration using xcscope, and you can use sequences like C-c s g to quickly jump to a symbol.

xrefactory (though non-free) is very good with C++ navigation and can do refactoring. I have used IntelliSense with VAX and xrefactory is much better. This less known program is something incredible for C++.

rhizome31• 14 years ago

Also worth mentioning Rope for Python refactoring. Rope's extract method works rather well in Vim.

agumonkey2• 14 years ago

grammar/language oriented libs/functions are missing, nothing more. People still process language with regexps over bytestreams, while they shouldn't. Eventually, and not too long from now, this idea will sink in.

wink• 14 years ago

A great series of articles, well-written and informative, while not being zealous in any regard.

There wasn't much new to me in the shell-focused parts, but there's some great VIM advice hidden in there.

Despite the great VIM part it covers a lot of topics that should be common knowledge for people developping on unix-like operating systems (or at least they should've heard about them).

agumonkey2• 14 years ago

He landed a sweet spot here. Most blogs are either noob level introduction or fairly advanced, he wrote short/middle level. Very unixy style too.

scott_s• 14 years ago

I agree with his sentiment - it's something I have said here on HN several times - but I disagree with his word choice. Unix is not an integrated development environment. To me, an IDE implies a monolithic program. Unix is not that. To differentiate it, I'm tempted to say that Unix is a distributed development environment to get across that the tools you use for development do not exist in one monolithic program. But I don't like that phrasing, either. Rather, I prefer this:

Unix is a development environment.

rsanheim• 14 years ago

How about "DDE" -- decentralized development environment ?

malkia2• 14 years ago

A old windows hacker might get confused with Dynamic Data Exchange :)

Naah... the name is just good, call it what you want - IDE, PE,

mhd5• 14 years ago

Yes, one might say that the "I" part of IDE is there to differentiate it from the typical Unix style.

Kernighan and Pike called their book "The Unix Programming Environment" (sooo highly recommended), so I' go with "PE" instead of the high-falutin' "DE".

ajross3• 14 years ago

To be fair: the "I" differentiated it from the typical Unix style as implemented on MSDOS, which for obvious reasons sucked rocks. No editing while building, no pipes at all (and thus no log analysis in the traditional sense), etc...

tyrmored2• 14 years ago

This actually occurred to me as I was writing it; I tried to decide whether it was better to call it more accurately what it was, or refer to it as something with which people were more familiar and hence have a concept to which to relate it. So for reference I actually agree with your comment.

6ren• 14 years ago

Clayton Christensen talked about this idea as integrated vs. modular. He actually used the example of unix as modular and Windows as integrated (though he there was talking about the ability to create different distributions of unix, mix-and-matching components).

BTW: I like his use of "IDE" to convey his point succinctly. Though I had the same reaction as you, he's using it as a word, not an acronym. Also, there's a category error of application vs. OS which I think adds to the wry quirkiness.

DanielRibeiro• 14 years ago

I used to agree with this, but my mind changed after seeing Brent Victor's presentation[1] on how code and result can be integrated. Even Smalltalk's ambient can't compare (which is understandable, as it is much older).

[1] http://vimeo.com/36579366

olalonde• 14 years ago

`tree` is also a nice utility to make recursive directory listings.

kayoone2• 14 years ago

Id really love to do this, but there are a few things that would slow me down

* no autocomplete... On huge projects that are not based on some predefined framework structure (think rails) this results in a lot of "what was the parameter order of that function again or how was it even called ?"

* no easy refactoring. In VS i love how i can just hit F2 on a method name, rename it, VS shows me all the places where it will rename it and pressing OK. This works flawless everytime and makes refactoring so much more painless than working with a scripting language and a text-editor only

+ numerous other things like looking up where certain methods get called, where they are defined etc.

mjdwitt3• 14 years ago

"* no easy refactoring. In VS i love how i can just hit F2 on a method name, rename it, VS shows me all the places where it will rename it and pressing OK. This works flawless everytime and makes refactoring so much more painless than working with a scripting language and a text-editor only"

Using vim (or sed for whole directories of files) you can do the same thing using a search and replace regex.

    in vim:
    :%s/<find>/<replace>/gc

gnaritas3• 14 years ago

That's not at all equivalent.

bermanoid3• 14 years ago

...which works great until you have to rename a field with the name 'value' in one particular highly-used class in a system with ten-thousand classes, many of which will also have fields named 'value'. That's where actual IDE knowledge of the language semantics is more or less required, to be able to track the actual object being referenced based on the particular imports, inheritance hierarchy, etc.

kayoone2• 14 years ago

thats works for some cases but can by no means be described as "works flawless everytime" like the refactoring features of modern IDEs!

jiggy2011• 14 years ago

The thought of doing global search and replace across an entire codebase (or even a section thereof) gives me a knot in my stomach.

Hope you have very good unit tests.

ruediger2• 14 years ago

You can have all of that in Emacs and probably in vim as well.

See ctags/etags, cedet, ropemacs (refactoring Python), xrefactory (refactoring C and C++), wrangler (refactoring Erlang), etc.

gchpaco2• 14 years ago

True, but that could be argued to be Emacs-as-IDE. Cscope enables a better solution to this problem (with a clumsy interface) but solving it for good would require solving the structured-data-on-the-Unix-streams-model problem and I've yet to see a good solution to that.

People kludge them to fit all the time—look how older ctags implementations worked, after all—but it reminds me of a guy I who would write paragraphs so that each sentence had its own "line" and relied on implicit word wrap in LaTeX, HTML and the like to format it for human consumption. He did this so it would work better with tools like diff, patch, and version control in general. I thought "hey, worth a shot", tried it, and found it too clumsy for me during the writing process and abandoned it soon after.

bh42222• 14 years ago

For autocomplete, I don't know enough about vi or vim, but emacs has it:

http://stackoverflow.com/questions/1616753/make-emacs-semant...

For refactoring, that's just an advanced text across many files manipulation feature, and emacs certainly can do it, but you might have to work on your .emacs file.

The numerous other things sound like either things you can do in gdb or valgrind, or again just advanced text search/editing.

Now all of this is not elegantly pulled together. You need to create this for yourself, make all these tools work together by developing a development environment for yourself. But once you do, you may be surprised just how elegantly everything works together.

irahul2• 14 years ago

> For refactoring, that's just an advanced text across many files manipulation feature, and emacs certainly can do it, but you might have to work on your .emacs file.

Re-factoring is text editing, if you consider parsing to be advanced text editing(which it is not). Consider a class Foo implementing method foo, and a class Bar implementing a method of the same name foo. When I am renaming Foo.foo to Foo.bar, a refactoring tool will rename based on the semantics.

Renaming manually will always require looking through all occurrences, compared to clicking a couple of buttons in Eclipse and being assured it renamed all occurrences without any false positives or negatives.

Also, the nature of dynamic languages make it harder to refactor:

    class Foo:
        def foo():
            pass

    class Bar:
        def foo():
            pass

    def a_func(arg):
        arg.foo()

Now, in arg.foo, arg can be an instance of Foo, or Bar, or Baz...In a large codebase, manually doing it will be a night mare.

I mostly use vim, linux and dynamic languages. But that doesn't change the fact that Eclipse/Java is leaps and bounds ahead when we are talking re-factoring.

Also, I use rope for Python. Though it mitigates most of the warts, it can't mitigate the issue in the snippet above. There is an `unsure` mode which renames when unsure, but that's not a smart thing to do on a large codebase.

drhowarddrfine2• 14 years ago

You NEED those things in VS in order to keep track of the 20K+ objects, code, libraries, etc. to code for Windows and .NET. You don't need all that in Unix.

ianpurton• 14 years ago

Unix also makes a great web based IDE.

http://servermonitoringhq.com/blog/the_ultimate_web_based_id...

un1xl0ser• 14 years ago

So I work in C, and navigating the code can be a problem from a terminal. I use cscope, but it wasn't really useful until I started using a cool set of scripts that this consultant had which opened new screen (or tmux, if you modify the scripts) windows for each file I am editing. I'll nag him to post this on github.

I used to find/xargs/grep to find code snippets.

microtonal• 14 years ago

If you’re using version control for your project, this often includes metadata in the .svn, .git, or .hg directories. This is dealt with easily enough by excluding (grep -v)

Actually, some version control systems allow you to grep managed files specifically. E.g. in Git using git grep, including specific commits, branches, etc. Tim Pope's Fugitive git plugin for vim also does this (:Ggrep), and puts results on the quickfix list.

tyrmored2• 14 years ago

Op here. That part is mostly to use a familiar tool first and then talk people into trying Ack out. I don't use grep anymore for code work either.

tutysara2• 14 years ago

I use Eclipse for my day job, working on a enterprise java project. Here are some things I got used to working with Eclipse,

1. Most of the coding are done to interfaces and concrete classes are wired using factories or using spring dependency injection like,

InterfaceName obj=ObjFactory.getImpl(args)

I have to look at Class hierarchy often to go to the method that is being invoked, this is very easy in Eclipse (C+T)

2. Code completion for all methods (methods in different source files and from jars).

3. Highlight errors while editing.

4. Tracing call hierarchy

I tried Emacs some time back for shell scripting on a remote machine and I found that to be very useful and a right fit.

I would love to use Emacs for my java development if there are tools out there which can help me do things that I do with Eclipse. I already feel sluggish with Eclipse some times moving the mouse ;-)

irahul2• 14 years ago

If you predominantly code in Java, you are better off working in Eclipse, and using some plugin for whatever happens to be your favorite editor.

I used to do a lot of Java in the past, and I have used eclipse and vim. Java in plain Vim is a huge PITA - auto complete, generating dummy getter setter, refactoring, fixing imports, checking errors without leaving the ide etc etc. I haphazardly fixed a couple of them using vim plugins and custom keybindings; and then I found eclim, which uses eclipse as a server and provides the functionality in Vim.

tutysara2• 14 years ago

eclim would be a better fit if i am going to use emacs as my do-everything IDE. I found that eclim has even a connector for emacs as it has for vim.

tikhonj3• 14 years ago

Yeah, I've used Emacs for Java in the past and the support is not good even if you use JDEE (which is outdated and slow anyhow). I'm an Emacs fanatic, but even so think you would be better off with an IDE in nontrivial Java programs.

I solved the problem by not using Java, but this is obviously not an easy option for you :). Emacs does support Scala fairly well (from what I've heard) with Ensime, for it's worth.

mhd5• 14 years ago

Don't forget that at one time, Apple packaged a whole Unix environment explicitly for that purpose (MPW).

drhowarddrfine2• 14 years ago

OSX is certified Unix.

mhd5• 14 years ago

Which is why you don't need MPW anymore, but in the days of System 7…

chj• 14 years ago

yes, it is an IDE, but sorry it is a very bad IDE.

telent• 14 years ago

"I think that trying to shoehorn Vim or Emacs into becoming something that it’s not [an IDE] isn’t quite thinking about the problem in the right way."

This may or may not be true for Vim, I don't know. But I think the majority of GNU Emacs users would view it as, to a large extent, supplanting the unix shell environment. Stallman notably didn't even use unix until he had written enough of emacs to be able to avoid ed and vi: he edited on another system and transferred the files across the network

slowpoke• 14 years ago

That's something I've been saying since a while, and a very nice reference for various tools.

Also, there's another strong argument I like to use: the UNIX philosophy of "Write programs to do one thing, and do it well". An IDE tries to do way too much (ie, everything) at the same time, from start to finish. That's highly inflexible and - in my opinion - terrible design.

brnstz• 14 years ago

I work like this usually. Most I do Python, PHP, Javascript, and database work. I spend the vast majority of my day in iTerm or a web browser. Sometimes I wish I set up something more "integrated" but grep ends up being good enough.

When I worked in Java, I used Eclipse and hated it. Programming felt like filling out a contract.

When I worked in Common Lisp, I used Emacs and in combination with either Slime or Allegro environments. These were ideal to me. It got out of your way, but it had enough power and introspection into the code so that it could find almost anything (function/class defs, etc.) The debugger was also really good.

nagnatron• 14 years ago

I use Unix as an IDE but refactoring is much, much harder without dedicated IDE support.

enneff• 14 years ago

Nothing explains this philosophy better than Pike and Kernighan's Unix Programming Environment. Worth reading even if you consider yourself fluent in the Unix environment.

jfb• 14 years ago

sort? grep? find? ls? Maybe I'm just jaundiced but those tools are far too primitive to be even compared to e.g. a Symbolics machine, let alone a modern IDE. And POSIX is laughably far from treating everything as a file, or having a small set of orthogonal tools, or indeed having anything like a coherent worldview.

gchpaco2• 14 years ago

Yes, this is true, modern Unix isn't remotely pure. When I think of where it could be for these discussions, I think of Plan 9. But even on Unix, syntax aware tools like cscope and ctags are fairly well integrated. The real weakness is the debugging story; modern Unix simply isn't as good at introspection as it could be, or arguably should be. Some of this is the ahead of time C compilation model making hot fixes hard or impossible, some of it is limitations in debugging tools and tracing tools, some of it is that the Unix stream model doesn't do a very good job of communicating structured data; it's a hard problem.

But, then, even Eclipse compares poorly to Genera, and it evolved from VisualAge for Java which was a near port of the Smalltalk product. So it isn't as though the competition is distinguishing itself through its excellence; look at how much effort in terms of code it is to integrate FindBugs with Eclipse—and all it does is some trivial static analysis!

malkia2• 14 years ago

Even on Windows, I do work mostly from the command-line, but with batteries - that is I use Far Manager (think about midnight commander). Then I have few shortcuts, to cmd.exe, and some to cmd.exe + visual studio's cl.exe in the path (32-bit one, and 64-bit another). I used to had even set-up to run Far Manager under other shells.

And then I have cygwin, sometimes it's pain, for the symlinked files, as they don't run directly from cmd.exe, but a simple 'sh -c "gitk --all"' works. And now with the latest cygwin, that moved totally to X11 for tcl/tk, I need startxwin.cmd started somewhere (startup).

For places where cygwin overrides some of the CMD shell commands (and I'm quite fluent .bat file writer, and still use them a lot), I just do "call dir *.c /s/b", call, because cygwin has overriden my dir with their application, and I got tired of deleting it, rather I do "call dir" if I need the CMD shell.

grep, sed, bash, find, etc. are really awesome tools.

Lately I've been making a portable luajit distribution, and found out that BUSYBOX, being almost linux exclusive, can be compiled for Windows (mingw) and OSX. It packs a lot of the commands + shell + few other things in one executable.

And most of the time these commands would work the same way on each system. So "sed" would be the same sed on OSX, Linux and Windows... I'm thinking of moving to it, as right now my reliance on cygwin, while being good to me, makes me isolated if I want to share some of the scripts with coworkers (I can't make them install cygwin, but they won't have problems with 600kb busybox.exe)

The best IDE so far for me has been the command-line.

nitrogen• 14 years ago

The primary rationale for using an IDE is that it gathers all your tools in the same place, and you can use them in concert with roughly the same user interface paradigm, and without having to exert too much effort to make separate applications cooperate. The reason this becomes especially desirable with GUI applications is because it’s very difficult to make windowed applications speak a common language or work well with each other; aside from cutting and pasting text, they don’t share a common interface.

This makes me wonder -- could one create something like the GUI equivalent of UNIX's modular philosophy? Desktop systems like OLE and KDE's kparts don't seem to have accomplished that goal. What about Web Intents?

dfc3• 14 years ago

What does this mean:

    /[a-z]{3,4}/i

eapen• 14 years ago

This is a case-insensitive regex search for strings with 3-4 letters in them.

dfc3• 14 years ago

Yikes, I'm sorry I pasted incorrectly. I meant to post:

What does this mean:

    Just another  /[a-z]{3,4}/i hacker

I understand the re but I am curious about the context.

luriel• 14 years ago

"The Unix Programming Environment" by Brian W. Kernighan and Rob Pike, published in 1984.

Still by far the best thing written about programming with Unix, and perhaps even the best programming book ever written (up there with K&R).

maw• 14 years ago

Grey text on a black background. Won't read it.

Who thinks it's acceptable to do this?

chernevik• 14 years ago

I've done my learning in the Unix programming environment. It has forced me to learn the Unix toolset: grep, find, piping, little bash scripts for common tasks. Getting closer to the operating environment has helped make sense of the interactions among distinct components, ie how Python talks to Apache and the database.

I sort of trusted that lots of smart people have used this environment and that I'd learn a lot trying to emulate them. So far the only major disappointment has been the vi scripting tool, vimscript just feels like a dead end.

I do imagine that there is a lot of received wisdom embedded in something like Eclipse -- just browsing the menus would probably suggest all sorts of helpful tools and practices. I keep meaning to check it out, but just haven't made the time.

Unix as IDE

Related Stories