Where is the casual programming?

http://iamsamy.blogspot.com/2012/04/where-is-casual-programming.html

by mohamedsa • 14 years ago

79 79 14 years ago

iamsamy.blogspot.com

>feels too much like piecing together jigsaw puzzles where you first have to know which pieces actually exist.

Well... that describes programming in many ways. That's not to say that shell scripting is the perfect casual programming environment - in fact, it certainly isn't! - but that anyone dabbling in things will certainly come to feel that problem. When I hacked in BASIC thirty years ago, then moved to Pascal, and then to C, that's largely how I felt the whole time.

petercooper2• 14 years ago

As a direct answer to the question and based on the problems the author raises, OS X's "Automator" is just that. It's casual yet still pretty powerful automation/programming. It loads fast, it has a visual IDE of sorts, and it can do a lot of jobs you'd otherwise use shell scripts for.

mohamedsa8• 14 years ago

(author here) I'm not a Mac user, but took a look at Automator now. It seems a big step in that direction. Still, I'm not sure it makes the casual programming problem solved. Maybe I misrepresented the problem; let me give some more examples on casual programs that could be made easier...

* Quickly testing small game ideas or demos (programming as self-expression).

* Solving an ACM problem or one of those programming job puzzles without too much setup.

* Writing the blog tagger example in the blog post, and other "do my web tasks for me" code.

I think a good casual IDE could be written for various types of needs, not just automating the OS or Apps..which are certainly an important part of the problem.

petercooper2• 14 years ago

Yeah, Automator is essentially a sort of group of visual DSLs for some very defined domains. Your examples here are definitely more in the realm of general programming languages.

It's definitely not as easy as the environment you speculate, but Xcode and iOS development have been making big inroads lately, even with total beginners. Certainly, it could be made a lot easier, but efforts to do this in the past have tended to fall flat as people think that the skills they learn won't be useful in "real" environments later on.. which turns it into quite the chicken and egg problem :-)

I wonder, however, if JavaScript and Web oriented development environments will ultimately fill the gap you're identifying. The browser is an ideal environment for the tasks you outline, and JavaScript is a fine 'base' language for DSLs to fall on top of while still giving access to the full language.

j_baker• 14 years ago

It sounds like you might be looking for something more like squeak: http://www.squeak.org/

juiceandjuice• 14 years ago

Are you prolific or at least very capable at using grep, sed, awk, tr, sort, uniq, cut, wc, and cat?

I don't think you need to read a book about shell scripting, but I think if you are familiar with these tools, there's a whole lot of things you can do with pipes. If you are opposed to learning them (and regular expressions), then there's no language that will ever please you.

Most importantly, except for some caveats, these languages are portable and preinstalled on most everything posix.

muyuu3• 14 years ago

Give this a shot http://docs.racket-lang.org/quick/index.html

See the bottom link too.

dasil003• 14 years ago

I think there are two problems here, both of which could be chipped away at, but neither will be completely solved anytime soon.

First, you have the fact that users don't know enough to solve abstract problems with programming. As programmers we have this deep base knowledge that we subconsciously draw upon. It's impossible to create a platform that makes it trivially easy to "test small game ideas" without narrowing the scope to trivial game types. People need a foundation to figure out how to put the pieces together. Fortunately, a lot of this is just familiarity with computers, so every new generation gets better at this, making the problem more tractable over time.

The other, is that software is truly an organic ecosystem. There is no designer that is coordinating the software world. The closest we have are closed platforms like Apple provides (which is still limited in overall scope) or standards (which are defined in a deliberately narrow scope for modularity's sake). So to make a single uniform environment that could do everything a casual programmer might want to do, and make it consistent, you'd have to write some kind of monolithic standard and convince a large minority of all software writers to support it in some fashion. The UNIX philosophy and POSIX are attenots to do something like this, but as you alluded to, it's still highly non-trivial to get your footing underneath you. To create an even higher-level standard seems almost impossibly difficult, but I suppose it's possible that progress could be made by focusing purely on a new high-level language and GUI. I'm not holding my breath though.

zmj• 14 years ago

Casual programming happens in Excel every day.

ken• 14 years ago

Indeed -- as long as the problem domain is "strings and numbers", and doesn't require any I/O. That's a pretty small fraction of the things I actually need to do. What would his 7-line shell script look like in Excel?

The great thing about functional languages is they have no side-effects. The lousy thing about functional languages is they have no side-effects. :-)

justincormack• 14 years ago

Thats true, but people use other worksheets as kind of declarative IO areas (that they print). And Google Docs is a big improvement as your spreadsheets get URLs and so enter into the outside world.

MichaelGG• 14 years ago

There is one statically typed language with a nice toolchain that allows interactive (REPL) programming and scripting: F#.

Try the steps here[1]. They're old, but still work, and work pretty well. It's fast; there's no need to create a project or solution. You can type your code all you want, use IntelliSense and all that, then select and Alt-Enter to execute it.

If you don't want to load VS, you can just run "fsi" for the interactive shell alone. You'll lose the full IntelliSense and colour highlighting, but you still get tab-completion. Plus, F#'s syntax is much lighter than C# or Java, so it's nicer for quick scripts.

Quite frankly, for any level of programming, I find having a REPL available is such a massive productivity boost. I shouldn't have to give up static type checking and a slick IDE to get it.

http://blogs.msdn.com/b/jomo_fisher/archive/2008/08/25/f-scr...

spacemanaki2• 14 years ago

And F# is not the only statically typed language with a (built-in or out of the box) REPL: ML, Haskell, OCaml, and Scala all have REPLs.

Some of these predate Java and C#, but I don't know what it is about those languages and the cultures around them that has ignored the value of a REPL. I guess part of it is just inertia at this point, but it's definitely one of the biggest things I miss when working in Java on Android.

jetti• 14 years ago

The problem is F# has arguably the best tool chain out of all of the languages you listed. Hell, I still have problems getting Scala to work with Netbeans 7.1 and I'm not sure of any IDE (for windows at least) for ML, Haskell or OCaml.

j_s• 14 years ago

If you've got Silverlight, http://www.tryfsharp.org/Tutorials.aspx

uncr3ative• 14 years ago

with ipython you do get some of that auto completion for python. It'll autocomplete the module name for you, but you do have to figure out you want that module first, so you'd need to google and realize you want shutil for dealing with files. I use it a lot when I'm interacting with new APIs or even just for writing the code as I go.

  In [1]: import sh
  sha     shelve  shlex   shutil  
  In [1]: import shutil
  In [2]: shutil.
  ...
  shutil.copy  shutil.errno  shutil.move
  In [2]: help(shutil.move)

To do other more powerful automation kind of stuff, this is pretty nifty: http://sikuli.org/

kami8845• 14 years ago

check out path.py

https://github.com/dottedmag/path.py

it's awesome for dealing with paths instead of having to do stuff like

    os.path.abspath(os.path.join(os.path.join(x,y),z))

everytime, it gives you nice abstractions like

    path('/home/me').files('*.jpg')

wiredfool• 14 years ago

Os.path.join can take more than two arguments. So it's not as bad as that example.

_rs• 14 years ago

I started programming with PHP, and I was easily able to apply all that knowledge of (rather basic) web-dev to the CLI. If I need a quick script my first instinct (besides bash) would be to fire up TextMate and write some PHP. As a beginner, whenever I needed to do something Google always had the right answer (e.g. "php check if string contains string"). This helped because I never felt like I was reading tons of documentation, and always gave me a quick answer. Once you use the same function a few times you generally remember it.

gte910h2• 14 years ago

He sounds like's he's described VBA

mohamedsa8• 14 years ago

VBA was a very nice step towards the goals I've described!

gte910h2• 14 years ago

As a person who did a good bit of that professionally back in the day (I wrote interface layers between it and deeper C++ code), I'd have to say, it's probably not "the thing" you really need, but it does sound quite close to what you requested.

read_wharf7• 14 years ago

Maybe, sorta.

Why don't people use VBA more for casual programming? Is it because it's Office-centric-ish?

Or, why don't more people do casual programming of any kind? Certainly lack of tools is one. Another is probably that most people don't exist in a culture where people do casual programming.

The biggest problem is that the things above the level of raw bits, the objects like documents and blog posts and check registers, are not themselves built to be manipulated in a casual environment. They're not built as part of a casually programmable environment. They're bits.

If you had a stack of books in English and Arabic and wanted to categorize them, you'd grab them and put them in their piles without thinking about it.

A blog post is nothing but bits. The only reason it's an English or Arabic post is that every program that touches it is required to, and agrees to, sniff it at a very low level and pretend that it's a blog post. The inside of a blog post looks exactly the same as the inside of a check register, and it's a huge leap from the bits that they are to the pseudo-objects that all relevant programs agree to pretend that they are. There is no casual middle ground, and so there's nothing that can be casually picked up and manipulated. You're a wizard, or an ardent apprentice, or you're nothing.

I think a real casual programming environment would look much less like app-centric VBA, or Excel (mentioned elsewhere), and more like Squeak (mentioned elsewhere). But it would work best if all the things on the system, at least the things that aren't dangerous to expose, are fundamentally casual objects. The browser window should be casual, as well as the post plugged into it.

georgieporgie• 14 years ago

why don't more people do casual programming of any kind?

I'm not sure I understood what the author meant by casual programming, but the first thing I thought of as I read your comment was HyperCard.

mvanveen• 14 years ago

My teammember's remark when I mentioned this made front page: "Still reindexing."

mdn0420• 14 years ago

Writing a shell script after some trial+error+googling sounds pretty casual to me.

drewcrawford2• 14 years ago

It's not as if Python or Ruby have set out to be deliberately difficult. They've grown difficulty because they're used to solve real problems, and real problems are hard.

Implied inside your question is the idea that we can, in advance, choose the right level of abstraction. That we have a library / language / set of verbs that is neither too high level (like Automator) nor too low-level (like your bash script example) to solve the problem. While I think we can improve a little bit on the set of abstractions that we have, I think a broad solution is naive.

As a point example, I used to be frustrated that so-called 3D engines tended to grow until they included the kitchen sink. Why do I need a menu system as part of my 3D engine? Why do I need a sound engine? Can't I just use system libraries like a sane person? etc. All I want is a set of primitives that let me render models to the screen. How hard can it be?

It turns out that the problem of rendering graphics to a screen quickly is so hard that it actually infects the hardness of other problems. Doing elastic collisions is a trivial thing that any first-year physics student can do. Until, that is, your game loop skips six frames because it's busy rendering graphics code, at which time you need to retroactively handle collisions that have occurred in the past. Now we're doing rocket science, and so your graphics library needs its own physics system, because you're not going to derive that stuff. And that, basically, is the story of every feature added to every serious game engine that exists today.

You see this problem everywhere you look, for example, at one point people thought POSIX and C were reasonable building blocks that everyone would use, and at another point people thought that about Java. At first, BSD sockets was the primitive, now we're starting to think about HTTP connections as the primitive, maybe we should work to make REST requests in Python not suck so damn much? But at this point SPDY looks like it might be turning into a thing, so maybe we should just build primitives for that instead? If you look at this staggering tower of abstractions you start to see things that seem crazy, like SPDY including its own flow control even though TCP already has it, because TCP has some bad assumptions that should be really be fixed at the network level, but will never be fixed there as a practical matter. I hope you followed Inception, because if not you're gonna have a bad time.

I'm doing a bad job of explaining this, but I had this insight first after reading an article by Kamp, the author of Varnish: https://www.varnish-cache.org/trac/wiki/ArchitectNotes. He makes the argument with a specific example that as we have moved to higher-level primitives, we've built an overly simplistic mental model of how the lower-level ones like OS and hardware work. As a result, we implement our own caching code, because we've forgotten that OSes were designed to do that for us. So in a sense we have more power because we have this nice set of high-level primitives, but in practice we've forgotten all the things that those primitives wrap, we've started to believe that they are simple because they look simple. And so, we muddy up our application code with all this stuff that has already been solved.

I doubt very much that there is any software program that people actually use that is not either undergoing a big refactor right now or has one scheduled, either to make things "simpler" (less abstraction) or "more powerful" (more abstraction). Just thumbing through the projects that I work on, I am adding features to a mental backlog for a future refactor for every single one of them. This seems to be like a fundamental problem that all software projects have, and creating a new set of primitives and saying "no seriously, that should do the trick" is at best going to yield a nice little language like Python or Ruby that somebody will post articles about on HN saying why it's so hard for casual use.

mohamedsa8• 14 years ago

The languages are mostly fine (until some next-gen language surprises us all); I think the current problems lie generally within tooling and infrastructure, which in turn come from our mentality and the existing workflow that we're used to.

That said, I enjoyed reading your very insightful post :)

read_wharf7• 14 years ago

"It's not as if Python or Ruby have set out to be deliberately difficult. They've grown difficulty because they're used to solve real problems, and real problems are hard."

The problem of getting from the East coast to the West coast has not changed in 100 years (move body until done), yet when was the last time you had to set the spark and choke and hand-crank a car? Cars solve real problems, and today's cars are sophisticated enough that essentially anyone can use one. To solve real problems.

Certainly the problems solved by a car are in a more narrow domain than programming languages. But cars themselves are in a narrower domain than internal combustion engines.

Edit: stealth.

jacalata• 14 years ago

Cars aren't used to cross the continent today, airplanes are. And most people have no idea how to use (pilot) an airplane, they pay someone else to do it for them.

mechanical_fish• 14 years ago

Certainly the problems solved by a car are in a more narrow domain than programming languages.

You do realize that this is like saying "certainly two times N is smaller than two to the Nth power"? ;)

drewcrawford2• 14 years ago

> The problem of getting from the East coast to the West coast has not changed in 100 years

But computing problems _have changed_. Not necessarily in areas like problem size (although that has happened) but also in areas of usability and convenience.

A much more apt analogy would be: the problem of stopping at the grocery store after work is 50 years old, why can't I get an optimal set of directions that accounts for rush hour traffic in 2012? I can give you any number of reasons: we've built more roads, there are more drivers, travelling salesman turned out to be a hard problem, game theory is hard, etc. In some dimensions (cranking the car, proclivity of gas stations, etc.) the situation has improved, but solving some problems re-emphasize the ones we haven't solved.

sophacles2• 14 years ago

I'm sorry, but this analogy is awful. Getting from coast to coast certainly HAS changed in the last 100 years. 100 years ago, you had railroads or mostly dirt roads. A crap ton of infrastructure has been put into it (interstates and the intensely complex logistics of maintaining them). A pile of services exists around this problem (airlines, and the even more complex logistics of keeping them running safely and profitably). Going broader, you are ignoring all types of "get between coasts" that apply to stuff rather than "me" - what is the best way to get a pile of widgets from LA to NYC? Airplane? Train? Some trucks? -- the answer is "well that depends on $logistical_considerations". Going narrower, choosing the right car for you is a pretty complicated process, trading off features, options, gas mileage, capacity and so on - just like any set of programing options, only internalized by the culture".

Further, a car is simple to use out of the box, but customization beyond do-dads and bling? Nope, not really easier than it ever was. Different tools? sure. Different base skills? not really. Anyone can do it? Not so much.

And before I go into tech equivalencies here, if you think cars are pretty much the same and can be abstracted to "the car" for the purposes of this discussion, go buy a van or pick up truck. Watch your social graph get all buzzy and see the request pour in for help, turns out your form of tradeoff in "car" are very very useful to people, and suddenly a car isn't just a car.

Further, I would argue that we have done wonders for the tech equivalent of "car". We have amazing computation devices in our pocket that just do what people never knew they wanted to do in the form of "phone". We have awesome abilities to have web presence unthinkable (outside of internet famous blog machines) 10 years ago, in the form "social graph". These are great and just work for most people, until they need something different...

In the infrastructure department: the ability to do semi-custom stuff has grown in ways unimaginable in the car world. Need a website to do your magic? Here are a bunch of platforms like app engine or heroku, all you need to do is get your custom stuff in place, don't worry about the machinery to feed it. Up and down the customization level there are more entry points as well. Going back to the car world, how easy is it really to get a ricer put together? How easy is it to put in crazy hydrolics or a good thumping stereo system? Need to rip off the doors, reshape the body and put in a different engine? Good luck getting a doctor to just do that without serious learning curve.

Basically, my point here is that hard stuff is hard. If it isn't your specific field, it is easy to just blow off the stuff that caters to the masses as "they made it easy", and at the same time, it is easy to blow off the stuff in your field which caters to the masses as "not done well".

Final thought: a lot of people look at cars and computers the same way, as soon as the littlest thing goes wrong, they freak. or they take it to the mechanic who asks "why haven't you changed the oil in 7 years? (c.f. our IT guys who ask why they don't have anti-virus installed).

_delirium• 14 years ago

The 3d-engine example is interesting, because I think there really have been huge improvements there in the past 5 years, through a mixture of UI and architecture. There are a wide range of "should be relatively simple" things that used to be unreasonably hard, but which with the advent of Unity3d are now much easier. Doesn't solve everything, but has given a several times improvement for a lot of low- to mid-hanging fruit.

defdac• 14 years ago

I have started to look into Unity3D the past couple of weeks and I'm impressed. Deliver to Android/iPhone/PC/mac/xbox etc. I have never felt so close to make my dream game.

RollAHardSix• 14 years ago

No, you did a very good job of explaining this. Just wanted to let you know.

conductr• 14 years ago

It sounds more like you just want the machine to read your mind.

edit: not saying that's a bad thing, IBM says 5 years to go http://mashable.com/2011/12/19/ibm-mind-reading/

aidenn0• 14 years ago

Lisp people don't remember the various functions or their arguments, they use SLIME which autocompletes and allows for looking up in the hyperspec.

josyw• 14 years ago

F# 3.0?

http://msdn.microsoft.com/en-us/library/hh156509(v=vs.110).a...

irishcoffee• 14 years ago

Like anything else, things worth doing well take time to learn.

There are great tutorials out there to teach you the very basics of programming in a few hours. There are also great tutorials out there that teach you to play a basic song on the guitar in a few hours.

After that, its up to you. If it were easy, everyone would do it. I've been programming for close two two decades, and I still feel like I'm scratching the surface of programming. As various iterations of the saying go: The more I learn about programming, the more amazed I am that computers even boot up and function.

ef4• 14 years ago

It's a red herring to bring up the static vs dynamic typing issue, because both can readily support interactive autocompletion & documentation.

Even something as Spartan as the Javascript REPL in Chrome will do autocompletion of method names, despite being in a completely dynamically typed environment.

strager2• 14 years ago

> var n = 42; n.[[TAB]]

No auto-complete. It's severely limited to only live objects. Which may be an argument for REPL, but it's not an argument for "dynamic languages can have autocomplete".

heretohelp• 14 years ago

I have intelligent object method/property auto-completion in Python in Emacs.

The argument for "dynamic languages can have autocomplete" is that they already do in a variety of modern editors (like Emacs) and IDEs (like PyCharm et al).

You're not only ignorant, but proudly so.

vectorpush2• 14 years ago

I can't think of anything more casual than a shell script. Once you grok the syntax of the shell's control structures it becomes pretty easy to rig up just about any behavior one can imagine. I suppose some might consider the command line excluded from the category of casual by default but most of those people would be surprised at how insanely simple the command line actually is.

read_wharf7• 14 years ago

For us, but not for people who don't program otherwise, yet would like to casually do more with their computers than what the installed apps do. Thinking of what you want to do is the more important part of doing something; it seems like, in the spirit of the OP's post, you shouldn't have to make such a large leap from what you want to how to do it.

It is what it is today, merely because it is what it is.

vectorpush2• 14 years ago

Thinking of what you want to do is the more important part of doing something;

Agreed, and those who don't program otherwise are at the cognitive mercy of the metaphors defined by a "user-friendly" abstraction. Of course, there is always a balancing act; we all rely on abstractions, but at some level a user has to understand how a system works in order to adequately conceptualize the task they want to execute.

This isn't to say that we should expect anyone who wants to get more out of their machine to learn shell scripting or that we can't make things easier for non-programmers, but just like playing an instrument or writing a book, hobbyists should readily expect that they won't be able to do much unless they take the time to learn the system and the tools.

drivebyacct2• 14 years ago

I've been programming since I was in 6th grade and I still want to cause physical harm to myself instead of write a shell script for any shell.

jisaacstone• 14 years ago

I forget everything all the time.

I have found a great solution for this, but it is not an IDE(!)

It is the bpython CLI. (I hear ipython is good too)

When I forget some string method's name, I just type "str." and a dialog box appears with all the options. If I am learning a new module I use "help(<function>)" or "<function>.__doc__"

oh yeah, and there is auto-complete. and syntax highlighting.

a_bonobo• 14 years ago

I do not understand his first reason for understanding programming AT ALL.

He writes: >To have an aid for thinking and expressing ideas. What Steve Jobs called "A bicycle for the mind".

But if he can't express any ideas due to trying to be "casual", how has he reached his goal in any way?

English is not my first language, yet I had to get a pretty firm grip on it in order to being able to express my ideas in English: there is no "casual English-learning", you either learn it or you don't.

twelvechairs• 14 years ago

I'm a little disappointed that none of the posts here actually discuss how these (pretty pertinent) issues might be solved in the future in any way different to how they are at present.

The issue of having to choose a combination of 1000 overlapping libraries and languages, for instance, is one that affects us all, and is actually pretty recent. Whats the best solution for the future? Surely its not just 'google them all'....

Languages and libraries are, after all, supposed to make the leap between the conception of a simple idea and its implementation easy. If they fail to do this something will change.

bfrs• 14 years ago

matlab has a lot of these features, and so did genera for lisp machines. Proprietary development stacks win when it comes to solid documentation.

winkerVSbecks• 14 years ago

It seems like you literally are describing Processing. I'm not sure if you've heard of it: http://processing.org

It is designed for non programmers – artists and designers, so do a lot of things you are asking for.

swaraj• 14 years ago

Try writing python with Pyscripter and see if you can't code something useful in an hr

lsiebert• 14 years ago

It seems like one of the big things he is talking about is good documentation which includes usable example code.

Bash Shell scripting would be a lot simpler to use if each man page for a program or shell function had 10 or so examples with the most common command uses which were easy to paste into an editor.

Course my real preference would be the inverse. Search how to do something, return code in the specified language/shell to do it. Right now that's done by Google and other search engines, though rather poorly. Also google doesn't compare and contrast implentations.

The faster we can find a way to implement our goal, the greater our efficiency. That is something abstraction does of course , but we should remember that documentation also serves programmers.

EdiX• 14 years ago

Where is the casual carpentry? Why can't I make my own tables and chairs? How do I know which tools to buy? Why so many different types of screw heads with no clear indication in the box as to which kind should I use? Tapered shank what? How do rivets even work?! And it's not like it's a new thing either we have had wood manufacture for millennia!

IMHO some things will never get past the hobbist-accessible stage.

ttt_• 14 years ago

I agree. If the OP was talking about mechanics, he could be saying: "I want to build a hotrod from scratch, but oh look so many little parts!".

I think that what many people fail to realize is that although the intended purpose seems simple enough, the tools to do it are sitting atop so many levels of abstractions that without prior knowledge you just have no way to figure where to start assembling the parts. The only way for a hobbist to manage that, would be with the right tools pre-assembled for some finite set of purposes that could guide most of the work.

mohamedsa8• 14 years ago

There are many facets to the problem, some of them are easily solved with the technology we have. We need to learn from the tools people already casually program with: VBA, Matlab, Excel, Turbo pascal in its days...etc.

They may have many defects as languages, but they got a lot of the 'casual' right; we could start from there.

HeyLaughingBoy2• 14 years ago

... or to learn what the abstractions are, why they are used, and break through them to learn the basic steps.

What you describe is a very common problem on electronics forums. Consumer electronics are cheap, sophisticated and ubiquitous, so the newcomer to electronics DIY thinks that a cell phone as a first project should be easy and then gets angry when everyone tells him how hard it is to actually design & build one and that he should start by learning how to make an Arduino blink an LED instead.

When I consider all the problems that had to be solved to make a disposable $20 phone, it's absolutely mindblowing that we could ever have reached this point in technology.

ckpwong• 14 years ago

The question is, do people at large even want to do casual programming?

Stepping back, what does "casual programming" mean anyway? Automate certain repetitive tasks? Do some complicated calculation? Write a two-player tic-tac-toe? Make a Sudoku solver?

For the most part, "casual programming" for people who are not into the abstraction layer have been mostly done in Excel/VBA. Most people just don't realize that IS a valid form of "programming" (albeit a very limited subset of it). It's like people who play Angry Birds/Farmville (or Minesweeper in the '90s) obsessively don't consider themselves as gamers, even though they might have spent more time "gaming" than most Counter Strike "gamers".

On the other hand, if you want to dig more into programming, tools like Lego Mindstorm or Scratch teach the concepts rather beautifully. It makes moving onto an actual general purpose programming language much easier.

Where is the casual programming?

Related Stories