When is visual programming too visual?

http://removingalldoubt.com/programming/2015/07/30/a-touchy-subject/

by clouddrover • 11 years ago

42 28 11 years ago

seanwilson3• 11 years ago

It hurts when it tries to visualize something better left in code.

If you try to make a visual language as flexible as a general purpose programming language, it's going to be as complex and difficult to learn as any other general purpose programming language in my opinion. I think there's a fundamental limit to how simple you can make programming. Domain specific languages is a different beast however.

CyberDildonics• 11 years ago

Just because no one had done it doesn't mean it can't be done. A lot of difficult in programming stems from how opaque it is. It is like doing a rubiks cube in a black box, taking a look inside, then putting the lid on and starting again.

seanwilson3• 11 years ago

I just think it's unlikely outside of domain specific problems. For example, flow charts and state machine diagrams can elegantly describe some algorithms but they have limited power which limits the complexity of the diagrams. If you want a visual language with programming features like function calls, state, threads, recursion etc. it's going to end up being of comparable complexity to just writing it in code in my opinion. Even visualising how a relative simple algorithm like quicksort works in an intuitive way is challenging let alone a fully working application. Happy to see examples that go against this though.

TuringTest3• 11 years ago

If you try to make a visual language as flexible as a general purpose programming language

Visual languages don't need to fully replace a general purpose language to be useful; a good combination is where you use text to represent the abstractions in your program, and visual tools to represent the values.

You can see this at work in one of the most successful visual tools, the interactive debugger, now embedded in every IDE worth its salt. The program's architecture is written in pure text, but the execution traces can be filtered, watched and poked interactively, rather than the textual alternative of writing everything to a giant linear log file.

Visual environments are best at the initial exploratory phase of software design, when requirements are not fully specified and the code structure is still fluid.

What all visual languages so far are missing is a way to build (textual) abstractions from the initial concrete, visual design. This would allow the programmer to grow the program "organically" by working with the data, without the need to write down every abstraction from scratch in text. There are Programming By Example[1] techniques that could help with this, but they are not well known nor widespread.

I have hope that some breakthrough in visual programming is still possible by creating programming languages that don't try to copy imperative programming, but instead exploiting the strengths of a visual environment. Tools like Apparatus[2] show that this approach is a viable one.

[1] https://en.wikipedia.org/wiki/Programming_by_example

[2] http://aprt.us/

adrtessier• 11 years ago

The MSFT Visual .NET tools are probably the most visual you can get and still be productive.

One of the things I really like about beginners having interface builders is that they can bridge the context between the code and the user interface in a way that is more meaningful than you get out of even web applications. To get started building interactive prototypes, you can draw first and flesh it out later, vs. learn how to draw in markup/code and then learn how to glue it together.

Some of my earliest memories of finding real accomplishment in programming were using Visual C++ and Visual Basic. While I had written a ton of absolute crap in BASIC and C before, the toy projects were relegated mostly to scripts and whatever text or ANSI interfaces I could dream up. I felt like I had made something accessible to other people. Also, my UI design skills got 1000 times better when I had an easy, accessible way to approach user-oriented design simultaneously with the behavior, and it helped me better learn how to divide my code up in ways that made sense.

analog31• 11 years ago

Oddly enough, Visual Basic (and before that, HyperCard) got me over the hurdle of understanding OOP, because I got to use objects before I had to create them.

frozenport• 11 years ago

GameMaker is my favorite visual programming language: You could insert custom blocks of Delphi but for the most part it was blocks of objects.

http://dist.alternativeto.net/s/bbb84d61-31f2-de11-8cae-0022...

Hytosys• 11 years ago

I loved GameMaker so much as a kid. I was sad to see it get taken over by money. :( Are there any modern free & open source alternatives?

Correction: I remember the underlying scripting language being GML (GameMaker Language). Maybe Delphi was embeddable into early versions of GameMaker? I also know that you could link plugins as DLLs...

Agentlien3• 11 years ago

When I was a kid, I spent some time playing around with Klik & Play. https://en.wikipedia.org/wiki/Klik

It was a lot of fun and I found it impressive how easily you could throw things together. It was a nice change of pace from learning C++, where I had not yet gotten into graphical libraries.

On a related note, I also experimented a bit with RPG Maker, which allowed for some simple scripting and made it very easy to quickly create environments using sprites. https://en.wikipedia.org/wiki/RPG_Maker

overgard• 11 years ago

Visual programming mostly makes sense when your model implies some sort of flow of information. For instance, there are a lot of visual programming languages for materials/shaders in graphics visualization. In that context it totally makes sense, because it's about transforming a pixel buffer from one state to others via a lot of operations.

I think where it really breaks down is when you have loops. And loops within loops. Text is much more friendly for that.

TuringTest3• 11 years ago

I've programmed professionally in a visual flow language which supported loops, and you're dead-on about nested loops being the hardest part. My understanding is that the problem is caused by it being hard to think about the actual values on which each component is working.

I have this idea that this problem could be solved with a visual flow language where, at runtime, you could see the collection of all values that each node is emitting during program execution, rather than just the current value. I have not seen that approach in any flow language, so maybe I'll implement this idea in the future as an experiment.

InclinedPlane• 11 years ago

The ultimate dream of visual programming cannot work, at least not at present because we lack the models to represent software adequately using visual schematics.

Consider electronics. One engineer can create an electronics schematic and hand it to someone else who will build it and the second engineer will build what the first engineer specified. Right now, no such thing like that exists for software that isn't effectively a 1:1 mapping to source code. Flow diagrams and UML ain't it. And there's so much ambiguity in higher level diagrams (here's some layers, here's some lines, here's a thing that looks like a cloud) to make them only barely useful in describing software.

There is no standardized nor de facto schematic visual language of software that enables one human software engineer to specify the design (not even the full implementation) of a software system sufficiently enough to be able to hand that off to other engineers to build and be successful and predictable at doing so consistently. And if we can't get a system like that to work with humans in the loop we really aren't going to do much better purely digitally. You can program visually, but so often you end up spending more effort on minutiae than you would if you were coding textually, and so often you run into a very low ceiling where complexity becomes unmanageable.

P.S. Using interactive visual tools for things like UI design, that can be and always has been very beneficial, but that's a far cry from "visual programming".

pjmlp2• 11 years ago

When I look at functional programming, I see the building blocks of digital electronics.

Maybe that is the path we should take.

Create IC modules for apply, map, filter, fold, ... instead of adder, half-adder and so on.

I bet lots of lambda calculus programs could easily map to such constructs.

krisdol• 11 years ago

I think Unreal Engine 4's Blueprints are a bit "too visual". They're difficult to manage at scale and, if anything, make understanding the "big picture" harder than just writing out code.

Agentlien3• 11 years ago

I've been working a lot with something similar to Blueprints. We see it primarily as a way to allow non-coders to build a lot of the game logic. As programmers, we make reusable entities which are then connected to each other in a visual environment. Working in this environment requires little understanding of technical matters.

It's certainly not perfect. In practice, I quickly found that it adds a new level of obfuscation, due to the physical constraints of the 2D-space in which it is being visualized. In short, it easily becomes a mess of tangled connections and sometimes you may have difficulty physically locating the block you want to interact with.

Still, there's not a doubt in my mind that this is a great tool. Not a great tool for a team of programmers, but a great tool for a game studio as a whole. It allows designers and artists with limited technical knowledge to prototype game modes and develop user interfaces, without involving any programmers. Sure, they may need some help from software engineers who create new entities or guide them towards a cleaner design. The end result is still impressive: non-coders are building large parts of the game logic in a very intuitive way which requires no knowledge of code of any kind.

In essence, I see this type of language not as an alternative to writing code, but as an amazing tool allowing non-coders to to develop large portions of a program's functionality. Naturally, my own experience is essentially limited to the niche of game development. I don't know how useful this would be in developing, say, a website centered around an online shopping portal.

greggman• 11 years ago

My experience is that stuff like blueprints is great for the smallest stuff only. Connect 3 switches to and "and" object. Connect "and" object to door's "open" input. If all 3 switches are pressed door opens.

But scripting anything more complex and especially user interfaces is awful. The designers will back themselves into all kinds of corners and there will be all kinds of edge cases not handled and subtle bugs and tracking them down is very very hard.

There was one company started by x-Insomniac people IIRC that decided there should be no designers scripting. All scripting should be done by programmers. Their experience was that trying to make programming easier for non-programmers sounds great, and it is for introducing people to programming concepts and enabling them to make things, but when it comes to shipping code it's not great and too much time is spent dealing with the mess of spaghetti they make.

Note I'm not harshly judging designers here. Great designers make great design. I'm only saying that experienced programmers are more likely to write efficient, maintainable, bug free code than non-programmers.

weitzj• 11 years ago

I once had to do a project in LabView, where you either can go down the visual path or the C/C++ path. So you had to draw a switch statement or a while loop and attach a cable for the `while` condition. To get a feeling how this looks like: http://etchingpathways.blogspot.de/2012/06/labview-event-str...

This was way too visual for me, once you got accustomed to writing code, checking it into your VCS.

TheOtherHobbes• 11 years ago

Max/MSP is increasingly popular in music:

https://cycling74.com/products/max/

It looks like something people coming from a modular synthesizer background should be able to get.

In fact it makes a lot of simple tasks hard because something trivial like a loop takes an unexpectedly distracting amount of pointing, clicking, typing, dragging, and joining. The DSP and openGL parts aren't too bad, but hacking together non-trivial event and data processing logic can be torturous.

This is the big problem with dataflow - it makes you keep switching mental modes from visual object recognition to spatial understanding to text parsing/editing to causal modelling to UI design.

Text has its own issues, but it lets you stay in one mode.

I think VB and Delphi had a lot of success because they hit a sweet spot between the two. They let you solve visual and UI problems visually, and logic problems with code.

Webspace still doesn't have an exact equivalent. Flash came close, but the animation stage added an extra layer of complication. Most web design tools don't do a good job of simplifying and abstracting the relationship between onscreen elements and code the way VB and Delphi did.

brenschluss• 11 years ago

In fact it makes a lot of simple tasks hard because something trivial like a loop takes an unexpectedly distracting amount of pointing, clicking, typing, dragging, and joining.

That's because you don't create loops or if-statements in graphical programming languages, in the same way that you don't create loops or conditionals in functional programming; you use map, filter, etc.

JohnDoe365• 11 years ago

When maintaining the visual representation becomes more overhead than the textual representation.

osullivj• 11 years ago

AVS/Express [1] had a great visual programming system 20 years ago for building 3D visualisation systems. They called it the Network Editor [2]. It was very similar to the Unreal Engine Blueprint system. I've also played with MIT's Scratch environment, which is very visual too, but a little more like a traditional imperative language that the data flow connection style of AVS/Express or UE4 Blueprints. So I strongly agree with the author's contention that "there are some mistakes that are so appealing they will be made over and over again". Appealing because visual programming environments hold out the promise of enabling non developers to develop software. They do a good job of hiding the syntactic complexity of traditional programming, but they can't hide all the semantic complexity of objects, memory, stacks, variables, processes, threads, databases etc. But there is one functional, grid based visual programming environment that has been hugely successful in enabling end users to create their own software solutions: Microsoft Excel.

[1] http://www.avs.com/solutions/express/ [2] https://cdsweb.cern.ch/record/400582/files/p69.pdf

sklogic• 11 years ago

It is very confusing that two totally different things are called "visual programming". The first one does not really deserve it, it is just a visual UI design, while the programming part is nowhere near anything visual.

The real visual programming is not quite there yet, and may never deliver at all: it's the stuff like LabView, Scratch, DRAKON, etc.

Tha said Delphi managed to sit somewhere in between, a little bit, which makes it an interesting case.

pjmlp2• 11 years ago

A Delphi blog with history insights!

Lots of great info, thanks for posting it.

strongai• 11 years ago

We've been obsessed with this for some time. In the 1990s, Prograph CP/X was ahead of the pack in this space: https://en.wikipedia.org/wiki/Prograph

I haven't tried it, but I was pleased to see that there's at least one current product that supports the underlying dataflow language: http://www.andescotia.com/products/marten/

ClayFerguson• 11 years ago

In short, the 'acutal' programming is NEVER visual. The best way to express commands to a machine is in a language by some syntactical text. Pictures are for children. Unless you are studying something inherently graphical, like trees, graphs, charts, you just need code to be as 'efficient' as possible. Usually a chart that does "If(X) then Y", takes just as much effort as typing that by hand.

When is visual programming too visual?

Related Stories