Our approach to subjective programming questions

http://blog.slant.co/post/42487997802/tired-of-being-told-you-aren-t-constructive

by natosaichek • 13 years ago

110 59 13 years ago

blog.slant.co

I think it's great to have a centralized place where you can find answers to subjective questions; however, the current format makes it difficult to digest __why__ something is considered better than its competitors.

It would be useful to have a set list of pros and cons of each entry.

StuieK15• 13 years ago

Hey mate!

So the idea is to have the list of Pros/Cons! For example checkout this one:

http://slant.co/topics/what-is-the-best-search-engine-for-we...

It tells you what its good for, as well as the "cons"

Edit: Also topics like: http://slant.co/topics/what-is-the-best-javascript-drawing-l... make it clear what the best use-case is for the various options. Do you want something a little more like a "comparison matrix"? Thats also in the works to help compare specs/basic features.

RKoutnik• 13 years ago

As the author of the recently closed/opened/locked question about brackets [0] on Programmers.SE thanks so much for creating something like this. I'm really enjoying all the additional detail on why something is/isn't the best. Keep it up!

http://programmers.stackexchange.com/q/188455/54164

StuieK15• 13 years ago

Thanks!

itafroma• 13 years ago

Wow, they closed and locked that? That type of question is pretty much exactly what Programmers.SE was intended for.

But unless I'm not understanding Slant's mission correctly, this type of question wouldn't work there: it's for things like "What programming language should I learn next?" or, a little closer to your closed question, "Should I use curly or straight brackets to denote scope when designing a programming language?"

te2• 13 years ago

Guys, I really like what you are doing, I think there is a market opening for this type of site. Couple comments: (1) You probably ought to get tags implemented ASAP so it's easier to find stuff in broad categories (2) Some ppl might not be too thrilled with your lifting so much content from SO. I realize you're probably just trying to seed some content, but still. Might leave a bad taste for some ppl.

StuieK15• 13 years ago

Thanks! Agreed on the SO point, I actually just tried it out on the "Haskel" topic and maybe a couple others with it all fully cited but stopped as it didn't feel right...We have over 300 topics right now with only those few from SO.

Working on point #1 right now.

mendicant• 13 years ago

I like the idea, but it crashes Opera.

StuieK15• 13 years ago

Really sorry about that :( it's very much in an Alpha state. Will get this fixed ASAP.

T-R4• 13 years ago

Thanks for the feedback. One of the projects that our front-end engineer is currently working on should fix this issue, though it may not be live for a few days. Very sorry for the inconvenience.

martinced• 13 years ago

Yes, yes and yes!

Why limit topics to stuff where there's no objective answer? There are a lot of people (including people with 20K rep+ on SO) who quit SO in disgust and who would just love to share their knowledge on both objective and non-objective question.

Pivot immediately. Do both objective and non-objective : )

Instant winner IMHO, going to help people there immediately!

StuieK15• 13 years ago

Well you just made my day, thanks so much.

On the objective comment, the format actually does work for objective questions just fine without us changing anything. We just excel at subjective questions so thats how we are positioning ourselves.

fishtoaster• 13 years ago

I think people want the ability to do both, but there was a huge need for a better way to answer objective questions. In the pre-SO world, forums and bulletin boards served subjective questions mediocrely, but objective question abysmally.

Solving the objective question problem is and always has been SO's mission. It's a hard problem, and they solved it well- at least partially by firmly avoiding discussion questions.

So, while I think there's totally room and a need for a better solution to discussion questions (and there are several active attempts: discourse, slate, quora, etc), I hardly see it as a fault of SO's that they pushed away subjective questions.

pnathan• 13 years ago

I found SO (over time) to give less and less useful information: the best information was always in the quasi-subjective questions. It's nice to know how to use something, but really, I want to understand why. The SO/SE theory is against those questions. I have 19K rep on SO, 5Kish on P.SE, and I barely contribute anymore.

So I am planning on creating a Slant account and experimenting when I get home.

charlieflowers2• 13 years ago

I feel exactly the same way (not with the high rep you have, but a decent rep that I worked hard to cultivate in the past).

I think there are a good number of us who feel this way.

It is frustrating for a company to say it wants to "discover what its users are doing and enable it", and then to see that company squash so many individual pieces of excellent content those users have created.

danso• 13 years ago

I do like the content so far, but this isn't really a Q&A...it's more of a WikiAnswers, isn't it?

The good thing about StackOverflow is that, when in doubt, you can always use a user's score/comment history to gauge their credibility (though yes, that is an argument by appeal to authority...)

With Slant, the tracing of comments/submissions is as hard as it is on Wikipedia.

And the "the answer must be in article-ish form" has its disadvantages, of course, not least of which is that some of the best answers come through repartee, as they have in StackOverflow.

Also, I'm not sold on the font. It's not great at small sizes and you may want to stick with a sans-serif.

StuieK15• 13 years ago

Hi Mate,

> With Slant, the tracing of comments/submissions is as hard as it is on Wikipedia.

We are building an "authority" SO style karma system, and unlike Wikipedia you can easily see who is a contributor to a post. This is something we agree is really important, but just didn't make the cut for the Alpha product. Please also note that your edits need to be initially approved by the existing contributors, which helps keep quality high and helps you trust the content.

> I do like the content so far, but this isn't really a Q&A...it's more of a WikiAnswers, isn't it?

Well there are questions asked and answers listed.. but you're right in that it's not really Q&A. Someone the other day told me it was "Q&Os" (Question and opinions)

Glide• 13 years ago

This is awesome. I browsed stackoverflow when it first started and just wondered when the first wave of overzealous "editors" would take over the site. It took a while, but those people killed spirit of the site for me.

This looks great.

IMO, a lot of the difficult things are subjective. Good luck.

Falling3• 13 years ago

Looking pretty good. A few things that I noticed:

- "Read More" and "Seem n more viewpoint(s)" go to the topic page when I expected them to just load some additional information.

- "Read More" link is still present on individual topic page even though it goes nowhere.

StuieK15• 13 years ago

Totally agree and we have plans to fix both points :) Thanks for the feedback! Was there anything else you noticed that was confusing?

tobyjsullivan• 13 years ago

At first blush, this looks fantastic and I'm excited for it's potential. Particularly in my experimental side-projects, I'm constantly looking for some guidance from others.

Closed StackOverflow questions infuriate me to no end - not least because that's where the experts are that are eager to help!

StuieK15• 13 years ago

Thanks for the comment and the content you added Toby!

nirvanatikku• 13 years ago

Gotta say, at first glance I really like this approach..

Here are a few things that are going on in the back of my mind.

> I'd love to see a time based filter. For example, consider the evolution of JavaScript libraries. For simplicity's sake, let's take the shift from prototype to jQuery. What you guys are creating could effectively answer this question: 'When did jQuery surpass Prototype as being the premier library, and why?' The filter would have to be periodic (since votes are going to be spread out), but I would find it fascinating to glean the shifts in developer technologies. Think Backbone, ember, angular, etc..

> I see a lot of ULs in place of what seem to be 'features'. While this may only be relevant for things that are capable of being compared (apples-to-apples so to speak), I imagine significant value will be come out of a resulting table of comparison; that is, if each bullet point could somehow be categorized, tagged or curated in some way. This may be hard to achieve (and may not be relevant in some cases), but you could effectively build out a report highlighting things that _are_ and _are not_ available (instead of having contributors write long lists that then users have to parse and reconcile). Does that make sense?

> In the blog post you state "we aren’t actually trying to tell you what the best programming language to learn first is" -- I appreciate that you don't want to suggest the 'best', but I imagine that's how most developers (particularly newcomers) are going to interpret the results. I'm curious if such a forum would influence new developers for the better or worse.

> Sign in with Facebook? What about stack exchange/github? I don't know what you guys think about leveraging reputation, but it would be nice to bring my existing recognition - score and tags (albeit slim) - along with me. Even integrating LinkedIn could be interesting with skills. Just a thought.

StuieK15• 13 years ago

Awesome feedback.

1: That is a really cool idea and something I have been thinking about since day 1. I want to be able to show a graph that displays the relative change in opinion over time, annotated with the addition of new citations to the Viewpoints. So when someone updates the Ember Viewpoint with new information and the citation for it, you can do some basic "cause analysis" on what changed,when and why :)

2: Absolutely makes sense. We are still debating the best way to do this, but one simple solution is an editable comparison matrix that shows the simple features/specs that the Viewpoints have. The ULs would then be used to further flesh out the more significant points that can't be explained in matrix format (for eg how nice the UI is for a particular app)

3: Again, we debate this point all the time. What do you think about the phrasing "What are the different programming languages suitable to learn first?" or something along those lines?

4: Awesome idea, we are going to add additional login options soon, but the idea of bringing in external rep is very interesting!

te2• 13 years ago

I agree that the time-based view will be important, as the answers do change over time. One way to approach is to show votes in last 12 months, instead of all votes ever. Side benefit of this is that it encourages users to stay active and engaged with the site (to maintain and update their views as they evolve).

TeeDub• 13 years ago

Maybe you could present the different options in reverse order of votes? (filtered a bit so that spammy answers aren't always first.) Changing the order of presentation can be a good way to stem cascade effects from seniority/etc. For example, the US supreme court votes in reverse order of seniority to help prevent cascading effects. (That way less senior justices can vote Wo undue influence...)

tgflynn• 13 years ago

Seeing the vote breakdown for alternatives could be one of the best features of this site. Can users vote easily without leaving comments ?

Being able to get a quick overview of community sentiment on what's the best library/tool to use for XXX on/in YYY could be very helpful when one is researching alternatives.

StuieK15• 13 years ago

You can vote without leaving comments, but you have to create an account to do it.

lelandbatey• 13 years ago

I think this is awesome, especially since I asked about this exact thing just hours ago before you posted.[0] I'm really excited and I can't wait to see you take off!

[0]http://news.ycombinator.com/item?id=5286601

StuieK15• 13 years ago

Thank you!

niggler• 13 years ago

Please please please don't pull a quora and force people to have to register to see the vote counts.

6ren• 13 years ago

I think many people, like me, have been waiting for someone to address this obvious niche that has been crying out for attention.

There's also an interesting path to monetization: it's pretty much models the buying process, esp for the enterprise. Also generalises to purchasing decisions, in general - it's relevant to all transactions. i.e. the whole economy.

However... very hard to do well. It's good start, but nailing it will take a lot of time, effort and luck. Starting in this very focussed and narrow niche is a great way to build understanding of what's needed and how to do it.

EDIT I found the site slow to load (actually got the wait or cancel dialog). Because such a site thrives on as many users as possible, it may be more important to make it accessible to more people at the expense of prettiness (I assume the pie-charts are partly to blame for the slowness). It also biases the answers and votes (maybe a good thing - target that niche?); might explain why emacs got few votes.

The presentation of information is too "detail-heavy" (i) the list of questions includes 3 alternatives and even some lines about each. The typical user wants to scan the topics to find one of interest. (though I guess having a some context is good, since that's what SO does; but only to clarify the question, not the answer).

(ii) in the actual answer, there's no summary of the results [EDIT whoops, there is a summary; somehow I went directly to one of alternatives at first]; no comparison of the alternatives; and no indication of what context each tool best suits ("job" for the tool); instead, each page is all about one alternative. I would guess that something like a grid might work better, with the tools listed vertically, and the attributes horizontally (with the "job"/context first). The user can then scroll down to examine each point in detail (or maybe drill-down, with expanding sections, for truly secondary/tertiary content, like comments/debates). Attributes could be voted on.

In short: organise it to suit the user-task, not the subject matter.

StuieK15• 13 years ago

> I found the site slow to load (actually got the wait or cancel dialog).

Sorry about that, we are under some heavy load and didn't have time to do even the most basic speed optimisations (gzip etc). Its very high priority.

> The presentation of information is too "detail-heavy" (i) the list of questions includes 3 alternatives and even some lines about each. The typical user wants to scan the topics to find one of interest. (though I guess having a some context is good, since that's what SO does; but only to clarify the question, not the answer).

Totally agreed. I know how to fix this, will be done soon.

> (ii) in the actual answer, there's no summary of the results; no comparison of the alternatives; and no indication of what context each tool best suits ("job" for the tool); instead, each page is all about one alternative. I would guess that something like a grid might work better, with the tools listed vertically, and the attributes horizontally (with the "job"/context first). The user can then scroll down to examine each point in detail (or maybe drill-down, with expanding sections, for truly secondary/tertiary content, like comments/debates). Attributes could be voted on.

We have this page: http://slant.co/topics/what-s-the-best-jabber-gchat-client-f... that shows ALL the options and an overview of what they are suitable for. To further improve this we are working on a comparison matrix style feature that is approximately what you described here.

Would it be possible for you to email me? (stuart@slant.co) I'd love to be able to show you some 'in the works' features as they are being built for some feedback.

argonaut• 13 years ago

Another thing I particularly like about threads comparing two or more different technologies/frameworks/languages is when someone says "X does Z badly, and here is why Y does Z better." Slant seems to not be able to facilitate this very well.

dinkumthinkum2• 13 years ago

The strange thing is if SO actually decided to cater for this, which would be pretty trivial, they would win this game by default. It's basically just ocd run amok.

adev• 13 years ago

I like this, but I'm having serious issues reading the content in a manner I'd use it.

It's the font choice, size, and line-spacing to start with. It doesn't lend itself to quick skimming, and to succeed in this segment you really need to nail this aspect -- because this is what we all want... quick answers.

StuieK15• 13 years ago

Really good points and I do agree that we need some work on the presentation. Thanks for the feedback.

michaelmior• 13 years ago

Looks like a great start. One thing that strikes me is that it's really hard to see how useful the data is on first glance. I kind of like that the pie chart is front and center, but for me it's useless without showing the total number of votes.

StuieK15• 13 years ago

> useless without showing the total number of votes.

Hover and you can see the % and on the Viewpoints it shows the votes they received but obviously to be even remotely trusted we need total votes. Consider it on the to-do.

xsmasher• 13 years ago

Maybe it's because I am not signed in, but it looks like you are missing a lot of the tools needed to spot sockpuppets / shills, like history of votes / edits for a user, and list or voters for each answer.

If your site gets popular you will be flooded with astroturfers and over-enthusiastic project promoters.

StuieK15• 13 years ago

Hey mate,

We are recording all that information and have put a lot of thought into how to combat those problems. Unfortunately they didn't make it in for the Alpha, but its definitely something we are aware of and are actively working on.

T-R4• 13 years ago

Thanks for the feedback.

Regarding sockpuppet accounts and the like, we do have that information - some of it will get more visible as we build out profiles and moderator tools, it's just been on hold while we focus on more user-facing features, since it hasn't been an issue yet. Account validation e-mails are also on the way.

Spam is definitely a concern, but we hope the fact that most content requires acceptance from another user before getting visibility will put up some barrier to entry, and in the cases where it's not sufficient, it should help us get some information on associated accounts. In cases where that fails, there's always manual moderation.

Regarding astroturfing, we welcome it, to some degree - we don't expect each viewpoint to maintain NPOV, we just want to make sure that each viewpoint gets visibility. Cons/criticisms will eventually broken out into a separate feature to reflect this.

logn• 13 years ago

This is super awesome. You're getting criticism of the font. But I have to say that I really like it. I just created my own issue and like the UI all around.

For karma, I'd recommend that each person gets points in proportion to the number of upvotes combined with the number of characters they added to an answer that appear in the final answer.

Also, I think you need some sort of comparison or overview for each issue. And next to the pie chart I think you should allow for a 140 character final recommendation, listed individually for each user. E.g., "Dan recommends SilkJS because it's easy to write, fast, and better than Node"

T-R4• 13 years ago

Great ideas! Thanks kindly.

anateus• 13 years ago

I like a lot of this approach. But, please replace the pie charts.

http://en.wikipedia.org/wiki/Pie_chart#Use.2C_effectiveness_...

Humans aren't that good at telling more than very rough ratios in angle rather than length. Yes, when the differences are large you could tell at a glance the overall distribution of answers, but when it is close, you lose that. A horizontal stacked bar would achieve a similar visual flair but would be much more intuitively accurate.

jyrkesh• 13 years ago

I really like the idea, and what few bits of criticism I had you have already seemed to address.

I just wanted to compliment you on something specifically that I though was really great: the registration process. Very similar, no confirmation fields, and, most importantly, I was signed in immediately after clicking finish. It didn't take me away from or reload the page I was on, and it didn't ask me to confirm my email. The small things really matter.

charlieflowers2• 13 years ago

Cool! You have "discovered what Stackoverflow users were doing," and where Stackoverflow stomped it out like a plague, you have made an offering for it. Good job!

I'm still amazed at Stackoverflow's failure to follow their own stated philosophy ("discover what your users are doing and enable it") in this particular area.

dinkumthinkum2• 13 years ago

Pretty much for any question you find via Google that you are really interested in, you will find an SO question that fits the bill and has some big long jibber jabber filled message about being closed for such and such reason.

The strange thing to me was that when Jeff left was like when it went full tilt. It's like Mao died and the rest of the party big wigs realized they weren't being the best communists they could be and just started going overboard to stomp any possible conflict with Maoist ideology.

pasbesoin• 13 years ago

One of the most frustrating parts of some early experiences I had with some programming classes, was a lack of "meta". So, we're learning some specific constructs... but what is the fucking context!

This is one reason such topics come up, whether on StackLatest or wherever. The new "team" members want to know where the fuck to start and what it all means.

I'm put in mind of Organic Chemistry I. A great class for "weeding out" before majors are declared. At least as I experienced it, a boatload of often "arbitrary" memorization... A real slog, when you couldn't see the forest for the trees.

Mind you, some significant percentage of "What's best?" posts are indeed people just too clueless or lazy to tackle any element of the question themselves. But... that's what voting and moderation are for.

StackOverflow seemed to be dealing with this pretty well. I don't see what the need for the change (in attitude, if not stated policy) was.

wanderingstan• 13 years ago

I was just thinking of this idea over dinner tonight, when a friend was giving his recommendations on PHP and Ruby frameworks and their pro/cons. "There should be a site for this type of knowledge!" I thought. And --boom!-- I get home, and here it is on HN.

epaga• 13 years ago

It always seemed to me that StackExchange's own http://programmers.stackexchange.com was intended for this very concept. From the FAQ:

What about subjective questions?

Subjective questions are allowed, but subjective does not mean “anything goes”. Please keep it professional at all times. If this is a question you'd be uncomfortable discussing with your colleagues in a work environment, it's probably not appropriate here, either.

For example, here is a question re: CSS pre-processors that seems pretty subjective: http://programmers.stackexchange.com/questions/181536/are-th...

fuzzix• 13 years ago

Until your highly subjective question gets closed:

http://programmers.stackexchange.com/questions/188455/why-do...

TheCoelacanth• 13 years ago

The problem with that question is that there isn't really any answer to it. It basically just ends up being that at some point someone made an arbitrary decision to do something that way and a bunch of other people copied them.

woah• 13 years ago

Beautiful and informative. Great job.

Flenser• 13 years ago

It would be great if you could let users indicate preferences between answers so that you could produce an ordered graph of which answers are better than others. I'm thinking of something similar to the less wrong textbook recommendations[1] with it's rules:

   1. Post the title of your favorite textbook on a given subject.
   2. You must have read at least two other textbooks on that same subject.
   3. You must briefly name the other books you've read on the subject and explain 
      why you think your chosen textbook is superior to them.
   
   Rules #2 and #3 are to protect against recommending a bad book that only seems
   impressive because it's the only book you've read on the subject.

AFTER EDIT: I'm thinking this would be particularly useful if you've narrowed your choice down to a few options and wanted to see comparisons between those options.

[1] http://lesswrong.com/lw/3gu/the_best_textbooks_on_every_subj...

greggman• 13 years ago

I like idea but... the first question that interested me was

"What the best 2D Game Engine" http://slant.co/topics/what-is-the-best-2d-game-engine

And I'm confused by my options on slant. What is a 2D game engine? Can we even discuss that? Example: Phonegap, should I add it? People make games with it. They also make non-games with it. I didn't see a place to bring up this discussion (meta-discussion?)

Next: I've been looking at 2D engines recently so I thought about adding entries but I don't know the engines enough to do anything other than add them to the list. Except apparently I'm not supposed to do that? But even that is useful info (ie, to know what the choices are)

Anyway, not sure if that's useful feedback.

prtamil• 13 years ago

Thanks you guys, i feel like you are the guys invented democracy to save us from SO tyranny.

Small Suggestion: All caps in Title makes the title very harder to read. so instead of WHAT IS BEST EDITOR can you change like What is Best Editor.

bconway• 13 years ago

Great site, I've signed up.

Suggestion: Disambiguate questions where voting on multiple choices is recommended. It makes sense to be able to choose more than one answer on a what-are-your-favorite-... question, but not on a yes/no question.

Our approach to subjective programming questions

Related Stories