HN user

jethroalias97

129 karma
Posts4
Comments68
View on HN

Turning thoughts into software is what software engineers do every day. At every point along the technology trajectory technologists have created more and more powerful abstractions. When transistors where first created: print "Hello World" would have seemed a magical melding of the minds of machine and man, whereas today we can create customized eCommerce social networks with a few keystrokes.

However, as powerful as the tools get, you still have to tell the machine exactly what you want it to do. Depending on the tool, sometimes that is easy, sometimes that is hard. I feel like there should be a fundamental law of "ability to easily express" vs "flexibility of expression." I'm not sure how much machine learning can serve as our savior in this regard, even if we can eliminate "make an Amazon clone" posts on cheap-outsourced-developer.com by just using ML to classify it and spit out the boiler plate.

In order for something to be profitable it has to be customized, and when this happens the abstractions inevitably seem to break down. Based on this I think turning thoughts into software will happen around the same time thinking becomes mass-produced. How can you create, "eBay, but for car buying, with a twitter messaging component built for the Brazilian market" without first beating the Turing test as a sub-problem?

Any thoughts on when the pagination goes live? I can't find any information on it online. Memcache would be a good choice, but I am wondering, if I have a few secondary indexes with over a million indexes each, wouldn't continually recreating this cache irreparably bog down the cluster?

I have been using Riak's secondary indexes for my latest project and have generally found them a joy to use. However, I do have to have to question a bit the way they are architected.

Assuming you are using Riak's default configuration each range query hits 1/3 of the cluster, which could get pretty hairy on large clusters that have lots of requests. Also, there is no pagination, so if an index has a million objects you'll have to be prepared to wait even if you only want the first part of the query.

You could solve this by putting a sort value in the key and using a range query, but this wouldn't work if you want the most recent items keyed with time, because the items could be unevenly spaced back in time. Also, Riak, like many databases based on Dynamo, thrives on fat data which one would think would favor lists. LevelDB is also supposedly slower than Bitcask, the default backend, but I'm not sure if this is still true.

I've been trying to think of ways around these problems. A simple thought I had was to simply cache the response as pages in Riak. Although this introduces new problems like how to know how often to reset the cache, too often and I may as well not have this cache, too infrequently and users get stale data. I would also have to handle this using worker threads because I wouldn't want the odd 100th user to get a big latency hit. The database would also either have to be continually polled, wasting CPU, or potentially not have the data cached when needed.

Another solution I've been considering is to write a secondary index layer on top of Riak using a skiplist or btree to know where to add and remove data when it gets to be very large. This seems like a cool idea, but might be tricky to implement and do conflict resolutions on.

My last idea was the most ambitious, which was to implement a separate distributed database specifically for secondary indexes and range queries which would not be bound by Dynamo. The idea here is to have each node in charge of a segment of the key space (like Big Table) and then have it split and coalesce not only based on size, but also on frequency of reads and writes to handle the bottleneck problem.

I initially was going to have this paired with the Dynamo database (https://github.com/dbunker/Dynago) I was experimenting with using Go and LevelDB, but there is no reason it couldn't work with any Key-Value eventually-consistent hyper-reliable database to provide light-weight secondary indexes. Having it constantly check the core key-value database would mean it wouldn't have to be super reliable in its own right and so could be kept relatively simple.

But again the simplest solution may ultimately be the way to go, I'm not sure, all these seem to have pretty big trade offs.

Assuming you are using a distributed architecture, there is no way to verify a user without at least one database lookup because the request could be coming into any API server. So in most cases we're not avoiding cookies and sessions just for the sake of it.

People always say that remote working will be the way of the future, but humans remain extremely social animals. So much of what we do is guided by genetics and it seems to be human nature that if you don't see someone's face on a regular basis it is very difficult to create an intimate or trusting relationship.

This seems to be one of the main pain points when it comes to outsourcing work abroad, people seem to work best and collaborate the most when they interact frequently in person. In Pixar Story, I recall Jobs saying he designed the headquarters specifically for unplanned collaboration. As much as the world changes, people remain the same.

Introducing Octokit 13 years ago

When I first read 'GitHub-maintained client libraries' I initially thought they meant built in libraries or templated stub functions, sort of like what they offer for .gitignore for various programming environments and languages. I was a little disappointed when I saw it was just for the Github API, although I expect this to still be useful.

I feel like this is something they could offer in the future though as many IDEs automatically build your environment when a new project is initialized. Assuming you use github as your IDE (i.e. not really using any IDE) project initialization might make sense.

I expect people would be reluctant to use a client not sanctioned by bitcoin.org officially, although I'm not sure how difficult that is. It may be as simple as sending an email to get them to add it to the list.

I would think that any new bitcoin wallet could be introducing a substantial security risk before it has been extensively field tested. If someone where to sneak some code in, or even just screw up a security protocol, all the stored bitcoins would be at risk.

1) http://bitcoin.org/en/choose-your-wallet

2) https://en.bitcoin.it/wiki/Wallet

> I see bitcoins as the storage of value, but not method of payment.

If this comes true, the bitcoin system is in trouble. Block reward is down to 25 bitcoins a block. It will half one more time then no more money for mining, it'll all be through transaction fees.

If people are just storing bitcoins and not using them, there wont be enough transaction fees and lots of miners will drop out, making the system more vulnerable to takeover by government or malicious miners.

I feel like what they are describing is more defragmentation than garbage collection. The article uses garbage collection to mean taking sparsely filled blocks and consolidating them into a single block to free the sparsely filled ones. Garbage collection in the OO context is where you free memory the program can no longer interact with. Perhaps it means different things in different contexts?

What you have described is more or less Diffie-Hellman. Unfortunately this alone wont guarantee your package's safe passage. If it did Verisign, would be out of a job.

The flaw is, you can basically just have the same scenario but replace Alice with the post office:

Bob puts the item in a box with a lock and sends to Alice. Post office intercepts the box, places their lock on it and sends it back to Bob. Bob removes his lock and sends the box back to Alice. Now the post office removes their lock and obtains the package contents.

The only way around this is to have some truly trusted third party. Even in RSA, if you aren't absolutely certain of the other user's public key, it won't work, which is why web-of-trust and other techniques are used.

Although much changes with technology, much remains the same. People have had the technological ability to communicate solely through telephone/computer for many years which easily beats face to face communication in terms of efficiency in a geographically diverse group, but people are still shelling out money for plane tickets. There is something to be said for the ability to chat to someone about a ball game without speaking and hearing through a machine.

Although I'd say human to human and human to computer communication are very different, I'd agree with you that some languages are objectively better. There are ancient tribes that don't have words for numbers above two. Esperanto seems to be one of the easier languages to learn, at least for those already fluent in european languages. But when it comes to adoption, culture and ubiquity seem to be the most important factors.

What would be interesting is if there were real online bitcoin banks that leant or invested the bitcoins to both make a profit themselves and offer their customers some interest on their savings.

On the other hand, infrastructure can only be as useful as the applications they support. Technologists love solving hard technical problems [1], but ultimately, isn't the main goal to improve peoples' lives? Almost all the engineers I know would prefer to work on sophisticated interesting problems than "another web-app," but there isn't much point if it is just for our own edification.

I can understand your disdain for Instagram's frivolity, but people said the same about Twitter and it ended up playing a major role in the Arab Spring, whereas almost no one outside the techworld knows or really cares who runs their infrastructure. Of course without this infrastructure it all wouldn't work.

Like you say, comparing the two is like comparing apples to oranges, you can't really argue one is fundamentally more important than the other.

[1] http://jeffhuang.com/best_paper_awards.html

Hashing each character doesn't provide any additional security. The main reason why we hash passwords in the first place is so if an attacker where to get read access, they couldn't deduce the password. But if they are looking in the database and see 15 or so fields of hashes, they can deduce any user's password in at most 256*15=3840 (assuming ascii) tries, which is trivial.

  (People with serious, chronic pre-existing conditions are another matter entirely)
They may be another matter, but does that mean they don't matter?
  I expect that number to be really, really low. 
  Insuring this entire population against that event seems silly.
That is sort of the definition of insurance: Trading a small regular payment in exchange for being protected against very unlikely events. The argument, "it's ok if some people fall through the cracks because it is so unlikely," in the case of insurance doesn't make a lot sense.

From 1980 to today the population has increased by roughly 36% while college enrollment increased by roughly 68%. More people feel they need to go to college to get a job, so when they say, "in 1961, full-time students spent about 40 hours each week in class and studying. By 2003, they were investing about 27 hours a week." That shouldn't indicate as implied by the article that we are becoming lazier, merely there is a larger percentage of the population in the mix.

1) http://nces.ed.gov/fastfacts/display.asp?id=98

2) http://en.wikipedia.org/wiki/Demographics_of_the_United_Stat...

What if you want to update a value, but you want to store the number of times it was updated to be returned in the JSON (not idempotent, but not an unreasonable thing to want)? Would you still use PUT?

My point is not that the ascribed meaning of POST and PUT is bad, I think the concepts behind both these functions are great, but what if you want to make a different or more complicated function? It makes sense to me to invent your own rather than be limited to what HTTP gives you.

I don't really understand the obsession with using HTTP methods. OP brings up a good point that using only GET/PUT/POST/DELETE to only represent CRUD is extremely limiting, but suggests that using even more obscure HTTP methods would be less limiting? I would argue not only is it limiting, it is also obscure, what is it about "PUT" that implies update vs. create?

If you look at the twitter API (https://dev.twitter.com/docs/api) (commonly held up as exemplary REST) you will see exactly 2 methods, GET and POST. GET means no state change other than logging, POST implies state change. This makes so much more sense because then you can use ?command= and have any command you want and not have people confused about what it means.