HN user

Monkeyget

636 karma

@tburette

Posts7
Comments133
View on HN

[..]I think tech companies may have something to do with it too. These companies basically create massive book assets (and income streams) without requiring much capital, or being capital constrained.

John Van Reenen talks about this in this paper [1]. He has been looking into how and why the giant companies have been concentrating all power in their market : Amazon, Google,...

His hypothesis is that to become a giant company you used to need capital to buy the best factory and the best machines but that nowadays to become big you need to have the best version of intangible capital : proprietary software, users' data,...

Markets become winner takes all and this creates the rise of "superstar companies". To become a giant company you can't just dump money at the problem anymore to build, say, a new factory. The only way to dominate search is to have Google's tech along with users and the data they generate. To become a retail giant you invest heavily in logistics and inventory control management.

This has several consequences :

- You don't need to borrow mass amount of money to become big.

- Changing the interest rate doesn't have as big as an impact as before (if you don't need large amount of capital to grow big, making money more expensive won't prevent companies to grow big).

- wages inequality rise (the few workers that produce the intangible capital in the have high wage while wage become stagnant for the rest)

[1] Increasing Differences between firms: Market Power and the Macro-Economy http://mitsloan.mit.edu/shared/ods/documents/?DocumentID=504...

I have a Lenovo X1 Yoga which, confusingly, is completely different from the "non X1" Yoga rviewed here. Also not to be confused with the X1 Carbon which is pretty much the same product as the X1 Yoga but belongs to a different line. Thanks for the clarity Lenovo.

Anyway I like the balance the X1 Yoga strikes : it has a regular Thinkpad keyboard which is nice to write code on AND it is a 2-in-1 touchscreen that you can flip into a tablet. The keys recess into the case when you flip it into tablet mode.

How does it compare to SICP? It's hard not to see the similarities : an introductory MIT book about programming written in Lisp.

Several contemporary settlements have been excavated: Nevalı Çori, Jerf el Ahmar, Mureybet. So we know what the houses, communal buildings and granaries looked like. Göbekli doesn't look anything like that. The location also reveals that it wasn't used to live, gather food or trade: It's built on top of a rocky hill without access to food or water. You don't extract 15 tons pillars, carve them and move them atop a hill to make things prettier.

The buildings themselves are the biggest proof. No roof, inconvenient and small entrance, holes to let the soul escape. The carvings found at Göbekli were also found on other sites which were associated with religion and death. One example would be the carving of vultures and headless bodies. Vultures which are associated with death and the ritual of excarnation: the floor of the buildings were made waterproof and sloped to allow the draining of bodily liquids. We know that people in that area would bury bodies without their head or even reopen the grave to remove the head. They also have been known to put plaster on skulls and keep the heads around.

Göbekli Tepe is an amazing site that gets surprisingly little attention given its importance.

It is the oldest building in the world. The only older constructions that we know of is a small wall made from piled rock to protect a cave from the weather. Not only is it the oldest but it also is large : 1000 feet across and at least 50 feet deep with several layers covering at least two millennia. The site was only discovered 20 years ago and only about 20% have been excavated so far!

Cultivation first started in the Gobekli area. We know from DNA analysis that the wheat we use today is based on a wild variety of wheat that grew miles away from Gobekli.

It is located in the middle of the fertile crescent: the birthplace of civilization. It is located between the Tigris and Euphrates near their source which is the location of Eden in the bible. It also is located right next to Şanlıurfa which is said to be the birthplace of Abraham.

The size of the monuments also are impressive given that the people who built them were still hunter-harvesters living in small tribes without the knowledge of the wheel or pottery. It required a large workforce working over a large amount of time.

I am happy that the recent discovery seems to confirm my personal theory that Gobekli is a Dakhma [0], a place where bodies were left in the open for vulture to cleanse.

[0] https://en.wikipedia.org/wiki/Tower_of_Silence

The next step is to realize that you can use streams to communicate between processes. Stream parallelization is used not just to speed things up but as an easier way to write massively parallel programs.

Instead of bug inducing mutexes and semaphores, you write a series of threads who are only able to communicate between themselves through streams. Each thread possesses input streams to receive data from other threads and output streams to talk to other threads. Each thread is idle until it receives an input from one of the streams. It then performs some processing which can include sending messages to output streams before going back to sleep. A system may possess a large number of threads and

I wrote a pacman this way where interactive object was a thread: the pacman, ghosts, bonuses,... Even the score was in its own thread. Since the game was real time there was a special clock process to generate the ticks to advance through each frame of the game.

Except for the initial wiring up and flow control (when the emitter of the stream is faster than the receiver), the system is easy to reason with and debug. By looking at the stream you can get a high level of visualization.

I guess the next step after that is to realize that you can apply this to the entire system and replace messy one-to-one ESB RPC calls with something like Event Sourcing.

Sounds like a textbook case of a capability trap and a failure to recognize that an improvement programs may actually result in a reduction of productivity in the short term.

From 'Nobody Ever Gets Credit for Fixing Problems That Never Happened'[0] that was discussed a few days ago [1] :

In one firm we studied, the general manager laid out his goals for improving the product development process [...]. At the same time, they launched many new development projects in anticipation of the expected productivity gains. Viewed through the lens of management’s mental model these decisions were entirely rational. However, that mental model [...] led them to the erroneous belief that the delay between improvement effort and results was short and that their engineers were under-utilized, undisciplined, unmotivated, and unwilling to adhere to the specified process.

The company spent millions and invested countless person-hours to create a new product development process. The new process [...] also increased monitoring, including a structured stage-gate review process and mandated use of project management software. While there were some pockets of success, in most cases the effort had little impact. The leaders of the change effort often attributed its failure to the engineers’ lack of discipline:

“A lot of the engineers felt that [the new process] was no value-add and that they should have spent all their time doing engineering and not filling out project worksheets. It’s brushed off as bureaucratic.”—Manager A

Yet, when we asked engineers why the effort failed, we got a different story:

“People had to do their normal work as well as [use the new project management system]. There just weren’t enough hours in the day, and the work wasn’t going to wait.”—Engineer B

“The new process is a good one. Someday I’d like to work on a project that actually uses it.”—Engineer E

While managers felt the engineers had little interest in following the process, engineers became increasingly frustrated with leaders they felt had no understanding of what was really required to develop new products. Faced with the double bind of hitting aggressive performance targets and equally aggressive improvement targets, they were forced to cut corners while still appearing to follow the process. As one engineer remarked,

As management discovers the engineers’ shortcuts and workarounds, their view that the engineers can’t be trusted is confirmed, and they are forced to step up their monitoring.

[0] http://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.p...

[1] https://news.ycombinator.com/item?id=8940820

Great project!

I've done some python code optimization recently and I was fairly disappointed in the profiling options; more precisely the profiling visualization options. Standard python provides pstats, which is basic and has an awkward command based GUI. I've tried runsnakerun and snakevis which do provide a nice graphical view but little else.

Easy to use, hierarchical list, clear output. I like this. I never used live profiling before and I can already see where it could be super handy.

Remarks:

-how do I pass arguments to my program. profiling seems to absorb all parameters

-Is it possible to access standard input/output when live profiling?

-Is it possible to reset the profiling statistics when live profiling? Eg.: I profile a webserver, profile a request, reset stats, profile another request,...

-I'd love to see an integration with IPython. Interactive optimization could be great: I profile some code, tweak it, profile again,...

I want to write a simple node application that fetches an http document and write it to the disk.

The only requirement is for the application to print out in plain English if there is one of the following error:

* the document doesn't exist

* there is a connection error during download

* the local file couldn't be opened

* an error occured during writing.

For any other error(out of memory,...) the program can crash.

How do I do that in node? I don't know.

The request package only tells me the callback's first parameter is "An error when applicable (usually from http.ClientRequest object)". The fs package only tells me "Event: 'error': Emitted if there was an error when writing or piping data."

I only picked error reporting as an example so I could showcase the problem of lack of documentation on the most basic APIs (http get and writing file). By the way the Error Handling article [0] on Joyent's website is absolutely wonderful.

Call me old fashioned but how can I use an API that doesn't tell me which function to call, what a function accepts as parameters, what it returns or how it signals error?

[0] https://www.joyent.com/developers/node/design/errors

I take it that by design you mean code quality and that you include error handling in edge cases.

Yes bad things do happen in the real world: connections fail, files are missing, input are invalid, programs have bugs,...

Yes you do have to handle it: validate data, check status code, add guard conditions, report errors, document input, returned codes and thrown exceptions. Here is a nice article on error handling in programs if you are interested (for node.js but most is universal): https://www.joyent.com/developers/node/design/errors.

Yes it makes program larger. That cute 10 liners that throw an exception to the user's face if anything goes wrong will end up larger.

But does it make the code bad? No it does not. It makes the code correct. In business software, edge cases and error handling routinely take more effort than the standard case where everything goes right. It's not a wart, it's a fact of life.

A piece of code is not finished when it appear to work. It is finished when it is written, robust, clean, documented and tested. Cutting corner means that when the bug report inevitably arrives a lot of time and energy will need to be spent. Way more than doing the right thing the first time.

"that's exactly what the really productive, fast paced coders in my company do.". They appear productive and fast paced but they are not. They only appear to be so because they twist the definition of done. By moving the goalpost they throw a bunch of garbage over the wall wishing good luck for some sucker later on. But, hey, it sures look good for management. That's not even adding technical debt, technical dept implies you make a deliberate trade-off. This is just burning money.

8Mb is less than a thousand of a percent of my hard drive. That space is well worth the convenience of not having to worry about installing a language/checking version/compiling/whatever.

It takes a year because you learn it over time. Learning a tool progressively is the best way to go.

There is a wonderful presentation called '7 Habits For Effective Text Editing 2.0' [1] explaining this philosophy. The idea is to get the basics down upfront and then get more proficient gradually over time. When you use the tool you stay aware of what you are doing. When you find an inefficiency in the way you use the tool, find a quicker way and make it an habit. This is the most efficient.

If you only care about the code you have to type right now and never read the editor's documentation nor learn new commands you'll keep using only basics commands and never will gain proficiency. Not efficient.

If on the other hand you try to learn every feature immediately you'll waste time learning things you won't use and forget most of them anyway. Not efficient.

[1] https://www.youtube.com/watch?v=p6K4iIMlouI#t=2085

Another section of MMM directly related to this article would be the chapter "ten Pounds in a Five-Pound Sack".

It talks about allocating memory to the various components of the OS. It begins with a delightfully outdated section on the cost of memory: if the OS weighs 400ko it costs the user $4800 per month when renting just for that memory!

His take-aways on allocating a limited resource (memory here but it could be $) are:

* Budget every aspect, not just the resource itself. Otherwise implementers will end up swapping to disk to respect the limit, killing performances.

* Define what must be achieved along with the budget. Otherwise things will be thrown over the fence to a neighbour.

* Perspective and overall integrity must be watched : "The project was large enough and management communication poor enough to prompt many members of the team to see themselves as contestants making brownie points, rather than as builders making programming products. Each suboptimized his piece to meet his targets; few stopped to think about the total effect on the customer. [...] Fostering a total-system, user-oriented attitude may well be the most important function of the programming manager".