HN user

dpkp

55 karma

http://github.com/dpkp

Posts0
Comments19
View on HN
No posts found.

I can understand your frustration about Rich's development process. But clojure is most definitely not closed source. The source is right here: https://github.com/clojure/clojure and the license that allows you to copy, modify, and redistribute that source is here: https://opensource.org/licenses/eclipse-1.0.php

Rich has a fairly strict development approach and wants to personally review and approve all changes to the core. There are complaints about that process, and that's fair. But as far as I have seen, most large, successful projects have similar personalities leading them (Stallman, Linus, Larry Wall, Guido...).

Finally, I should add -- if what you are looking for is software freedom... then you should absolutely consider using a Lisp like clojure. Lisp's give you the power to control your language through macros and non-core libraries. Unlike other languages, you do not need a core development team to make language changes for you. Perhaps this is why clojure is so powerful... because the core process issues you have heard about are not actually that important, and in fact the language itself enables substantially more software freedom than perhaps you are giving it credit for.

have you written your own 2/3 compatibility layer? I can't imagine writing anything large without six...

fwiw, six can easily be vendored into a project to avoid the technical external dependency. that is how we manage it for kafka-python.

I guess I just have never found support for that in the nakamoto paper. You obviously have strong feelings about it, and I'm not trying to provoke an argument. I just still don't understand where in the paper there is any discussion of non-mining nodes. The only place I can find mention of nodes that are not working on constructing new blocks on the network is in Section 8 re: the simplified verification protocol.

That seems inconsistent with the paper. For example, from the conclusion:

  [Nodes] vote with their CPU power, expressing their acceptance of
  valid blocks by working on extending them and rejecting invalid blocks by refusing to work on
  them. Any needed rules and incentives can be enforced with this consensus mechanism.
I do see the separate section on "Simplified Payment Verification", which does seem distinct from mining. Is that what you are referring to?
Why Clojure? (2010) 10 years ago

In FP you will end with garnishFingerFood and garnishCocktail because you need to encode somewhere a specifics of garnish action. In OOP you will have garnish methods on Coctail and FingerFood and specifics and related knowledge how you need to perform garnish will be on object itself.

You're complecting. In the real world of Clojure, you could simply define a protocol and provide different implementations of "garnish".

No apology required. Though note that pykafka requires >=0.8.2 , and is only forwards compatible w/ newer brokers. This means that pykafka implements the 0.8.2 feature set. Newer brokers support that feature set, but you are not taking advantage of 0.9 or 0.10 features if you connect to them. kafka-python, on the other hand is both forwards and backwards compatible. It supports all feature sets: from no offsets in 0.8, to zk offests in 0.8.1, to kafka offsets in 0.8.2, to group management in 0.9, to message timestamps and relative-offset compressed messages in 0.10. The feature set to use is chosen based on the broker version we're connected to. As far as I know, no other client supports this approach -- not python, not java, etc. [Though KIP-35 should open this up to other clients for backwards compatibility starting at 0.10]

I enjoyed your blog post, but I don't think this is a fair characterization: kafka-python is not "mostly 0.9+ focused..." kafka-python is the only driver that is both forward and backwards compatible w/ kafka 0.8 through 0.10. As I'm sure you remember, kafka-python was the original 0.8 driver, written to support the 0.8 protocol b/c Samsa (pykafka's previous incarnation) was only supporting 0.7 and did not have any plans to upgrade.

kafka-python maintainer here. Our library is designed to be correct first, easy to use second, and fast third. It should not be surprising to anyone that using C extensions improves python performance. I have avoided requiring C compilation in kafka-python primarily because I've found that very few python users care about processing >10K messages per second per core (remember in python w/o C extensions you are generally bound to a single CPU, so spinning up multiple processes usually improves performance. see multiprocessing). I've also found the python infrastructure for distributing C extensions to be not easy (see goal #2 above). But that is changing! I would definitely consider leveraging C extensions for wire protocol decoding given the recent improvements to wheel distribution on linux. I'm not sure whether I would go so far as to delegate the entire client to a C extension. Part of the fun of python is that you can play with all of the guts at runtime. I've found users are very willing to hack up kafka-python internals to help debug issues. I dont think I could expect the same community involvement if it was all distributed as a complied C extension. But I could be wrong.

Anyways, always fun to read benchmarks. I hope kafka-python makes someone out there smile. That's the best benchmark in my book.

Labels deal in sound recordings, which have historically been licensed independently of the underlying music compositions. Most distributors like Spotify will obtain separate licensing for compositions, whether from collecting societies like ASCAP, BMI, or SESAC, and/or through direct deals with artist / publishers themselves.

This setup may seem odd, but but it has historically been used to provide some protection against labels "screwing" artists - b/c although a label takes the sound recording rights and therefore controls royalty payments back to artists for the recording [if any -- after recoupment, etc], the artist typically retains their own publishing rights and can license those separately for $$ not subject to their label deal (though subject to cuts from societies, and their individual deals with publishers etc). This opens up new revenue streams for artists to get paid outside the control of their label: by radio stations/bars/restaurants via PROs, or in sync-right deals for placement in movies or on TV.