mattcopp

I've wanted to fake my death in Facebook for some time. A kind of blaze of glory.

It seems remarkably simple, all I needed to do is to get an obituary in a local paper, and fill in this form https://www.facebook.com/help/contact/234739086860192. How hard can that be right?

Unfortunately my wife put a stop to it soon as I told her.

Hi, I'm the co-author of a book by PacktPub.

Throwing some incoherent thoughts together:

I was approach by PacktPub, and I did a bit of Google'ing before I started, to check it wasn't a scam but didn't find a great deal of advice. I really wish I had the good sense to ask on HN, like yourself, before I picked up the contract. I've been meaning to write it up on a blog, when I put mine together (ironically considering the subject matter of the book).

So for me I knew it was never about the money, I was intending at least not to take the advance (which is actually a kind of loan that comes out from your royalties), it was about having my name on a book, but in the end it was so much work I figured I deserved it at the very least.

The main thing that shocked me was how much time it would take to do, in the end 18 months and that was from the half way point that my co-author had got it to, to the point where I gave up with it.

It should be noted I'm not a free-lance developer, I have a full time job so all this work had to be done in my spare time. This was not well understood by the publisher even though I explained it often, but deadlines came and went without any feedback. I was asked to make Skype calls during my working hours, never times that inconvenienced them, just when it inconvenienced myself. I believe they were based in India so calling in my evenings would be 2am or so for them, but I wouldn't be able to write their book at all if I was fired.

I went though 3 (or 4) different 'project leads' who always told me the end was just around the corner. There was no outline as to how much work was involved, the contract stipulated chapters and some re-drafts, but there is WAY more effort in it than that. No one seemed to have a coherent view of what was going on.

In the time I wrote the book and it being published, I got engaged, and married, and the thing that really annoyed me, and the point where I just stopped responding to them, is that I was still being contacted while on my honeymoon after being explicitly asked not to, that and I wasted my time 2 days before my wedding writing the pre-amble that turned out had already been written by my co-author after I was told it was just one last thing (again).

As for the quality of the book? I just don't know. There are errata which get sent to me, but I am done writing books for a very long time.

There is also another project in Rust, it looks more active: https://github.com/GGist/bip-rs

Thanks, I had seen that one, but forgot about it. I think it's a great project, but it's really just a collection of libraries that don't really tell you how it all fits together, which when I was picking stuff up wasn't very helpful. Hopefully now I have a better understanding of the client design I can make something from that.

Doesn't look vague at all. What do you think is missing from it?

For a comparison I would recommend reading a few (what I would consider) good protocol docs. Docs that you could read and implement, and probably get working very quickly, for example:

- XMPP's XEPs (one picked for similarity in usage to BitTorrent) https://xmpp.org/extensions/xep-0020.html - Lots of examples in there for what messages should look like, which is always helpful.

- The BitTorrent RFC doc (linked in the original post) http://jonas.nitro.dk/bittorrent/bittorrent-rfc.html - Sums the situation up nicely with the layout of messages and value lengths.

I think the main thing that makes the biggest difference is adhering to a language spec such as RFC 2119 which recommends using "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", etc. which makes it really clear what you're meant to do or not to.

Specifically for the vagueness of BEP-3, how about this example that made me rage on IRC. In the description for the info_hash field in the Tracker section.

    This value will almost certainly have to be escaped.

What this turned out to mean was that the 20-bit binary sha1 hash MUST be URL encoded, and not hex encoded.

I would love to see someone try to build a BitTorrent client for the first time based solely on this doc.

---

BEP-3 also seems more interested in implementation detail, than describing the protocol. Take the last paragraph (before Copyright) as an example.

Something else which occurred to me today is that BitTorrent is not a spec, it's not been developed, it has evolved. Along with being built in a very modular way, i.e.: DHTs can replace trackers and simply dropped in, magnet URIs can replace Torrent files. This probably contributes it's success and longevity, but what this also means is that there is a lot of stuff, like metainfo, trackers, bencoding, that SHOULD belong in their own spec docs, which form a collective whole.

Great work! Quite annoying actually. I finished my own implementation in Python at about 10pm last night, this would have been most useful. I'm no C# coder, but it's nicely readable, and this is a much better write up than I'm sure I could do.

If anyone who hasn't tried doing this before, the "official" BitTorrent spec docs, namely BEP-3 (http://bittorrent.org/beps/bep_0003.html), seem little more than a vague blog post turned in to a "spec". However, somewhat conversely, this has lead to is a wealth of articles describing how to do it.

The three guides I used were:

- A 2 part blog post which has a bit of a Python bent http://www.kristenwidman.com/blog/33/how-to-write-a-bittorre...

- The unofficial specs https://wiki.theory.org/BitTorrentSpecification, and

- An incomplete Python client https://github.com/JosephSalisbury/python-bittorrent

I didn't know of the RFC mentioned in the post, that would have also been really useful.

A lot of BitTorrent stuff for Python is remarkably hard to find in all the noise of Deluge, the original client, and libtorrent wrappers, but none that existed were sophisticated (or at least well documented) enough for my experiments, they have different focuses.

I never went as far as implementing my own BEncoder library, a billion seem to exist in multiple languages and install any BitTorrent Python library and it seems to come with their own copy. (I suspect due to the way BEncoder was bundled in the original client, see: https://pypi.python.org/pypi/bencode)

I also found a Rust implementation which seems not to compile, but is useful as I'm trying to teach myself Rust https://github.com/kenpratt/rusty_torrent I think the work to get it to compile might be minimal.

That's entirely fair. I realise my use case wasn't the same as your intended goals so I was trying to not be too critical -- also half remembered ;)

You've done some really great stuff in terms of documentation and platform support and it's been designed perfectly for people to start hacking on, which I hope will bring in more interest and in turn bring in faster development.

I went through the code a little while ago when I first saw this on HN.

My thoughts:

It really doesn't scale. It is designed for running on a single instance of physical hardware, RPi. In it's current guise it is really difficult to separate out, particularly, the TTS part which is tied to outputting to a speaker.

There is also no way to process different parts on different machines, wrap up the answer and post it back to a client device, for example a phone.

The detection of command is really just regular expressions, simple enough for a lot of people to work with but is obviously inflexible, the priority setting is useful as long as you have everything in the right order (imagine ordering 20 or more modules).

What is nice is that processing component could be very easily replaced with one that does some NLP, and fortunately Python (which it is writen in) does have some excellent libraries for that (NLTK). This means you could more naturally detect 'weather <city>' or 'forecast <country>' with some entity matching.

The other nice bit is they have done a considerable amount of work abstracting STT and TTS libraries. By itself this would be pretty useful.

Inline documentation is excellent.