HN user

jstedfast

1,091 karma

Software Engineer working on Mono and Moonlight, but also contributing to a number of other Free Software projects in my spare time (probably most famous for my MIME parser library, GMime).

Posts22
Comments85
View on HN
jeffreystedfast.blogspot.com 11y ago

Code Review: Microsoft's System.Net.Mail Implementation

jstedfast
1pts0
tirania.org 11y ago

Mono for Unreal Engine

jstedfast
221pts90
msopentech.com 12y ago

.NET team makes Portable Class Library (PCL) available on all platforms

jstedfast
2pts0
jeffreystedfast.blogspot.com 12y ago

Optimization Tips & Tricks used by MimeKit: Part 1

jstedfast
1pts0
jeffreystedfast.blogspot.com 12y ago

Introducing MimeKit: A robust open source MIME and mbox parser

jstedfast
1pts0
blog.xamarin.com 12y ago

Porting existing .NET apps to iOS

jstedfast
1pts0
xamar.in 13y ago

A quick look at Dropbox’s new Datastore API

jstedfast
50pts7
xamarin.com 13y ago

Xamarin Evolve 2013 Keynote

jstedfast
1pts0
jdnash.com 13y ago

SOPA is not dead — now it is an Executive Order

jstedfast
22pts3
tirania.org 14y ago

CXXI: Bridging the C++ and C# worlds.

jstedfast
47pts18
toshokelectric.com 14y ago

Introducing CoffeeKit - a JavaScript binding for iOS

jstedfast
8pts0
www.datamation.com 14y ago

Xamarin Brings Mono to IceCream Sandwich

jstedfast
3pts2
www.zdnet.co.uk 14y ago

Mono: A cure for Microsoft monotheism

jstedfast
28pts30
nat.org 15y ago

Instant Company

jstedfast
255pts34
nat.org 15y ago

Nat Friedman joins Xamarin as CEO

jstedfast
2pts0
tirania.org 15y ago

Mono Developers start their own company

jstedfast
325pts80
www.techdirt.com 15y ago

Why Red Hat Is Wrong That It's Better To Just Pay Patent Trolls Sometimes

jstedfast
5pts0
altdevblogaday.org 15y ago

Chaining Compute Shaders

jstedfast
2pts0
altdevblogaday.org 15y ago

Ready, Set, Allocate (Part 2) by Paul Laska

jstedfast
2pts0
altdevblogaday.org 15y ago

Survey of Fast Compression Algorithms (Part 1) by Richard Sim

jstedfast
15pts0
www.computerworld.com 15y ago

Novell's Moonlight shines on Google Android

jstedfast
1pts0
jeffreystedfast.blogspot.com 15y ago

Moonlight on Android

jstedfast
18pts22

You don't need to use an IDE in order to write Xamarin.iOS or Android apps. You can just use a text editor (although Xamarin Studio is free).

The MSBuild logic, at least for Xamarin.Mac and Xamarin.iOS has been made completely open source, so all the logic to build your apps + the SDKs are all open source.

It's also because Apple would be considered a distributor of the final application, thus according to the rules of the GPL, they'd be forced to provide the source code for any app that used GPL code.

Since they don't have access to the source code, they can't fulfill the legal obligation outlined in the GPL license.

Then why did you say you couldn't if "yea, no shit" you knew you could?

If you've already grabbed ENVELOPE info for all of the older messages (i.e. all of the messages from a previous session), you can just download the ENVELOPE info of new messages and then sort locally.

I mean, if only a few messages have arrived since the previous session, it's probably faster to grab the ENVELOPEs and then sort locally.

If there's a lot of new mail, it might be more efficient to sort server side and just fetch the ENVELOPEs for the messages in view and lazy-load the rest when the user is idle (which is most of the time).

A dead-simple heuristic would be if the number of new messages is fewer than the number of messages you can show in a list, then it's probably not worth doing server-side sorting, right? Does that make sense? Especially as mobile devices get more and more powerful. You just need to design your sqlite database such that it's setup to do optimal sorting by whatever sorting method you are using (by date should be easy, right?).

I've actually done this sort of thing before (although not with IMAP), so I know it's doable. It just takes thought to design it and time to implement it.

I've studied the IMAP specs in my spare time over the past year or so and know them pretty well and I'm confident I could get good results if I had the time to implement it, but it's not like anyone in the world could implement a solid IMAP client in a week or even a month working on it full time and seeing as how I couldn't spend full-time working on writing a mail client due to the fact that I have a job that doesn't involve me writing said mail client, I don't see how you could possibly expect me to implement one in a month.

I can show you, right now, a 500,000 message view across multiple folders infinite scrolling with JMAP and I can jump to the middle of that view and load a screenful of messages. You CAN NOT do an IMAP sort without fetching one record for every message in the folder. If you have a million messages in a folder (yes, we have customers who do that) then you need to fetch a million records.

If I've got the sort order (from a UID SORT REVERSE DATE command), and a user jumps to the middle of the list, I can FETCH just the visible subset of messages by calculating what the UIDs would be and then doing:

TAG UID FETCH <visible-uid-set> (ENVELOPE FLAGS ...)

I'm not sure why you think you'd have to issue a FETCH command for each message individually.

It'll take me longer than a month to implement my own IMAP client to do things efficiently.

As it just so happens, the company I work for is having a "work on anything you want as long as it dog-foods our products" week next week and I was already planning on writing an IMAP mail client for iOS that didn't suck. Unfortunately, due to the complexity of writing a mail client, I doubt I'll finish in a week.

That said, thinking about the problem a bit more since last night, there's 1 major performance bottleneck in the IMAP protocol for this particular use-case but it could be solved with an extension to SORT.

So, as I'm sure you know, the SORT command (even with ESORT) only allows returning MIN, MAX, MODSEQ, and COUNT (from memory there might be 1 more, but I forget what it is) in addition to the matching UIDs/indexes. It's not that these aren't useful, but to optimize the "open a mailbox for the first time and display a message list to the user in the fastest way possible", they aren't all that helpful. Instead, what is needed is something more akin to the following (made up) command:

TAG001 UID SORT RETURN (OFFSET 0 LIMIT 50 FETCH (ENVELOPE FLAGS ...)) UTF-8 REVERSE DATE ALL

And this command could return untagged FETCH responses with the requested info - since we'd need a way to determine order, we could say that the FETCH results should be in the order requested or we could also have it return an untagged ESEARCH response like extended SORT commands do now to define the UID ordering.

In other words, make it so that SORT (and SEARCH, for that matter) return the information that we ultimately care about rather than the list of UIDs which we then would have to use to fetch the information we want.

(in hindsight, this is probably what you meant by "chatty" whereas I interpreted it to mean "verbosity" last night)

Basically, make it a bit more SQL-like.

Problem solved.

* designed a lot more around server side sort/search, because UID ordering sucks in various ways - try moving some messages into your INBOX and noticing that iOS mail displays them in APPEND order rather than date order.

Why not just sort the messages? Just because iOS Mail shows the message in APPEND order doesn't you have to show them in that order. You would expect to see them in APPEND order if you are listing them in APPEND order.

As I'm sure you know, IMAP also has server-side SORT extensions. I will agree that they could be expanded a bit, but you can do client-side sorting just fine (and in fact need to if you want offline support). JMAP doesn't fix that.

SORT is really only needed on the server side for calculating which subset of messages to FETCH summary information for (to display in your message-list) if you are incrementally displaying messages as the user scrolls and you have a sort-order applied that isn't "APPEND order".

* connectionless / stateless as far as possible, because connections drop in the IMAP world, particularly in mobile, and the cost of a new SELECT is high, even with QRESYNC. It still has to create a COW view of the message space because it has to keep message numbers that don't track EXPUNGEs just in case the client wants to run a pipelined command.

It's really not that bad.

I already mentioned the EXPUNGES being tied to sequence id's (aka indexes) as being a wart, but IIRC a newer extension replaces EXPUNGE events with VANISHED (I think it was CONDSTORE or QRESYNC) which uses UIDs and vastly reduces the chattyness of a large number of messages being expunged because it gives has a uid-set argument rather than getting 1 EXPUNGE event per message.

You argue that a stateful protocol is bad because it forces the client to re-sync with the server by re-SELECTing the folder and that it is slow even with QRESYNC. Okay, so.... what? A stateless protocol means that new messages can't arrive if I get momentarily disconnected? And no other client can change flags on messages while I was momentarily disconnected?

That's ridiculous. You need to re-sync even with a stateless client to make sure your local mirror is identical.

Sure, with a stateless protocol you don't need to do that, your client could blissfully ignore the fact that another client could have changed something, but then again, you could do the same thing with IMAP - nothing forces you to have to do anything more than re-SELECT a folder and just blindly assume everything is identical to where you left it when you got disconnected. It's foolish to do so, but it's foolish to do so even with a stateless protocol...

Go watch the video at http://jmap.io (recorded in Australia against a server in New York) and tell me you could do that via IMAP.

Okay, I watched the video. The IMAP client on the right was fetching far more information than it needed to which is why it was so much slower. You are comparing apples to oranges.

What do I mean by that? Well, it was pretty clear that the IMAP client was fetching information for far more messages than just the 5 or 6 that it was showing on the screen which is completely unnecessary. The "snippet" portion is slow, yes, because unfortunately right now, the client must wait for the initial FETCH request to return the BODYSTRUCTURE of all of the messages so that it can figure out which MIME part contains the message text and then issue FETCH requests for each of the messages one at a time (which is why you see each snippet load individually).

There is a current discussion on solving this problem with a SNIPPET extension for IMAP so that the client could request it in its initial FETCH request to get the summary info for the messages it will be displaying on the screen.

A lot of IMAP clients are extremely inefficient in the requests that they make and the fact that very few make use of PIPELINE'd commands.

JMAP is a batch protocol

IMAP is also a batch protocol.

Let me ask you this: you say that IMAP is really chatty, and I will agree that some parts of it are, but fetching summary information needed to populate a message list is really not that bad.

Let's take a look at a FETCH response for a single message:

* 1 FETCH (UID 1 ENVELOPE ("Fri, 17 Jul 2015 10:49:03 -0400" "This is the subject" (("Example From" NIL "from" "example.com")) (("Example Sender" NIL "sender" "example.com")) (("Example Reply-To" NIL "reply-to" "example.com")) (("Example To" NIL "to" "example.com")) (("Example Cc" NIL "cc" "example.com")) (("Example Bcc" NIL "bcc" "example.com")) "<in-reply-to@example.com>" "<message-id@example.com>") FLAGS (\Seen \Draft \Answered \Deleted))

Now let's take a look at what this would look like in JMAP:

{ messageId: "f123u457", date: "2015-07-17T10:49:03Z", subject: "This is the subject", from: [{name: "Example From", email: "from@example.com"}], sender: [{name: "Example Sender", email: "sender@example.com"}], replyTo: [{name: "Example Reply-To", email: "reply-to@example.com"}], to: [{name: "Example To", email: "to@example.com"}], cc: [{name: "Example Cc", email: "cc@example.com"}], bcc: [{name: "Example Bcc", email: "bcc@example.com"}], inReplyTo: "<in-reply-to@example.com">, isUnseen: false, isDraft: true, isAnswered: true, isDeleted: true }

This is longer than the IMAP response!

Do you see why I called bullshit on your video comparison? It was obvious that your example IMAP client is purposely inefficient to make JMAP look good.

But let's ignore that for a moment... let's pretend JMAP is all that and a bag of chips.

What now? Since IMAP with all of the latest extensions to optimize/fix all of the problems with the original protocol have been around for years and yet many servers still do not implement them (if they did, there wouldn't really be a need for JMAP), what makes you think mail hosts will suddenly jump at the chance to implement a whole new protocol when they couldn't be bothered to implement existing extensions that would have drastically improved the user experience for their customers?

An HTTP-based IMAP replacement makes me want to put a bullet in my brain.

IMAP is actually really good for the most part. It just has a few legacy warts (like EXPUNGE notifications being index based instead of UID-based), but a lot of the newer extensions fix these problems.

The CONDSTORE and QRESYNC extensions drastically simplify re-synchronizing the client and server and have been around for quite a few years at this point.

Honestly, if I was to design a brand new protocol for mail, it would be very close to what IMAP is today and just make a number of the newer extensions mandatory (UIDPLUS, LITERAL+ with Google's caveat for APPEND, NAMESPACE, SPECIAL-USE, SASL-IR, CONDSTORE, QRESYNC, perhaps a few others). I'd also include IDLE, but not as a distinct command - rather just have it always-on (at least as long as a folder is selected).

The #1 issue most developers have with IMAP is that every server implements a different subset of extensions. This isn't an IMAP-specific problem, however, it's just what happens to any protocol that gets as much adoption as IMAP and lives as long as IMAP has lived. There will always be new extensions for any protocol to improve upon it in various ways just like no framework API is ever complete.

Forcing all mail clients to go through HTTP for mail access is just retarded. It's depressing that so many people love the idea of using HTTP for everything. Honestly, wtf?

Agreed. I had to go back and re-read that sentence when I was reading the article. I was sure I must have misread it. Bush is about as suave as heavy grit sandpaper.

Our IDE is free for anyone to use, whether you are using our iOS, Android or Mac products or not. It's also open source (except for the addins that work with our iOS, Android and Mac products).

There are a lot of Unity customers on our forums using Xamarin Studio already.

Why choose Xamarin? 12 years ago

Everyone on the Xamarin Studio team uses Xamarin Studio to develop Xamarin Studio.

Smart Indent, afaik, has been fixed. The problem you describe was only an issue for multi-line lambdas, iirc.

Humans 1, Robots 0 13 years ago

Stop & Shop have an Android app (and an iOS app?) so you can use your phone as a scanner.

The idea of putting your most common food items that require a code in a toplevel "menu" would be great, though. Maybe you should suggest that to someone (I just don't know who).

Mono 3.2 13 years ago

The Mono project leaves it up to the packagers for each specific Linux distro do the packaging (just like nearly 100% of all other open source projects) because they are more knowledgable about the various packaging subtleties of their own distros.

GTK# is what is used to develop Xamarin Studio / MonoDevelop, so it is pretty well supported.

Mono 3.2 13 years ago

Mono 3.2 isn't mobile specific. Sure, mobile has become a very important part of the focus, but a huge portion of the development is useful beyond mobile.

if it was so obvious, why did it turn you against him? did your delicate sensibilities get hurt or something?

I honestly don't understand how this could have change your opinion on him based on something you not only acknowledge is true, but also as "obvious".

All of the Xamarin Studio changes were pushed upstream to MonoDevelop.

It's true that we don't currently have a version of the Mono-for-Android addin available for the Linux versions of MonoDevelop, though.

If the Ubuntu mobile/touch devices become popular, I have no doubt that we'll come out with a product for writing Ubuntu mobile apps as well.

We're always looking into supporting other mobile platforms.

We've just pushed all of our MonoDevelop source code to public github and I'm sure that the Ubuntu packagers will be packaging it for Ubuntu very soon.