Keeping Safari a secret

http://donmelton.com/2013/01/03/keeping-safari-a-secret/

by ryannielsen • 14 years ago

283 102 14 years ago

> We couldn’t ship with the real Safari user agent string disabled, but we came up with the next best thing — automatically enabling it after a certain date.

Does that mean that if you set the date of your computer back before 2003, the user agent will become Internet Explorer?

donmelton13• 14 years ago

If you were able to run a copy of Safari version 48, it would probably look like a Gecko-based browser before this date.

laurent1234563• 14 years ago

But he seems to be saying that the code that enables this is still in Safari.

bdash4• 14 years ago

v38. You'd not see it with v48.

barredo• 14 years ago

I just tried with Safari 6 on Mountain Lion and it does not.

Obviously this must have been a feature of Safari 1.0

donmelton13• 14 years ago

Yes, only the original version of Safari.

nsp• 14 years ago

Does anyone know why Apple stopped doing this(or stopped doing it as thoroughly)? With the possible exception of the original, sites have consistently reported visits from iOS/iPhone:iPad user agents (and claimed that the ip originated within apple) well before launch. Recent example: http://thenextweb.com/apple/2013/01/01/developers-begin-seei...

markrages• 14 years ago

I'm guessing it's not a secret or surprise to anyone that Apple would work on a new version of their operating system, nor that such operating system would include a browser.

Safari, on the other hand, was unexpected.

robmcm2• 14 years ago

Also, what were the chances that someone would even notice "Safari" in their log files had it of slipped out, vs some stats company who only work on iPhones noticing a new iPhone version.

Apple 10 years ago < Apple now

nirvana• 14 years ago

That Apple makes the iPhone and iPad is not a secret. The article you link to is talking about stats that apps are reporting over the internet... the apps have to know what kind of device they are running on, and have to be able to handle differences between the devices (eg: the screen difference between the iPhone 4S and iPhone 5).

No real information is released with these hardware model numbers that's not already reasonably known.

The only exception is for devices that people might think are end of life, or unlikely to get a refresh soon.

For instance, the Mac Pro may start showing up in logs, and that would be news, because it is so rarely updated... or it would be news if Tim Cook hadn't said that they planned to release an update (in 2013? not sure when he said it would be.)

So, knowing that iOS 7 (8, 9, etc.) are under development doesn't give away any of the secrets-- eg: what the special features of those OSes will be.

bgarbiak• 14 years ago

Well, you can get some real secrets by combining user agents with other data. For example: screen resolution.

zalew• 14 years ago

TLDR: for whatever reasons it was a very top-secret project, so they spoofed the user agent, the end.

gry• 14 years ago

The interesting story is Apple. We're hearing about it now.

jmspring• 14 years ago

Sometimes the backstory makes for a fun read. Looking for the cliff notes version all the time makes life boring.

donmelton13• 14 years ago

Thank you. That was my whole reason for writing it. :)

neumann_alfred4• 14 years ago

Looking for the cliff notes version all the time makes life boring.

Not for me, I prefer doing stuff (like typing this inane comment right here) to hearing about trivial things others did. In this case, even the cliff's notes are kinda "meh".

stock_toaster• 14 years ago

Well said.

Definitely a case where it wasn't the destination so much as the journey that was interesting. :)

georgemcbay4• 14 years ago

"Which explains why the Safari user agent string has so much extra information in it, e.g. KHTML, like Gecko — the names of other browser engines."

Wasn't Safari/WebKit largely based on KHTML in the beginning? This doesn't say it wasn't, but seems to imply that there was a separation of the two that was more complete than it really was.

I realize the OP is in a better position than I to know exactly what the reasoning here was, but it seems like reporting yourself as KHTML when you are largely based on the KHTML rendering engine would just be a sensible and practical thing to do outside of all this cloak and dagger stuff.

(Granted, I doubt there were very many websites, if any at all, that gave a rat's ass about special cases for KHTML compatibility).

guptaneil2• 14 years ago

That is exactly what he is saying. They added KHTML to the user agent string to improve compatibility with servers, since Webkit was essentially KHTML anyway, not to try to hide the actual user agent. Internet Explorer is what they used when hiding the user agent.

georgemcbay4• 14 years ago

If he is trying to say that, then he has my sincere apologies, but it doesn't read like that, at least to me.

eg. he says "the names of other browser engines", not "the name of the browser engine our code was originally based on".

The "like Gecko" part is a bit of a red herring here and mostly irrelevant, as that is simply what KHTML/Konquerer used in its own UA ("KHTML, like Gecko").

donmelton13• 14 years ago

Yes, WebKit was based on KHTML and KJS. Even in the beginning, we made a few fundamental changes. Some of those were folded back in, but eventually those differences grew much, much larger.

There actually were some sites (in Europe, as I recall) that behaved a bit better when "KHTML" was in the user agent string.

I also included it as a nod to the original project. I wanted to show respect for the fine work of Lars Knoll and the other contributors.

lelandbatey• 14 years ago

That's quite the story, and it's great that you'd keep a reference to a products origins (even if you didn't have to).

On a side note, is it still true that Apple has an entire /8 block of IP addresses? I know that a variety of companies where given them, but I was under the impression that large portions of most had been bought back/given back for use elsewhere?

rjzzleep• 14 years ago

well yeah, because you guys were sending huge blobs of text back instead of proper formatted patches. didn't make you the most welcome forkers out there.

oofabz• 14 years ago

At the time, Microsoft's hegemony over the web browser market was at its peak. Firefox was the only real competitor. If web pages checked the user agent, they probably only accounted for these two browsers. Anything else fell back to legacy behavior, disabling DHTML (later called Web 2.0). This made Gmail suck.

KHTML had almost no market share - it only ran on KDE, and most KDE users used Firefox. Nothing recognized its user agent string, so reporting "KHTML" would have been as ineffectual as reporting "Safari" or "WebKit".

georgemcbay4• 14 years ago

"At the time, Microsoft's hegemony over the web browser market was at its peak. Firefox was the only real competitor."

Actually at the time this is referencing Firefox didn't exist (Gecko did, but Firefox didn't). I'm well aware of the history and why browsers trick out their user agents. If you want to go one level deeper, look no further than the fact that IE still pretends to be Mozilla because of the early days when they were late to the party.

donmelton13• 14 years ago

Firefox wasn't a big factor when Safari debuted. And the project was still called "Phoenix" back then. I believe that other Mozilla and Gecko-based browsers were more widely used at that time.

mpyne2• 14 years ago

> KHTML had almost no market share - it only ran on KDE, and most KDE users used Firefox.

As mentioned before, Firefox didn't even exist at this point. And KHTML itself actually predates Linux Gecko by a few months as well.

Don Melton forked KHTML/KJS internally in 2001, and announced the work to KDE at the beginning of 2003... GMail wasn't released until 2004!

cpeterso• 14 years ago

If they were worried about server admins correlating Apple's IP addresses with a new User-Agent string, the team could have VPN'd to a non-Apple network to access the web.

laurent1234563• 14 years ago

I guess they were worried both about having recognizable IP addresses AND having the name of their new browser appear in logs all around the world.

The user agent was something like "Mozilla/5.0 (Macintosh; U; PPC Mac OS X; sv-se) AppleWebKit/85.7 (KHTML, like Gecko) Safari/85.5", which people would have quickly correlated to Apple, not matter what IP it came from.

donmelton13• 14 years ago

The string was originally just contained "WebKit." When it became "AppleWebKit" in mid-December (as I recall) then, yes, it was more likely to be discovered. But I felt that a 17.x.x.x address would be the clincher and the secret would be out.

duopixel• 14 years ago

I distinctly remember how the web dev world (myself included) groaned at the prospect of supporting another rendering engine. Gecko was clearly the best browser at the time, and the choice of KHTML seemed bizarre, to say the least.

Even the intention of Apple building it's own web browser seemed weird, Apple had failed at a previous homebrewed browser (Cyberdog), why not just build on top of what Mozilla had already done?

I remember Apple stating that Konqueror's code was much more leaner, faster and modern. I've always wondered if the birth of Safari was an early sign that Apple was interested in developing the pieces missing for an internet device. Safari seemed like a godsend foresight from Apple when they released the iPhone, but I can't help wonder if it was planned this way all along.

donmelton13• 14 years ago

I would love to claim the foresight of planning for the iPhone all along. Alas, that was not the case. :) For any of us.

We built our own browser because we didn't want to depend on another company for a critical application.

We built our own browser engine because we wanted to use the technology in more things than a browser.

We built that engine small and fast because Bertrand Serlet would have shot me if I had done otherwise. :)

You have to remember that Mac OS X itself was smaller in those days. Not iOS small, but considerably smaller that it is now.

After the initial success of Mac Safari, there was a time when I was second-guessed by some for choosing KHTML and KJS as the basis for WebKit. When we decided to do the iPhone, I was suddenly a genius again. :)

That's just the way it works in the real world.

dphase• 14 years ago

Don, you've had a couple of great stories make it to HN recently -- awesome reads. Looking forward to reading many more.

btipling• 14 years ago

That's twice you mentioned 'being shot' for failure. Sounds like a real culture of fear. I get the excitement for a chance to innovate the future of computing, but I sometimes get a sense of real Stockholm Syndrome from people at Apple. I personally don't want to work at a place where people are so afraid of failure. Look at Forstall's recent exit. All those years changing the world through computing, and at the end he gets a disgraceful boot out the door.

You can change the world from a growing startup without having to fear for your career at every turn. You can have fun without being afraid. Imagine that!

officemonkey• 14 years ago

>We built our own [map app] because we didn't want to depend on another company for a critical application.

I believe, in retrospect, Apple's Maps.app will be considered in the same league as Safari. It may have had a rougher start than Safari, but I think it's a great program.

lgg• 14 years ago

It is probably just the dumb luck of having started the same day as you and Maciej (and thus going through orientation at the same time), but it was completely obvious to me that you were building a browser team, at least once you stopped working on Mail.app bugs during the Puma crunch and he moved over to your team a few weeks later ;-)

To drive home Don's point about how secret Alexander was: I was given official access to Marklar months before Alexander. Since there was no Rosetta at that time the Intel builds of OS X had no bundled web browser (and porting Chimera or Mozilla were clearly nowhere on that team's priority list at the time, for obvious reasons). I was eventually given access to Alexander largely because I was doing enough Intel related work that it was necessary to have access to a browser on my Intel machines.

michael_miller• 14 years ago

I suspect the reason for choosing KHTML over Gecko may have been more political than anything else. Since Mozilla was the dominant non-IE browser at the time, it would be hard to justify any fork to the community without meaningfully given back. For example, Apple could go off into a cave and add touch handling support with momentum scrolling and GPU acceleration, and the Mozilla community would be absolutely furious with Apple for creating effectively unusable work, assuming they did the work over several months/years in secret. The last thing Apple wants is a large group of loud people complaining about itself.

By comparison, KHTML was not really a serious project. It had a few developers, but nowhere near the scale of what Mozilla had. Thus, when they forked it to do their work in secret, only a couple people complained.

ryandvm• 14 years ago

> By comparison, KHTML was not really a serious project. It had a few developers, but nowhere near the scale of what Mozilla had.

You'd be surprised at the negative impact that a large number of developers can have on the quality of a software project.

Someone2• 14 years ago

When Safari came out, I happened to just have looked at the Mozilla code. A significant part of the "scale of what Mozilla had" was due to the design decision to couple the major browser components using a home-grown (and, hence, included with the sources) clone of Microsoft's COM.

To make matters worse, unlike Microsoft's version, the Mozilla version needed to support multiple OSes and multiple compilers.

Because of that, I believe Apple's claim that the choice for KHTML was on technical grounds.

(I don't remember whether Mozilla already used XUL at the time. If so, that could have been another argument for choosing KHTML)

lmm• 14 years ago

Konqueror was by some distance the best browser available at the time (even today there is Konqueror 3.2 functionality that I can't find in anything else), and the fact that they were able to keep it that way with about 5 devs total spoke to the quality of the code.

Yaggo2• 14 years ago

Agree. I used KHTML (later known as Konqueror) as my primary browser (as a webdev!) since early days of KDE until I switched to Mac in 2006. Konqueror always felt right on KDE, just like Safari feels right on OS X (native feel, integration, etc).

A little feature I still miss was that you could navigate up in URL hierarchy by alt+up, just like in file-browsing mode (not sure if it's still supported by today's Konqueror).

Someone2• 14 years ago

I would not call Cyberdog a failure. Yes, it wasn't fast and it didn't have a great rendering engine, but frankly, neither did the competition, and I found its look and feel so good that I kept using it for a long time (with IE as a backup for pages it did not handle, IIRC)

OpenDoc, on which it was built, failed, but even that, I would not call a complete failure. It may have just been bad timing (of the Newton vs iPhone kind). If you are looking for a large hobby project: I would love to see something OpenDoc-like today running in a browser. Require viewers to be freely downloadable from e.g. GitHub, but allow developers to sell their editors, and see where it gets.

Tloewald4• 14 years ago

Agreed — Cyberdog was the best thing about OpenDoc.

lnanek2• 14 years ago

Amusingly, before Chrome, Google used to contribute a lot of engineers to Firefox/Mozilla. They decided to do their own after deciding they weren't happy with it. I don't quite recall the main reason, might have had something to do with some sort of mess for cross compiling or variable sizes or something mainly.

jiggy2011• 14 years ago

Interesting, I wonder if it would be possible to start a rumour by having a number of people spoof their user-agent string?

Sure , they wouldn't route back to an Apple IP address but then who's to say Apple doesn't browse via proxy/vpn somewhere?

Tloewald4• 14 years ago

No-one would credit rumors that Apple was wasting its time creating a new browser.

sxp• 14 years ago

>Back around 1990, some forward-thinking IT person secured for Apple an entire Class A network of IP addresses.

I found that fact to be the most interesting part of the article. The list of other Class A holders is also interesting: http://www.aturtschi.com/whois/neta1.html

1amzave• 14 years ago

Indeed -- also interesting are the results of an RDNS query on the first IP address listed in the article (17.149.160.49).

davvid• 14 years ago

Back around 1990, some forward-thinking IT person secured for Apple an entire Class A network of IP addresses.

Pixar has a large block too... I wonder if Steve had something to do with it?

jdechko3• 14 years ago

I knew that going in, but only because of this: http://xkcd.com/195/

Raz0rblade• 14 years ago

must have been easier to use an ip filter and base the identifier on ip... duh..

to3m• 14 years ago

A device behind a router might be unable to reliably determine its outward-facing IP.

smackfu• 14 years ago

This blog is revealing more interesting tidbits from inside Apple than the official biographer of Steve Jobs. How sad is that?

TazeTSchnitzel• 14 years ago

Quite sad, but not honestly surprising. That was a biography made to be mainstream, to tell the general public how glorious Steve supposedly was.

petemc_• 14 years ago

I take it you didn't read it.

mbreckon• 14 years ago

Don, would you be willing to share a bit in future posts about how you grew the team, how you organised yourselves and how that changed over the course of the project? Also, what thought process did you go through in the decision of how much to include in V1.0 vs leave to later releases? As a team lead/product manager I'm always interested in hearing other people's experiences, and your experience and writing style put you in a better place than many to do that. It is the thought processes and the dynamics of interacting with others in an organisation that make software development stories interesting.

JohnFromBuffalo• 14 years ago

This still makes me think that Ninjas work at Apple. How other can they evade such a public part of the interwebs? http://www.askaninja.com/ I guess.

astrange• 14 years ago

What a nicely transparent spam post.

jagermo• 14 years ago

This seems like an insane waste of resources just to keep a secret.

Did this get you in any programming troubles in the long term?

Keeping Safari a secret

Related Stories