We reduced our iOS app launch time by 60%

https://doordash.engineering/2023/01/31/how-we-reduced-our-ios-app-launch-time-by-60/

by wallflower • 3 years ago

190 161 3 years ago

doordash.engineering

kogir• 3 years ago

I wish they’d name the third party library.

JKCalhoun• 3 years ago

I thought the same, especially after said-3rd-party didn't respond to their request for a lazy initializer.

But then I'm thinking, what does it matter? Is this the only selfish 3rd-party library out there?

Maybe better than: "Here is a bad library to link against", the takeaway is "all 3rd-party libraries are a potential liability."

ChrisMarshallNY• 3 years ago

"all 3rd-party libraries are a potential liability."

Yup. Even if they are responsive today, doesn't mean they will be, tomorrow.

My views on The Dependapocalypse are not usually welcome, in modern discussions of software strategy.

Not exactly sure why they are using string(describing:) in ship code. I suspect the architecture may need a bit of a look-see.

I only use it in debug tracking. It's the reflection API, and that's not really something I'd consider for runtime. Sort of like using exceptions for branching. It works, but your results may not be optimal.

UPDATED TO ADD: I did think of one place I might use it (actually, the ".description" computed property), and that's in a "one-off" bit of code that may be doing something like figuring out how to deal with some JSON I parsed, or reading stored prefs. Otherwise, I've usually figured out other ways to deal with it. One advantage I have, is I seldom use third-party libraries, and can actually go into my modules, and tweak them to serve the frontend better (in fact, I just did exactly that, this morning).

jmull• 3 years ago

I don't think we can conclude from this that there's anything wrong with the third-party library people... there are tradeoffs and doordash probably isn't their only user. Doordash may not be paying or contributing either. Maybe the 3rd-party library people could have sent a basic "Thanks for the feature request" reply for the sake of politeness... But lacking that polish doesn't make them "selfish".

danpalmer• 3 years ago

I'd put my money on either the Facebook SDK or a crash/bugreporting tool. They say it's important to UX, and both of those are (for sign-in-with-FB, or bugreporting).

hnburnsy• 3 years ago

Fingerprinting maybe?

ericlewis4• 3 years ago

No FB SDK, they do have a bug reporting tool - but you always would want it to init immediately simply to catch those crashes which occur before the rest of app setup. It is unlikely this. My money is on google maps.

saagarjha17• 3 years ago

Not Facebook either :)

andyjda• 3 years ago

I'm with you, but I also think that discussion about specific tools often is wrongly prioritized over discussion of general principles. The lesson should be "always profile your programs, and be especially wary of third-party dependencies" rather than "avoid this specific library at all costs."

I'm assuming they also understandably don't want to start a flame war, where their specific criticism of the library is going to be taken to mean "this library is always bad, do not use it, even if its effect on performance is negligible in your case."

KoftaBob• 3 years ago

If I had to guess just based on the most contacted 3rd party domains their iOS app pings, it's either Segment.io or Sentry.io

ericlewis4• 3 years ago

They use neither according to the IPA I just pulled apart.

ericlewis4• 3 years ago

I am going to guess it is google maps. Here is a non-exhaustive list of frameworks they have in the app:

- Google Maps

- Persona

- InstaBug

- CardVerify

- SendBird

- Stripe

- Google Sign-in

- PhoneNumberKit

- Riskified

- VGS

"The third-party framework in question had a total of nine module initializers". I would guess it is probably google maps simply because they tend to do things like this and IIRC you have to setup in the startup even if you use the map later.

Might have missed some frameworks but I think I got most of them.

Edit: maybe not google maps... it seems that 200ms added to any app that uses it would be fairly noticeable. Perhaps one of the lesser used ones.

philsnow• 3 years ago

out of curiosity, how did you find out the frameworks used in the app?

saagarjha17• 3 years ago

Not Google Maps (which by the way has 51 initializers) :)

taytus2• 3 years ago

I apologize if this comes out as negative, but it truly is an honest question.

They are a tens-of-billions company. Why is this news? Shouldn't this be just part of doing business?

kacy• 3 years ago

Because it's a helpful contribution to the the engineering community?

taytus2• 3 years ago

Helpful how? They used a third party library and they didn’t even mentioned it.

dmak• 3 years ago

Recruiting

joseph_grobbles• 3 years ago

It's an engineering blog targeting other engineers. Some engineers discovered something that had a significant impact so they are communicating it to more engineers.

weberer• 3 years ago

Many people here are iOS developers who would like to see how to improve launch times in their own app.

friedman23• 3 years ago

It's their engineering blog... This is what engineering blogs talk about and you are on hacker news where things like this are of interest.

lunarboy• 3 years ago

It's just PR, and/or some internal stunt by the eng team for better performance reviews. Both of which are fine, if the article itself is actually useful. In this case, I feel like the only thing that could have been interesting is the dyld workaround, but they don't go into depth with that at all. The first two "solutions" feel like they were just doing it inefficiently in the first place, rather than some novel technique worth sharing

saagarjha17• 3 years ago

Most performance wins are someone doing something inefficiently in the first place. Shocker!

dagmx• 3 years ago

That’s quite nihilistic.

Why are any of the other posts on HackerNews here? Many are less technical or less interesting, depending where your interests are.

This post at least shows how certain things might be debugged and some performance tweaks that people might forget to make.

The bar for interest is arbitrary. One can choose to not engage and move on if they don’t think it’s interesting.

bfeynman• 3 years ago

Doordash is a zombie company propped up by VCs, their tech is trash. I remember reading their "ML Engineering blog" and it was the most elementary crap every, they just used google maps apis for delivery and we're trying to do point in polygon calculations. The fact they herald things that are kind of simple and obvious to speaks volumes.

colpabar• 3 years ago

Now if only they could retain the search parameters when you hit the back button. Currently it takes you to the default list with no filters applied.

hnthrowaway03154• 3 years ago

Never done any serious development, can someone please educate me whether any of these three issues can and should be targeted earlier in development? For example is any of the first two directly going against some well known best practices? I'm judt curious, not trying to insinuating anything as I don't understand iOS development.

ab8• 3 years ago

No. Functionality first. Optimization later.

hnthrowaway03154• 3 years ago

Gotcha, makes sense.

coldpie3• 3 years ago

Using Strings in any performance-critical situation, like object identifiers, is a newbie mistake (but also extremely common). That one shouldn't have happened. The others just strike me as things one learns about and fixes as the project matures and someone bothers to do a perf analysis to find the low-hanging fruit.

hnthrowaway03154• 3 years ago

Thanks! Is the reason Strings impact performance because we can use smaller objects to do such work while Strings are larger?

saagarjha17• 3 years ago

…this is not very good advice? Using strings is fine, until it becomes a problem. People are going to read your comment and decide to change all their strings to integers or something at the cost of maintainability.

strulovich• 3 years ago

A lot is silly issues can be ignored in any small app, and will become big later on.

If you picked good habits from past scars, stick to them. Otherwise, concentrate on functionality, dogfooding on real devices that real people have as much as possible, and use profiling to find the bottlenecks that emerge when you find the time (since you’re focusing on functionality).

(Exceptions to this rule exist, like if your product is some cloud service then speed might be an actual functional requirement of whatever you’re building)

hnthrowaway03154• 3 years ago

Thanks!

bombcar• 3 years ago

Functionality first, but I'd say that you should have a healthy skepticism on third-party libraries - not a "we'll write our own instead" but a "do I really need this feature/tracking".

marginalia_nu2• 3 years ago

Actually, writing your own is often underrated.

In many cases the time and code required to incorporate some aspect of a 3rd party lib to do something is similar to the amount of time and code in building what you need. In that case, the dependency only serves to make your code more complex (and slow).

This is not always the case, but surprisingly often.

m3kw93• 3 years ago

No this would be over optimization

marginalia_nu2• 3 years ago

You'll want to fix most of the problems relatively late in development, although I can't recommend enough testing on the most low-powered hardware from the get-go. It will let you know immediately when you've written slow code.

It's easier to fix a design mistake just when you've made it than when you've got six months of development hinging on the poor design choice.

bcardarella• 3 years ago

It would be nice if DoorDash cared as much about their web app performance as they do their native app performance. Their web app perf is abysmal. Pathetically slow. Clicking takes seconds to see a response in the UI.

aio2• 3 years ago

I know this isn't the point, but I suffer bad connections all the time. I'm so used to a click taking 5 seconds.

Anyways, I do agree with your statement.

modeless2• 3 years ago

Oh my God, how did they do this? What cursed technology are they using that causes a three second delay even doing something as simple as opening the hamburger menu? It's infuriating.

dylan6044• 3 years ago

setTimeout(annoyUserIntoNativeApp(),3000);

CyberDildonics• 3 years ago

Maybe there is a conference somewhere that includes instacart, the reddit redesigners, twitter, facebook messenger and instagram and they all share their tips for making their interfaces as slow as molasses.

orangepurple• 3 years ago

Opening the menu launches a WebAssembly virtual machine for their Docker images to build and run to return the appropriate options /s

In all seriousness though, the tech industry is an immense contributor to global warming because of inefficient software which uses large quantities of energy and necessitates the creation of massive quantities of e-waste as older machines are unable to provide an adequate user experience for extremely computationally heavy designs.

Extremely inefficient software which is necessary for everyday tasks also means only the latest hardware from China/Taiwan is competitive. Bad software is a geopolitical and national security risk.

pimlottc• 3 years ago

Do you mean the hamburger menu or the hamburger menu? :)

sebdufbeau• 3 years ago

Quick look at the network tab shows two network requests, they are launched in parallel and take 250ms on my end.

The stunning part is one of these is to get a string of text that is displayed in a banner that is only visible if you scroll in the menu (like, at the very bottom). You could easily defer this call to happen once the menu is open and have 0 impact on the UX.

The other call, I have no idea what it does, but is launched both when you open and close the menu.

jaywalk• 3 years ago

Opening the hamburger menu results in 61 network requests!!!!!

yamtaddle3• 3 years ago

Well you see first you hit a Javascript click listener. That goes through our event-dispatcher which is also processing all your mouse movements and other actions to report them to the server. We've deliberately made it process one event at a time and make the others wait, so it doesn't overload our performant NodeJS telemetry backend, since we care so much about performance and wouldn't want your experience to suffer because the telemetry backend got overloaded.

Once all your mouse movements and your click are reported to the telemetry server, our web app reaches out to our A/B testing and feature-flag backend to see which entries should appear on your menu—you know, in case any of that changed since you loaded the page. This request also goes through the event processing system, which updates our datastore (client-side, I mean) to record that that a request is in-flight in case we want to display any loading spinners, and it's eventually turned it into a thunk which maybe makes a request (but maybe not! Hence the thunk!) which (maybe!) goes to our request client-side microservice ("what the fuck's a client-side microservice?" LOL OK boomer, are you a time traveller from 2005?) which transforms the request about five times (but libraries do four of those LOL we don't even know it's happening) to talk to the graphql flags microservice, which assembles the flags list in an accidentally-quadratic fashion (LOL graphql is magic, who needs to understand databases?), taking a full 500ms to process the request.

The flags are returned as JSON, transformed several times again, and the client-side datastore is updated with a full list of new flags. Since new flags can affect anything, everything re-renders. First, we need to update the shadow DOM, but to do that....

[... five paragraphs later ...]

And then your hamburger menu is on the screen!

Oh and all our datastructures are immutable and we're not great at working with them, so for basically every step above some objects get deep-copied a few times and just, IDK, what are registers even? Client- and server-side both. For safety! Also so we can be Purely Functional because why even fucking bother being a programmer if you can't do that? Like literally just die if you're not writing eighty HOFs a day. You have no idea how elegant all this code is. So elegant.

halikular• 3 years ago

How any of these developers can call themselves engineers and still look themselves in the mirror is beyond me. An engineer's job is to design the most efficient structure in as little time, and cheap manner as possible that still produce a reasonable and satisfactory result. What all modern web apps have in common is that they cut all corners on development time prior to release to then be forced to optimize the hell out of the mess afterwards in perpetuity. A simple tech stack would accomplish the same tasks with minimal extra initial development time and effort, but would remain performant for a very long time.

raggi• 3 years ago

They use tons and tons of third party trackers and advertising brokers for populating parts of the page.

They have broken sha512 subresource integrity links on the page.

The site produces no less than 11 samesite cookie misuse warnings.

5.71MB compressed. 520 requests to load the page.

Clicking on a store performs another 224 requests. Loads another 449kb.

Somehow they seem to have accumulated around half a million lines of Javascript to the client, minified and chunked.

eddieroger• 3 years ago

I am not disagreeing that they shouldn't focus on their web app, too, but I assume they look at how often the mobile app is used versus web app and are just prioritizing for user base. If more people used web, I'm sure it would get more love.

hansvm2• 3 years ago

They really need to look at counterfactuals though - if our webapp were better how many people would use it. The fact that a mediocre site with a better alternative (the app and competitors) doesn't get much traffic should hardly play into the calculation at all.

nicoburns• 3 years ago

More likely it's just a different team of people building it, and one cares more about performance than the other.

oefrha• 3 years ago

You should hope they don’t pay attention to their web app, since the vast of majority of web sites/apps from big companies seem to only get slower every time a developer touches it. Major redesign? Expect major slowdown at least half of the time.

I recently got back into posting videos to YouTube after ~7 years. Can’t fucking believe how slow the video upload UI has become. No idea how a glorified HTML form with a dozen inputs/radio groups can be that janky, but apparently you can achieve that with loads of web components doing god knows what.

taw28• 3 years ago

The cynical part of me thinks that the bad web app is intentional. They can get much more data from you if you use the app, so they want that to be the best experience. All others should be bad enough that you will only use it if there is no other way to give them your money.

reaperducer• 3 years ago

Even worse: Every time I order something from a restaurant online, only the first line of the address goes to the delivery driver.

Every. Single. Time. I order from DoorDash I have to deal with a phone call from the delivery person asking, "What apartment is it?" Or worse, living somewhere with a gate code. Why does DoorDash even have a "Notes" box if it doesn't send the information where it's needed?

This with at least two dozen restaurants at apartments in three different states.

siva72• 3 years ago

The backend story is still being prioritized for this task

rcme• 3 years ago

Just checked, they use React. React has set web performance back decades.

pengaru• 3 years ago

In a smartphone-dominated world, you're lucky there's any webapp at all.

thdc3• 3 years ago

Impressive improvement, but is mobile app startup time that important after a certain threshold? I'm not a mobile dev.

If the flame graph extends over the entire startup time, then I'd estimate it to be around 600-700ms (wish they gave the numbers rather than the percentage) which already sounds very nice as a user. If it was like 5 seconds then it would be amazing.

Furthermore, I'm assuming that this only affects when the app is initially started instead of when it is already running in the background or navigating between screens, which is why I feel the linked article on why latency matters in tfa doesn't really support it.

I'm curious about what metrics mobile developers use to prioritize tasks.

m3kw93• 3 years ago

Yeah 600ms to 300ms is noticeable, if you play online games 200ms lag gets killed by 50ms everytime

soiler5• 3 years ago

But that's exactly the poster's point. You don't get headshot in doordash. Slightly slower performance doesn't actually change anything about your use of the app, until it becomes slow enough to make you stop using it (thus, the threshold).

0x457• 3 years ago

Online multiplayer video game latency and app launch latency are very different things and perceived very differently, though.

kybernetyk• 3 years ago

For Doordash I'd say it doesn't matter if the app opens in 700ms or 7000ms. Hungry people are willing to wait a little to order their food.

But then again: How did this get so bad in the first place? I mean, yeah, premature optimization and stuff ... but ...

One of the biggest immediate standouts was the time we spent on Swift protocol conformance checks (checking if a type conforms to a protocol), but why?

Correct: But why? Why would you do this in a static typed language? Conformance checks should be a rare exception.

Architectural principles like the single responsibility principle, separation of concerns, and others, are key to how we write code at DoorDash.

Oh, OK, architecture astronautics at play, I guess. Sorry, but you can separate concerns and all that other stuff without ending up with what looks like a dynamic typing system.

/rant (I'm hungry)

sond813• 3 years ago

Apple recommends 400ms for startup time because that's how long the app open animation takes. It's also hard to tell from just one flamegraph how long it will really take in prod, that's why the percentages are nice to use. I've mostly used percentiles like p95 to track and prioritize mobile perf work. 600ms on one phone could be 2s on older hardware with other factors slowing it down, but it depends on the distribution of doordash users.

thdc3• 3 years ago

I actually came across the 400ms number in a 2019 WWDC talk while researching the topic! It said 100ms of that time was delegated to the iOS system for setup, leaving 300ms for whatever your app does.

Granted the 100ms section included System Interface - DYLD3 (dynamic linking) as part of it, which includes the 200ms portion that was optimized in tfa so I don't know how accurate that 100-300 breakup is.

The talk also says to avoid dynamic library loading during launch to improve time which is interesting, as it suggests there's an alternative, which may or may not be what Doordash did here.

PragmaticPulp• 3 years ago

I generally lead teams to move quickly to get products usable, deliver the most important features to feed the business, then go back and start optimizing things like launch times.

The DoorDash app is feature stable as far as I can tell as an infrequent user. It makes sense to have the team invest into small optimizations for experience and speed. Much better than letting idle hands start redesigning things that aren’t broken or doing new features for the sake of doing more things.

Also, a 600ms launch time on a high-end phone could be a 5s launch time for someone using a cheap, old phone. Optimizations are most noticed by the lower end of your userbase’s hardware.

MetaWhirledPeas2• 3 years ago

I'm curious about what metrics mobile developers use to prioritize tasks.

This is a very MBA take. Task priority should always be considered beforehand, but whenever there is some leeway (including "20% time") I contend that performance tasks should be given some preference. There is almost no end to the ill effects of slow software.

mattgreenrocks2• 3 years ago

Given the performance of most mobile phones, hitting below 400ms to launch and be usable feels like a low bar to cross.

saagarjha17• 3 years ago

Depends on what you consider important. Do you want to reduce churn? Optimizing startup performance is one of the best ways you can do that.

m463• 3 years ago

There are some tried-and-true principles that have worked since the dawn of computing.

For example, if response time is less than 0.1 sec, whatever the user is doing is "interactive" and does not disrupt the flow of the task.

I remember there were similar statistics for web page response time (number of lost views as response time increased). These seem like they might be similar to app launch speed.

That said, I wonder if this "ios spp launch time" post was something that management pursued in a data-driven way, or some engineer getting annoyed and scratching an itch (or something in the middle)

brankoB• 3 years ago

Great, now make it so tracking my delivery driver doesn't freeze my phone and force me to restart.

nblgbg• 3 years ago

In short, they are using a third party library (they have not named it though) that accounts to 40% of the app launch time and they deferred loading of this library to later point!

siva72• 3 years ago

Thanks, it's astonishing how many teams make this mistake, simply putting some hungry library into the code to solve some story without thinking much further about the overall implications

robbyking• 3 years ago

I don't know if it's still the case, but some third party libraries are required to run on launch.

I worked on a an app where the Facebook login library took 600ms to load, and was required to run on launch.

saagarjha17• 3 years ago

I'm like 90% confident that it's their crash tracking or analytics library :P

Edit: It's a framework from Salesforce called "ServiceCore". In case you were wondering it decided that it was necessary to request a list of all frameworks in a static constructor so it could find one that contains a certain class of theirs that does bundle-relative image loading. Pro tip, don't do this.

js2• 3 years ago

If it's a crash manager, you have to load it early if you want to catch early crashes. Still, only the Mach exception handler really needs to be installed. I wrote an in-house crash reporting SDK based around PLCrashReporter and the whole thing loads in under 20 ms on a modern device.

200 ms is insane for an SDK, any SDK. Our company had that as its target cold start time to first interaction across all of our apps and it's something we watched religiously as part of launching new app versions.

hawski• 3 years ago

Most work on making things faster is to give a shit.

lofaszvanitt• 3 years ago

These people are very professional. They must get a salary raise. Their tech is advanced, way above threshold and the service they offer is perfect. Give them a medal.

baggy_trough• 3 years ago

If they care so much about the user experience, how come they never came out with an iPad version?

notjonheyman• 3 years ago

This would be more impressive if I still used Door Dash.

idiot900• 3 years ago

The DoorDash iOS app is so horrendously slow and laggy on my iPhone 14 Pro (as it was on my 11 Pro) that I’m using other delivery services whenever I can. That means if the restaurant I want to order from is available on another platform, I’ll use that.

mech4bg2• 3 years ago

At DoorDash, we take app startup speed extremely seriously. We are obsessed with optimizing our customers' experience and making continuous improvements.

I wonder if they actually believe this? They make it almost impossible to report bugs in the app, and when I have customer support's reaction has been "there's not really anything I can do about this."

Spivak• 3 years ago

This is because most B2C apps have realized that bug reports are pretty much worthless and cost more time to sift through than any potential upside. Why report bugs when you have crash handlers, telemetry, and detailed logs of everything every user does in the app. Your metric dashboards and reports can actually see and measure issues that actually affect the business.

mech4bg2• 3 years ago

Not an unfair point, although it is alarming at work how many bugs end up being reported directly from customers.

rocket_surgeron• 3 years ago

An astronomical simulator loading dark sky data, grabbing your location, charting the current positions of the stars in relation to your location, displaying detailed graphics for constellations, implementing gyroscopic controls for real-time manipulation of the viewport, and loading all overhead objects into memory: loads nearly instantaneously, responds quickly.

A food delivery app: so bloated and sluggish it is borderline unusable and 100% embarrassing.

tdiff• 3 years ago

I wonder how exactly did they manage to prevent dynamic loader from initing that framework.

We reduced our iOS app launch time by 60%

Related Stories