Heap for iOS: the capture-everything approach to mobile analytics

http://blog.heapanalytics.com/?p=76

by raviparikh • 13 years ago

79 63 13 years ago

blog.heapanalytics.com

sama• 13 years ago

this is how analytics should be done. congrats, guys. proud to be an investor!

randall• 13 years ago

I've been having an awesome time with Heap for the web. Their response time is fantastic for user feedback too.

Glad they exist.

usujason• 13 years ago

Is the "capture everything" approach to analytics a way to differentiate themselves from more enterprise level mobile analytics solutions such as Localytics?

kevingibbon• 13 years ago

Been testing Heap on iOS out for the last few days. Awesome product. Congrats guys!

eliperkins• 13 years ago

Curious if this means that the Heap SDK will require for the shipped app to be symbolicated or not, given how their example is written.

nwhitehead• 13 years ago

Do you have any case studies showing what the actual overhead is? I love the idea but would worry about negatively impacting the customer experience.

matm8• 13 years ago

CPU-wise, our SDK adds up to 2% overhead to normal UIEvent-handling. This is with frantic and constant user input - in practice the overhead is much less.

Network-wise, we batch user activity and by default make a network request every 5-10 seconds. We'll expose a config variable so devs can tune this frequency on a per-app basis.

anonyfuss12• 13 years ago

You're keeping the WWAN/WiFi interfaces awake, which burns a significant amount of battery, just because you want to gather data from the user.

On top of that, burning 2% overhead on UIEvent handling (which also impacts battery).

What a waste for something that users don't actually want you to be doing in the first place.

DenisM7• 13 years ago

Keeping network alive like this is not cool - the radio will drain the battery. It would be much better to not only allow a configuration for how often the sync happens, but also allow the app to manually trigger sync, normally at the same time as other network activity is going on so that no extra power is spent bringing the radio up and down.

drewvolpe• 13 years ago

How much does it affect battery life ?

I would imagine 2% extra CPU plus frequent network connections would add up to a lot of extra battery drain.

boctor• 13 years ago

We need more mobile metrics innovation so I welcome this.

Coincidentally just today I released an open source library on iDevRecipes to do smarter metrics. My approach is to use base classes.

To achieve the no extra code part, is the SDK using swizzling‎?

anonyfuss12• 13 years ago

> To achieve the no extra code part, is the SDK using swizzling‎?

They must be, which is a serious reliability/stability/correctness issue in its own right.

idunno246• 13 years ago

more likely they are installing a custom responder. At least, I think it might be viable with that

DenisM7• 13 years ago

You can easily intercept all UI events via UIWindow or UIApplication subclass, no swizzling required. I do this routinely to detect when a user is idle: no events - no activity.

jazzychad2• 13 years ago

So, I'm curious how this will compete with free offerings like Flurry which do similar-ish mobile analytics tracking and can handle millions of users per app for free. I got very excited about this Heap announcement but looked at the pricing. Multiple thousands/month for sub-million unique users may be considered steep (but then again I'm a cheap developer). I've been looking for a Flurry alternative (since I just want analytics and none of their ad junk).

Mobile apps are kind of a different beast from websites. Apps can have huge number of unique users, whereas SaaS websites tend to have many fewer users by comparison.

matm8• 13 years ago

We'd definitely appreciate more data points on pricing - care to send us an email at team@heapanalytics.com, and we can discuss more there? It'd be great to understand what sort of price point works for people like you.

shloime• 13 years ago

I've wanted to use heap in the past, but the plans aren't very startup friendly..

matm8• 13 years ago

Similar to my comment above: it'd be great to understand what sort of price point works for people like you. Feel free to send us an email at team@heapanalytics.com.

DenisM7• 13 years ago

What is there on the analysis part?

Restoring user behavior from touch events is a bit like restoring a steak from ground beef. Do you guys offer any help with that? Something to string individual events into a path?

matm8• 13 years ago

You can try out the analysis part with some sample (web-only) data at: https://heapanalytics.com/dashboard/demo

We let you define events after-the-fact based on the gesture type and the corresponding target variable. Once that event is defined, you can include it in funnels, view it in user-specific activity streams, slice n' dice it, etc.

DenisM7• 13 years ago

Thanks for the demo link, that clears things up a bit.

FYI, I don't think you're using the work "cohort" correctly in the video. Cohorts are stable groups, composed of the same people over time. "Users who send 5 links" is not a cohort, as the group's composition changes over time.

DenisM7• 13 years ago

How do you define user's "join date"? Is it the first time you saw the user?

For that matter, how do you identify the user? If the user logged in from two different machines, does it count as one user or two?

eps• 13 years ago

Fucking hell. This is exceptionally disturbing.

This sort of service needs to be legally regulated now. The app devs must be required to warn in large block letters that evryhting you do in the app, even while offline, is streamed to some random company. You can argue all you want that "everyone's doing it", but it doesn't make this any less unacceptable and going against basic expectations of privacy.

ammmir• 13 years ago

i think app devs may already be liable if they haven't set up a proper privacy policy for their app that explains what data they collect and how it is used.

i would not automatically enable analytics tools like these without consulting a lawyer.

jstsch• 13 years ago

Agree 100%. I am amazed by all the positive remarks around here. I understand the startup-vibe, but I would expect 'hackers' around here to take note of privacy concerns.

jazzychad2• 13 years ago

Sorry to bust your bubble, but most every major app (and every single game app) is already doing this by sending activity data to one company or another.

anonyfuss12• 13 years ago

Doesn't make it right. It's also astonishingly user-hostile; wasting the user's resources (bandwidth, battery life, cpu) on things they don't care about, without their permission.

On top of that, doing anything economical with the analytics is very rare. It's very difficult to isolate meaningful variables, and you're almost always better off spending that money paying a designer to tackle issues and features that really ought to be obvious to you already ...

... unless you're Zynga, I guess.

interpol_p• 13 years ago

We don't do it in our apps or games. And I know quite a few developers who make analytics opt-in via settings (as well those who provide an opt-out). Big brand games probably do it, though.

DeusExMachina2• 13 years ago

Aside from using some user resource (like battery or network) how is this different from what website have been doing for a long time? There are tools to even track the mouse movement of the user.

Which is not the "everyone is doing it argument", but more "why does this need legal regulation now?". Why not before on web apps? The privacy implications look the same to me, but the point does not seem to be raised in the latter case.

anonyfuss12• 13 years ago

> Aside from using some user resource (like battery or network) how is this different from what website have been doing for a long time?

1) Network requests (and thus communication with a remote machine that can log those requests) is an innate facet of the web. You can't load a site without explicitly choosing to make a network request. An app, however, might have no innate reason to make a network request at all.

I think #1 is the most important, but also:

2) JavaScript can be disabled.

3) HTML/JavaScript and browser network requests can be reviewed. Mobile applications are almost completely opaque to users.

4) Cookies can be blocked to reduce tracking surface.

5) Web analytics and cross-web tracking is already something people are concerned about, resulting in the introduction of do-not-track and similar.

Desktop apps have almost always explicitly asked permission before tracking users, whereas the web community seems to have brought to mobile a blasé approach to privacy and user respect.

socillion2• 13 years ago

Disabling Javascript is an easy opt out for websites.

This service is designed to produce user specific data. judging from what they record on websites (see docs, tldr everything) I'd be surprised if the iPhone version doesn't record GPS locations or what other apps you use.

I'm also not sure how commonly websites record data in this detail that also maps to specific users, but if it's common on websites its definitely news to me - and all the more reason to use stuff like Ghostery/Disconnect/Noscript. Do these have iPhone equivalents yet?

huhtenberg• 13 years ago

Because website is a service and an installable app is a product.

You expect a coffee shop's camera to track you, but you sure as hell don't expect your moka pot do that. Moreover, if it were doing that, you would've not bought it in the first place.

DenisM7• 13 years ago

I think you're overreacting. Privacy policy disclosure is obviously in order, but that's about all there is to it.

anonyfuss12• 13 years ago

When is the last time you read a privacy policy disclosure?

A reasonable person would assume that if an application has no functionality that requires sending data over the network, the application won't surreptitiously spy on you and send your data over the network.

sjtgraham8• 13 years ago

I don't think this is an invasion of privacy at all. Heap hooks into single apps, it's not like it hooks into the OS to monitor user interaction with everything a user does in every app. I would sympathise with your reaction if this was the case.

I think it's perfectly good enough for a vendor to declare what data they are collecting in their ToS e.g. "we collect information on how you use the app" and why, e.g. "to consider for future versions so that we release the best possible, most useful software for our users". It's absurd to mandate that this be communicated in "large block letters" as this makes it into a bigger deal than it is.

Let's take off the tin foil hats and be pragmatic. I fully expect analytics to collected on everything I use and I understand the motivation behind it too, i.e. it's to improve whatever it is I am using to make it more useful, and therefore valuable to me.

irrationalidiom2• 13 years ago

> I don't think this is an invasion of privacy at all.

I vehemently disagree. Most companies will just hide the fact they are collecting (and often selling) user data within slimy legalese.

Consumer protection includes niceties such as warranties, and protection against false advertising, and clear display of nutritional information and safety warnings.

Privacy also deserves the same treatment so users can make an informed decision, without having to wade through a hefty licence agreement.

Edit: updated reply context

DenisM7• 13 years ago

So here's my quick feedback to the team.

Pros:

- Pricing is fine with me - I'm doing a paid SaaS app with relatively high per-user revenue. You will likely price out most of other devs, however.

- I like the retroactive analysis. The idea of grab everything and let god sort it out speaks to me.

- I like doing both device and web analytics in one place. I don't think doing one or the other makes sense anymore.

Cons:

- I don't see a way to make sure a user from different devices is counted as one user, not two. I want to track the user across all touch points. From experience I know solving this problem is very-very hard - sometimes you want to see what the user did before he identified himself, which is where your idea of "one user per session" falls apart - a session might have different users in it signin in and out, and then there is a period between signins which can be attributed to either of the authenticated users before or after in the same session. Not to mention apps with support for multiple-account apps, (e.g. iOS gmail app where identity depends on which part of the app you're in).

With all that said, I don't think I will be trying it out. There are tons of analytics services out there, and I don't feel like I can make an educated decision about which one to try first, or at all. Where you can help me is by providing a survey of the problem space, talking about the problems, potential solution methods, and then addressing which products implement which methods.

Good luck.

matm8• 13 years ago

Great, thanks for the thorough feedback!

Re: your first con. We offer a heap.identify() API call that associates activity with one canonical identity. For instance, if a user logs in on a new separate device, you can call heap.identify({handle: 'DenisM'}) to associate all activity on that device with the existing user DenisM.

We hope you reconsider and give us a go. We're always down to talk more in person and better understand your use cases. Just ping us at team@heapanalytics.com.

Heap for iOS: the capture-everything approach to mobile analytics

Related Stories