Privacy Loss in Apple’s Implementation of Differential Privacy (2017)

by ChrisCinelli • 7 years ago

59 8 7 years ago

To me it comes to:

1) "The privacy loss permitted by the system is not explained anywhere and takes significant effort to reverse-engineer. This is contrary to one of the main conceptual advantages of differential privacy – that a user can make an informed choice whether to opt-in to differentially private data collection based on the quantifiable knowledge of risk announced by the data collector."

2) "Furthermore, the lack of transparency on privacy loss opens the door for intentional or un-intentional abuse by Apple itself, e.g., by unilaterally changing either the per-datum privacy loss or the rate of privacy loss in a time period or by introducing additional BudgetKeyName(s), Apple may significantly weaken the privacy guarantees provided without anyone’s knowledge or consent."

Apple's "closed source and no documentation everything" does not help.

ChrisCinelli3• 7 years ago

Over time I saw a few examples of the attitude "We are Apple, these are our machines, there are our algorithms, we are smart, we do things right for you." but considering their track of records exposed by the problems routinely discovered by who reverse engineers their product, I wonder how useful that attitude really is.

londons_explore2• 7 years ago

Both apples security and privacy I'd say are industry leading...

Sure, you might still say it isn't good enough, but I personally trust them to make the right choices for me, even if they don't inform me exactly what those choices were and what tradeoffs were made.

nasut2• 7 years ago

This is an outdated document. Apple has a detailed whitepaper on the topic as well as the fact that you can see eqch submission and its parameters in the device logs.

https://machinelearning.apple.com/docs/learning-with-privacy...

ChrisCinelli3• 7 years ago

I do not see the date on this document you linked. But all the links point to 2016 articles or older. Are you sure that it is more updated?

I hope that they fixed the problem since this article came out but I am not so sure they are more open how they implement differential privacy.

nasut2• 7 years ago

The abriged version containing the link to the whitepaper was published in Dec 2017

https://machinelearning.apple.com/2017/12/06/learning-with-p...

Privacy Loss in Apple’s Implementation of Differential Privacy (2017)

Related Stories