Show HN: 128-bit, roughly-ordered, URL-safe UUIDs

https://github.com/anthonynsimon/timeflake

by amzans • 6 years ago

205 137 6 years ago

Query : For people who recommend UUIDs as a unique identifier in a distributed sytem, does the above satisfy the need?

In the SO answer[0] uuid v4 is being recommended but that is the most random version of uuid. Isn't that going to hurt the b-tree performance by having a very high fanout? How does it help the primary index?

[0] https://stackoverflow.com/questions/33274291/postgresql-uuid...

EDIT : Apologies. I meant low fanout. What I meant was that you need something ordered to have a better organised b-tree node. If all values are random, each node will have a lot of children, increasing the search space. It might turn into Linked list in the worst case scenario.

tomsmeding2• 6 years ago

Excuse my possible ignorance, but how is more entropy in a primary key going to hurt index performance? The fanout in a b-tree is completely in control of the database, and not dependent at all on properties of the data — apart, of course, from the size of the data, since typically the node fanout is precisely the number of fields that fit in one disk page. Randomness does not influence the size of a UUID on disk.

rapidlua• 6 years ago

With random keys updates are going to touch way more pages than with sequential ones. It increases IO due to more dirty buffers and journaling.

mytailorisrich3• 6 years ago

Isn't that going to hurt the b-tree performance by having a very high fanout?

High fanout helps minimise the depth of the tree and therefore helps performance as it minimises the number of nodes to check.

abhishekjha3• 6 years ago

Sorry, I meant low fan out, as in a b-tree node will end up have having a lot of singular valued children ultimately degrading to linked list.

StreamBright4• 6 years ago

We moved away from UUIDs for different reasons. It is hard to work with UUIDs when multiple entities have them. You do not know if you are looking at a user or a group if both is just a UUID. We created our own IDs with prefixes and using base58 encoding for the random part, for example: user-AhnPpn7CbbwPykmux11LJ39Jk. It has the same amount of randomness in it and we can understand what we are looking at.

DJBunnies• 6 years ago

Uuid v5 is namespaced.

globular-toast3• 6 years ago

Why would you ever be in a situation where you have a UUID but don't know what it refers to?

SigmundA9• 6 years ago

looks like its almost twice the size of a UUID at 30 bytes in this case with a short prefix, I'm assuming your storing this as a ascii string? That's a lot of extra bytes from some convenience when viewing logs or raw data etc.

UUID's stored properly in binary form are only 16 bytes which can still be a concern over an 8 or 4 byte integer when you have a lot of them all over the place in a large db. (larger index more memory use).

sltkr2• 6 years ago

This isn't strictly speaking a UUID. It's just a 128-bit identifier. It's "compatible with UUID" only in the sense that both are 128 bits wide, but this scheme doesn't have the structure of a UUID.

To summarize the scheme, a 128-bit id is generated by concatenating:

  - 48 bits timestamp (milliseconds since UNIX epoch)
  - 80 bits of random data

These ids have the property that they're ordered by generation time (ids generated at the same millisecond are ordered randomly). The result can then be encoded in base 16 or base 62 to make it URL-compatible.

By the way, encoding timestamps into an id is not always desirable. It has some privacy implications, since now you can tell from the id when the resource was created. If this corresponds with user activity (e.g. a user filling out a form or uploading an image) having this information encoded in a URL is possibly undesirable.

skissane3• 6 years ago

You are talking about a v1 UUID.

A random number is a valid v4 UUID, provided you set the version and variant bits to the correct value. A random 128-bit number won’t have those bits correctly set (except by chance). Set them, and you’ve made a valid v4 UUID

sltkr2• 6 years ago

As you say, you need to set certain bits to special values to encode a v4 UUID, leaving less than 128 bits for the payload, which means a random 128 bit number is not a valid UUID except by chance.

Second, a v4 UUID should set all remaining bits randomly. Encoding a timestamp in some of the bits is not random. Sure, you could argue these could have been randomly generated by chance, so any such UUID is not invalid, strictly speaking. Still, a collection of these v4 UUIDs don't have the properties that you would expect of random UUIDs.

As specificed, the scheme satisfies neither of these two requirements.

liversage• 6 years ago

Exactly. And the v4 UUID will have 122 bits of randomness.

sudhirj4• 6 years ago

There’s a spec called ULID that’s pretty much this with default base32 encoding

https://github.com/ulid/spec

I’ve also worked on a UUID-ULID bridge for Go

https://github.com/sudhirj/uulid.go

And seeing as this is just 128 bits it’s quite easy to move seamlessly between formats and representations.

I’ve found this concept especially useful in nosql stores like DynamoDB, where using a ULID primary key makes objects time sortable automatically. It’s also quite easy to query for items by zeroing out the random component and setting only the time stamp bytes.

mfollert• 6 years ago

This is super useful and exactly what I need for a current project. Thanks you for that hint!

mgkimsal2• 6 years ago

I've been using ULID on a couple projects. In one case, we're trying to convert an existing project to use them, and... it's a bit of a pain, but it's not ULID's fault as much as just... changing primary keys all over the place is time consuming and not simple.

The biggest pushback (and it's not a terribly big one, imo) I've read on ULID is that the time portion is sold as 'sortable' but is 'only ms-level resolution'. Apparently the critics I was initially reading on ULID needed sub millisecond accuracy, and somehow thought this made ULIDs bad. Seemed a non-useful criticism, as needing that level of time granularity isn't a day to day issue for most people, but might be something to be aware of if the idea of overloading one field with multiple uses is a requirement you have.

techdragon2• 6 years ago

If you’re an edge case you’re an edge case. It’s frustrating when people start assuming their use case is normal without any considerations of the wider world.

Sub millisecond precision sorting of high speed database inserts is hardly a normal requirement. At that point even the size of your transaction can matter (depending on the database) and you have to start asking questions like do I want ordering by the time the transaction starts or the time it ends? And if your loading existing data from other sources and care about date ordering you should just be using the a dedicated column with the database appropriate time/date time/time stamp field anyway.

spiffytech• 6 years ago

I adopted ULID for a project after seeing it recently mentioned in another HN comments thread[0] and it's very useful. The project uses Azure Table Storage, which encourages querying by ranges of record IDs for efficiency, instead of filtering datasets by arbitrary record properties.

My project revolves around processing windows of time in timestamp-ordered datasets, and ULID works well as a way to generate record IDs in a distributed system that naturally sort in timestamp order. With ULID, all of my major queries can operate on ranges of record IDs, as Azure Table Storage encourages.

[0] https://news.ycombinator.com/item?id=25650907

j-pb8• 6 years ago

ULID has a serious security flaw though. In order to try to improve performance / increase prefix similarity, there is a mode where it reuses the random value, and just increments it, for each new ULID.

With purely random UUID, you can be pretty sure that nobody can guess them, so they're essentially identifier and access token in one.

However once you can easily predict the next uuid from the previous one (as by incrementing the random part by one), the access token property vanishes.

At least this proposal doesn't make the same mistake, however 80 bit's isn't THAT much entropy. (Remember every bit counts with exponential growth, and 48 bits less, means 281474976710656 times easier to guess.)

So it boils down to. Would you be ok to secure all your data with 10 character one time passwords, which might not be guessed all at once, but where guessing SOME is sufficient to get owned?

Death by a thousand paper-cuts.

amzans3• 6 years ago

Thanks for the comment!

Yes, I'd like to clarify Timeflake (and many alternatives suggested) is NOT meant to be used for security.

There's an important note on the readme about this:

Since the timestamp part is predictable, the search space within any given millisecond is 2^80 random numbers, which is meant to avoid collisions, not to secure or hide information. You should not be using timeflakes for password-reset tokens, API keys or for anything which is security sensitive. There are better libraries which are meant for this use case (for example, the standard library's secrets module).

sudhirj4• 5 years ago

Which implementation are you referring to? The Go package uses crypto.random and generation blocks if the system can’t provide enough randomness. It’s possible to override this with custom implementations, but either ways it’s no less secure than a UUID unless the implementation is horrible.

The spec doesn’t specify a source of randomness - an implementation that uses https://xkcd.com/221/ will or course not be a very good one.

calpaterson2• 6 years ago

One of the issues this is trying to resolve is that v4 UUIDs wholly random nature makes them less than brilliant in the b-trees that SQL databases use for indices (I am actually not sure how much that really matters in practice for most projects).

However, Postgres has an alternate index implementation that uses a hash function. It is crash-safe now (previously wasn't) but you still can't use it to enforce a unique constraint. If you can use that, I suspect it might resolve the problem and would be easier than switching UUID generation algorithm.

withinboredom2• 5 years ago

It’s a shame v6 uuids didn’t become a thing[1]. It’s the entire reason they were invented.

1: https://datatracker.ietf.org/doc/draft-peabody-dispatch-new-...

caleblloyd• 6 years ago

This approach has become my preferred way of generating IDs for database rows. It is relatively space efficient and doesn't require a round trip to the database to generate an ID like auto increment does.

While working on a C# implementation for MySQL and we found that when the DB uses Little Endian Binary that the GUID must be reversed to store in order:

https://github.com/PomeloFoundation/Pomelo.EntityFrameworkCo...

abhishekjha3• 6 years ago

Does random UUID work as well as lets say an ordered integer id or ordered uuid? I thought the ids should have a definite order(not necessarily sequential) so that a binary search can be performed on the primary index. Similar for b-trees.

masklinn2• 6 years ago

Random uuids are perfectly ordered, there’s no issue looking up a single record by uuid.

The problems are that they’re scattered during reads and writes, and will require updating old pages.

Ellipsis7532• 6 years ago

I'd worry a bit about the chance of collisions.

They correctly state you can generate 50 million of them per millisecond with only a one in a billion chance of a collision (per millisecond).

Unfortunately this means the chance of a collision in a year is 99.999999999998% which isn't great!

(I think they've given too many bits to the timestamp.)

Also, the uuid format it creates isn't a valid uuid which is a pity.

If you want to have "only" a one in a billion chance of a collision a year, you can only generate about 300 a millisecond.

doersino• 6 years ago

Unfortunately this means the chance of a collision in a year is 99.999999999998% which isn't great!

How was this computed? My intuition begs to differ.

jychang• 6 years ago

There are 31,536,000,000 milliseconds in a year. If you do (1-1E-9)^3.1536E10, you get a 2E-14 chance of no collision in a year.

onetoo• 6 years ago

Probability of no collision in a single millisecond: p_1 = 999999999/1000000000

Milliseconds in a year: t = 1000 * 60 * 60 * 24 * 365

Probability of no collision happening for every millisecond of a year: p_2 = p_1^t

Probability of a collision happening for at least one millisecond of a year: p_3 = 1 - p_2

https://www.wolframalpha.com/input/?i=1+-+%28999999999%2F100...

mytailorisrich3• 6 years ago

There are close to 32 billion millisecond in a year. If the chance of a collision happening within a millisecond is 1 in a billion then my intuition is that the probability of a collision happening at least once a year is quite high, indeed.

It's like a dice: You have 1 in 6 chance of getting a 6 every time your roll the dice. Now, if you roll the dice 100 times the chance that you'll get a 6 at least once is then pretty high.

ktpsns• 6 years ago

I guess you assume to build one timeflake per ms for one year. That's about 3e13 timeflakes. Ordinary UUIDs have a collisions at the order of 1e18 hashes (cf. https://en.wikipedia.org/wiki/Universally_unique_identifier#...). So we are "only" 5 orders of magnitude away. For me the collision quality feels the same.

Ellipsis7532• 6 years ago

If you assume one per ms then timeflake and UUIDv1 both have a 0% chance of collision as they encode the timestamp.

You might go above this rate some of the time though and we might as well strive for perfection.

One of the nice things about UUIDs is that you can just merge data together willy-nilly without worrying about collisions - for example, if you gets bought by Google. You can't really do this with timeflake.

zuzun• 6 years ago

Look for a different solution if you need 1 quintillion IDs per year. Noted.

pas2• 6 years ago

You'd run into a bigger problem than collisions a lot sooner (IMHO): time ordering. "ms" precision is very misleading, because keeping clocks really in sync while processing this much data is a real challenge.

Sure, maybe it doesn't matter that one server/emitter and thus a bunch of IDs are out of order, but if this is really just for helping indexes, then a simple 0.1 second precision would make more sense. (Or even simply truncating the timestamp to however many bits to get ~2-30 sec precision, and using a data store with LevelDB-like staged compaction. Which is virtually a requirement at this scale.)

rafaelturk• 6 years ago

This is really nice, anyone aware of a NodeJs alternative?

mgkimsal2• 6 years ago

https://github.com/ulid/spec isn't timeflake, but is a good spec, and there are implementations in multiple languages. there are 2 listed for js.

https://github.com/aarondcohen/id128 and https://github.com/ulid/javascript

audionerd• 6 years ago

@thi.ng/ksuid https://github.com/thi-ng/umbrella/tree/develop/packages/ksu...

sparkling• 6 years ago

Roughly ordered (K-sortable), incremental timestamp in most significant bits enables faster indexing and less fragmentation on database indices (vs UUID v1/v4).

What does this mean for database performance in simple CRUD scenarios?

theta_d• 6 years ago

Using an ordered ID prevents index fragmentation. Inserts will be slower and ultimately so will reads. In a database like MS SQL Server this is especially true since data is physically stored on disk based on the clustering key which is the PK by default.

hankchinaski• 6 years ago

not quite sure why the requirement of being compatible with the UUID 128-bit representation. i personally try to use KSUID (160bit) when possible and can’t see any downside so far

Tostino6• 5 years ago

Being 128-bits allows you to utilize the same database/object types as UUID.

That's the main benefit IMO. I couldn't stick a KSUID into my Postgres UUID column, i'd have to write a new data type to store that.

chrisacky• 6 years ago

Can someone expand on this please.

They should be random, but roughly-ordered over time so that your MySQL/Postgres indices stay fast and efficient as the dataset grows.

I'm currently facing an issue where the chosen uuid format was v4. From my understanding this is the least preferable because it's true random, which causes hell for mysql indices.

What I don't quite udnerstand however, is why is sorted random better if I'm using an index in both cases? I'd like to know what I should be doing and if there is a way to move/migrate if possible

kijin• 6 years ago

In MySQL and some other database systems, primary keys are clustered indexes. A clustered index defines the order in which data is physically stored on disk.

If you have an auto-incrementing primary key like most sane people do, new rows will be sorted to the very end of the table. Inserting data to this table essentially becomes an append-only operation, which is very fast. On the other hand, inserting random values all over the place is like inserting data in the middle of a file and pushing back everything that comes afterward, over and over again. Even on modern SSD's, this is much slower than simply appending data to the end of a file.

For best performance, your primary keys should be as close as possible to auto-incrementing integers. In other words, each new value should be greater than all the values that currently exist in the table. Timestamps are quite useful for this purpose.

pas2• 6 years ago

This is an InnoDB thing for MySQL. (The old MyISAM was a simple append only thing with separate index file.)

masklinn2• 6 years ago

What I don't quite udnerstand however, is why is sorted random better if I'm using an index in both cases?

Data locality. The default index in most RDBMS is a btree which is ordered and clustered so when you look up items in an index if they sort similarly they’ll be in the same index page(s) and thus cheaper to retrieve. For PKs that means when you go through your records it’s cheap to get siblings.

When the ids are randomly distributed it’s way more expensive to add them because they get added at random positions (writes are scattered which is bad) and when you retrieve them you have to jump through the index at random loading a bunch of pages to get just one or two records from it.

Things are even worse if the table itself is clustered after the PK because now your scattered index is also ducking with your actual data reads.

101008• 6 years ago

The indexes advantages would be the same when you think of searching. But for inserting (and deleting) it is much faster if they are already orderded. You don't need (or well, the DB engine) to traverse any trees or anything. If they are 100% ordered, inserting new ones to that index would be just add something at the end of a list, much faster.

thristian• 6 years ago

As a rule of thumb, a table that grows over time probably has a "creation time" field, and a query that touches one row probably touches other rows created around the same time (like a weekly report). Since rows are probably stored on disk in chronological order, you can index the "creation time" field and do a good job, but if your primary key is already roughly chronological you may be able to get away with one fewer index and keep inserts/updates cheaper.

varispeed• 6 years ago

This is assuming that the time is linearly increasing, but what if you have a service running on a fast moving object between planets or in alternative dimension where time is not linear or not monotonic?

solicode2• 6 years ago

I like ordered UUIDs, but if you care about 100% strict and accurate ordering you need to be careful about generating them in app code vs. on the DB side. With the former + using multiple app servers, server clock drift can become an issue. Which is why I still generate my ordered UUIDs on my Postgres server. It means writing some funky PL/pgSQL (or however you decide to do it... I think you can run Python too), but it gets the job done.

I wish ordered UUIDs were built-in to more DBs natively. I remember seeing UUIDv6 being proposed, but I don't know if/when that will be standardised.

httgp• 6 years ago

How does one generate ordered UUIDs right on the PostgreSQL server?

solicode2• 6 years ago

There's probably a nicer way (or more performant at least), but I created a function that uses the built-in `uuid_generate_v1mc` and then reverse the order of the time component so that high values come first.

Tostino6• 5 years ago

Use this: https://github.com/tvondra/sequential-uuids

If you cannot install extensions, I just wrote a PL/PGSQL implementation for the time-based generator I could share.

techdragon2• 6 years ago

The biggest thing I can highlight after having used Standard UUIDs and heavily using ULID values including building my own library and testing compatibility and behavioural compliance of the library with other database variants than the big 4 (MySQL, MS SQL, SQLite and PostgreSQL) ... is that no PK format should ever be considered suitable for all use cases its all about trade offs.

If you care about the order of something you shouldn’t expect any PK to handle that alone, even auto-incremented integers will start to fail you once you have to scale your write workload horizontally. If there’s a property of your data that you want to be able to order it by then you just need to make a column of the appropriate data type and store it. If I had a nickel for every time someone has look at a schema and said something like “Do we really need this date for ordering? The table is already sorted by insert order...” I’d have a lot more nickels than I want.

m45t3r• 6 years ago

Datomic uses a similar approach by default. They called it squuid: https://docs.datomic.com/on-prem/identity.html#squuids

amzans3• 6 years ago

For those interested, here's (IMO) a good explanation on why completely random UUIDs hurt performance, and can lead to database fragmentation issues down the road (on MySQL):

https://www.percona.com/blog/2019/11/22/uuids-are-popular-bu...

It's part of what Timeflake aims to solve, but there's plenty alternatives too depending on your exact requirements.

ForHackernews• 6 years ago

This is not universally true. It depends on the database engine and how its indexes are implemented.

amzans3• 6 years ago

Thanks, I updated the comment to mention that the link is in the context of MySQL's case.

fastball• 5 years ago

Is this applicable to postgres?

zerd• 5 years ago

It's pretty much true for any B-tree based index. You could get around it by using a combined index on another timestamp value, but that's going to be a very large index. Or reduce it by partitioning by e.g. year-month. Even hash-based indices would be slightly impacted due to less locality.

j-pb8• 5 years ago

Have you considered using a 32 bit time value with second resolution?

That way you still get sort-ability, you get 16 bit more entropy, fare better on 32 bit systems, and still get 130 years of usage, out of them.

Also you actively discourage people from trying funny stuff like synchronisation and conflict resolution via the time component. Because 1ms "that sounds kinda good enough :?", while 1s "that'll never work!!!"

Ammendment:

Thinking about it. Using a 16bit timestamp prefix with second resolution and rollover should probably provide a good tradeoff between improving insertion performance (most expensive in terms of index touching), a 112 bit random value, and a dense usage of the timstamp field.

It also allows you to do thing like "forgetting" old values, within a 18h timeframe.

Tostino6• 5 years ago

I personally use: https://github.com/tvondra/sequential-uuids

The time component resolution is configurable, and you can make it wrap around every so often, filling in gaps left by page splits, updates, deletes, etc.

j-pb8• 5 years ago

Nice! This is kinda what I had in mind, while writing that ammendum!

Thanks for sharing :D

Sytten2• 5 years ago

Its just a bit annoying because it is a C extension and that can't be used in most managed DB in the cloud.

dkarl• 5 years ago

Because 1ms "that sounds kinda good enough :?"

Ugh. You wouldn't think so, but I caught a case of this in production, using millisecond timestamps as unique ids for browser analytics. The developer felt that even though two users could in rare cases click at exactly the same millisecond, their clocks weren't likely to be in perfect sync with each other, and that this "additional level" of randomness made a collision virtually impossible.

Quekid52• 5 years ago

Oh dear god.

I suppose it can be counter-intuitive, but people really need to learn about timer precision... and, more importantly, statistics (basics, at least!).

I absolutely hate the 'combinatorics' part of statistics, but distributions, etc. seemed pretty learnable to me... but then I like continuous math, so...

mywittyname2• 5 years ago

I worked on an analytics system where we appended a random 3 digits to the millisecond resolution timestamp for exactly this reason. It turns out, collisions are far, far more common that the math would suggest. Especially back in the aughts when browser had a lot of individual quirks to deal with.

XCSme• 5 years ago

even though two users could in rare cases click at exactly the same millisecond, their clocks weren't likely to be in perfect sync with each other, and that this "additional level" of randomness made a collision virtually impossible

I think that the clocks not being in sync doesn't increase randomness in any way, as two clients who didn't click at the same time might still get the same value.

Bogdanp• 5 years ago

You can also pick your own epoch and use centiseconds since that epoch for the time component. It's what I do in [1].

[1] https://docs.racket-lang.org/buid/index.html?q=buid#%28part....

j-pb8• 5 years ago

Yeah, I think the regular overlfow is actually an advantage.

A.) you pracice having overflows regularly, so the issue is resolved now and not left to some pooor schmuck world war XXVII Survivor in the year 35.000.

B.) It uses every bit, just like a fully random ID, as the key space is filled more densely with each overflow.

advisedwang• 5 years ago

Just be careful if you use these as keys for a distributed system that is susceptible to hotspotting. GCP's Cloud Spanner [1] has this problem and has a good document explaining it.

As always - understand what is going on in your stack and apply design with that in mind. There are no universals!

[1] https://cloud.google.com/spanner/docs/schema-design#primary-...

crescentfresh• 5 years ago

I had to pause when they use the term "anti-pattern". It may be an anti-pattern for cloud spanner but it's not an anti-pattern in other distributed systems, for example cassandra.

zug_zug2• 5 years ago

Right. This spanner thing is new to me, but looking at it, I don't see why spanner doesn't/couldn't shard by modulo / checksum to reduce this risk. And I think other systems do

majormajor• 5 years ago

The very next word after "anti-pattern" is if. "The following table definition that uses a timestamp-based primary key as the first key part is an anti-pattern if the table will see a high rate of insertion:"

Once your data gets big enough there's no easy one-size-fits-all. You gotta look at how you're going to be using the system.

Dylan168073• 5 years ago

Base62? That seems strictly worse than base64 to me. Why that choice?

zamalek• 5 years ago

The two (or three if you retain padding) additional characters are symbols and can cause issues in many places. Base64-url and other variants exist for this reason, but there is no single base64 that works everywhere.

Dylan168073• 5 years ago

There's no "single base62 that works everywhere" either. So while there's valid criticism to be had of base64, I still don't see how this choice is better.

cuspycode• 5 years ago

62 is the number of letters and digits in ASCII, so it's the most compact encoding that doesn't need special punctuation characters (which aren't always safe to use).

Dylan168073• 5 years ago

How many situations don't let you use _ and -? URLs certainly do.

josephernest• 5 years ago

Just to see the big picture, what are the main differences between this and Snowflake, or ULID (https://github.com/ulid/spec)?

mleonhard• 5 years ago

I filed an issue on the project, "Point out the privacy risks of timestamps in object IDs" [0].

----

Non-random object IDs have privacy issues. Developers need to learn about these before they choose to use non-random IDs over simple random IDs. How about adding a PRIVACY section to the readme?

Timeflake encodes the precise time that a user created an object. Timestamps in IDs can reveal:

User timezone.

Geographic location: If the client software creates multiple associated IDs at the same time (like an article and embedded media), then the differences in timestamps of the IDs can reveal the latency of the client's network connection to the server. This reveals user geographic location. This can also happen if the client creates a single ID and the server adds an additional timestamp to the object.

User identity (de-anonymizing)

- Most Android apps include Google's libraries for working with push notifications. And some iOS apps that use Google Cloud services also load the libraries. These Google libraries automatically load Google Analytics [1] which records the names of every screen the users view in the app, and sends them to Google. So Google knows that userN switched from screen "New Post" to screen "Published Post" at time K.

- Some ISPs record and sell user behavior data. For example, SAP knows [2] that userN made a request to appM's API at time K.

- Even if the posting app does not share its user behavior data with third-parties, the user could post and then immediately switch to an app that does share user behavior data. This provides data points like "userN stopped using an app that does not record analytics at time K".

- Operating Systems (Android [3], Windows [4], macOS [5]) send user behavior data to their respective companies.

- Browsers and Browser Extensions send user behavior data to many companies. Data points like "userN visited a URL at example.com at time K" can end up in many databases and sold.

- Posting times combined with traffic analysis can perfectly de-anonymize users.

How long the user took to write the post. This can happen if the app creates the ID when the user starts editing the post and also shares a timestamp of the publication or save time.

Whether or not the user edited the post after posting it. This can happen if the posts's displayed time doesn't match the timestamp in the ID.

Whether or not the user prepared the post in advance and set it to post automatically. If the timestamp is very close to a round numbered time like 21:00:00, it was likely posted automatically. If the posting platform does not provide such functionality, then the user must be using some third-party software or custom software to do it. This information can help de-anonymize the user.

----

[0] https://github.com/anthonynsimon/timeflake/issues/3

[1] https://firebase.google.com/docs/cloud-messaging/android/cli...

[2] https://www.eff.org/deeplinks/2017/03/five-creepy-things-you...

[3] https://digitalcontentnext.org/wp-content/uploads/2018/08/DC...

[4] https://www.eff.org/deeplinks/2016/08/windows-10-microsoft-b...

[5] https://www.eff.org/deeplinks/2020/11/macos-leaks-applicatio...

epakai• 5 years ago

UUIDs in URLs reminds me of dakboard's use of UUIDs for accessing their user customizable displays. There isn't any auth, just guess the right $uuid in dakboard.com/screen/uuid/$uuid and you've got access to whatever info people put on their board (lots of calendars). You can find several (mostly public churches/community/sports) just by googling the base url.

qwerty456127• 5 years ago

IMHO for small projects

where entities are generated in response to a manual action of a user (i.e. relatively rare - the chance 2 users on different machines do it in exactly same moment is very low)

or everything is done in one thread on one machine

a nanosecond/microsecond timestamp makes a practical unique identifier.

Too• 5 years ago

How does this compare to Mongodbs ObjectId?

They have roughly the same structure of timestamp+random but an additional incrementing counter-field at the end, sec-resolution rather than ms, and some drivers use something host-dependent in the random component. 96bit total.

Show HN: 128-bit, roughly-ordered, URL-safe UUIDs

Related Stories