Quaternion Knowledge Graph Embeddings (2019)

https://arxiv.org/abs/1904.10281

by teleforce • 2 years ago

100 41 2 years ago

palad1n• 2 years ago

Is anyone using this? It's from 2019.

cess112• 2 years ago

If someone is it's probably pretty niche. RDF is a bit simpler and still not commonly used due to its perceived complexity compared to RDBMS tables or JSON.

nl2• 2 years ago

This completely misunderstands what this is.

In this context "knowledge graph" means a representation of knowledge independent of it's serialization format (ie, RDF vs whatever).

It's usable wherever you have a key value (key:value of relationship) or a triple (key:relationship:something).

This is probably 85% of everything that is stored in a database, with the exception being paths that vary enough that you can't use the path as a key or relationship. In particular, most trees are suitable if you use collapse the path down to a single relationship.

I've done a bunch of work on graph embedding. They are very effective for use in anything that can be thought of as a recommendation system ("this person would like these books") or similarity ("this person is similar to these people").

Back when I was working on them I found Starspace wonderfully easy and effective: https://github.com/facebookresearch/StarSpace

VHRanger2• 2 years ago

These exotic (non linear algebra based) embedding representations are often slow to take off unless they have an obvious use case.

The other one that I've always been curious of is Poincarré Embeddings [1] - where the embedding also has a hierarchical representation of the space.

There's issues with these becoming popular:

1. Querying the embeddings requires more math knowledge than just "lol cosine similarity". This also requires to write code for the query

2. You can often easily match the performance with regular embeddings by just adding dimensions and training more. So the advantage of exotic embeddings has to be with the information in the more complex mathematical abstraction.

So they need a killer usecase to become popular, it's hard to move the needle.

[1] https://arxiv.org/abs/1705.08039

nico• 2 years ago

Super interesting

Do you think it makes sense to have a group of models, each with more ad-hoc embeddings, and coordinate them to respond according to the domain of the input?

Do multi-modal models use the same embedding type/structure for an image, sound, text?

adultSwim2• 2 years ago

The paper was cited over 500 times, https://scholar.google.com/scholar?cites=1505183897130217150...

adultSwim2• 2 years ago

QuatE is implemented in PyKEEN, a library for KnowEdge graph EmbeddiNgs, https://github.com/pykeen/pykeen

ziofill• 2 years ago

I always wondered why bother with quaternions and abstruse math when at the end of the day they behave like some matrices of floats.

hi-v-rocknroll3• 2 years ago

For 3D graphics, it makes rotation and translation simpler by adding another dimension. Maybe you had a less-than-stellar teacher for this topic. If so, here's a master lecturer elucidating it:

https://youtu.be/mHVwd8gYLnI

https://en.wikipedia.org/wiki/Transformation_matrix#Examples...

klyrs• 2 years ago

To be fair, some of us learned about quaternions over finite fields before anything about 3D graphics. The first time I heard about this application of them, I was gobsmacked. Saying this because my professor for that course was quite excellent.

globalnode• 2 years ago

hurts me how he keeps saying over and over that they were invented by someone named Heisenberg.

godelski• 2 years ago

The problem is most people were taught math wrong and this typically only gets corrected when you get to high level mathematics like abstract algebra.

Imaginary numbers aren't "imaginary" they are how to do consistent math in a 2D framework. Quaternions? Well that's about 4D. Poincaré once said that math isn't about numbers, but relationships between numbers. For some reason we don't talk about mathematical structure (explicitly) until the late stages. I wouldn't necessarily call these concepts "abstruse" but they are a bit more abstract. A big part of the problem though, we often ignore the ground we are building upon and so when you finally look at it, it is new and confusing. But then again, the success of Bourbaki's New Math is arguable[0]

[0] https://en.wikipedia.org/wiki/New_Math#In_other_countries

auraai• 2 years ago

Poincaré once said that math isn't about numbers, but relationships between numbers

He described math more poetically and profoundly: "the art of giving the same name to different things"

aap_• 2 years ago

Quaternions? Well that's about 4D

Quaternions are about 3D. You need 4 numbers for it, but the space that they operate on is 3D.

truckerbill• 2 years ago

'Why bother with programming languages when at the end of the day they are just pages of text'

TeMPOraL2• 2 years ago

Or even why bother with computer science and engineering, when at the end of the day, it's all just about counting very fast.

pinkmuffinere• 2 years ago

As a concrete example, feedback controllers for attitude control (ie, pointing something) are imo easier to develop in quarternions than with 3x3 matrices. For one, the quaternion formulation isn’t over-constrained by the extra parameters.

This is essentially because quaternions are a remarkably good representation of rotations.

TeMPOraL2• 2 years ago

Related, 3D math is done with 4x4 matrices, not 3x3 as the "3D" would make someone think, because you need that extra dimension to represent translations in matrix form, and once you do that, you can compose arbitrary transforms through multiplication.

authorfly• 2 years ago

Because they enable you to involve things typically unrelated (i.e. two types of embeddings), think and understand and manipulate them in a simpler mathematical model (a single vector) and see how you can then apply the most fundamental mathematical operations (e.g. cosine similarity) in terms of simpler outputs/metrics. It enables a form of creativity in other words; it makes you see things like the math in Transformers in a less anxious light as you understand how these things can overlay into a simpler intuitive understanding.

redwood• 2 years ago

Sounds similar to the power of LoRa; critical building blocks to building efficient scalable serving

barfbagginus2• 2 years ago

I recently took the plunge and learned unit quaternions and dual quaternions for representing rotations, compound rotation-translations, and kinematic chains and geometric interpolation (serial and parallel robot arms, character skeletons, skinning).

The advantages I found are:

Unit quaternions represent rotations with less redundancy than matrices

It's easier and more intuitive to derive, manipulate, simplify, interpolate, and solve quaternions than matrices.

They're less abstract/more concrete than matrices.

Rotating with quaternions takes fewer multiplications than matrices.

Dual quaternion inverse kinematics are easier to derive and faster than matrices

Unit and Dual quaternions have an efficient implicit form, which further speeds up IK. See: https://www.researchgate.net/profile/Neil-Dantam/publication...

captaincaveman• 2 years ago

https://en.wikipedia.org/wiki/Gimbal_lock

somenameforme• 2 years ago

One of the big motivations for quaternions is performance. Not only are they much more compact than a rotation matrix (4 float vs 9), but many important operations, such as composing rotations, are also much quicker. The only real downside of them is that they seem to drive everybody to this obsession to try to truly understand them. After years of using them, I've learned to embrace my ignorance. The underlying math is not terribly complex (hur hur), but relationships between interdependent periodic rotations all packed in a tidy imaginary number package, is simply not something you're going to really get a good intuitive feel for - numerically speaking.

ubj• 2 years ago

The only real downside of them is that they seem to drive everybody to this obsession to try to truly understand them.

This. If you only want to understand *how to apply* quaternions to represent rotations, it's not too hard to learn. However, if you want to understand *why* they work, that's a totally different story.

Many of my colleagues still use Euler angles to represent rotations. I always use quaternions, and am a bit baffled why people are so averse to them.

GuB-42• 2 years ago

If you are still interested, try this: https://marctenbosch.com/quaternions/

The idea is that quaternions are the wrong way to look at the problem and that a better approach would be to use geometric algebra, bivectors and rotors. The formulas are essentially the same as with quaternions, but at least for me, this approach make more intuitive sense. It also work in dimensions other than 3, which matter to the author as he is the author of "4D toys" and hopefully, Miegakure.

Here is another talk on the subject, by a different author: https://www.youtube.com/watch?v=htYh-Tq7ZBI It is not specifically about quaternions, it is about multiplying vectors and what you get from that, and it includes quaternions.

meindnoch• 2 years ago

Rotation matrices (i.e. orthonormal matrices) are a complicated manifold in 9 dimensional space.

Rotation quaternions (i.e. unit quaternions) are simply the unit sphere in 4 dimensional space.

pizza• 2 years ago

Simplicity depends on the observer

vslira• 2 years ago

I'm not nearly smart enough to state this confidently, but doesn't [1] imply that exotic embeddings can always be replaced by larger Euclidean embeddings?

[1] https://en.wikipedia.org/wiki/Nash_embedding_theorems

hansvm• 2 years ago

No.

Consider a square (with graph shortest path distances) for example. All euclidean embeddings have a minimum error of 20-40% or so. If you try to embed any graph containing that subgraph (embedding friends, advertisers, words, ...), you'll similarly have guaranteed error. Relaxing the square to a squircle, you'll see that error even for a really simple manifold.

Riemannian manifolds are a bit more special.

Quaternion Knowledge Graph Embeddings (2019)

Related Stories