Neural network spotted deep inside Samsung's Galaxy S7 silicon brain

http://www.theregister.co.uk/2016/08/22/samsung_m1_core/

by dragonbonheur • 10 years ago

269 99 10 years ago

theregister.co.uk

dingo_bat3• 10 years ago

This is literally insane. I've never heard of a neural net branch predictor.

Edit: my layman knowledge about branch prediction and neural networks is showing its gross inadequacy :/

nilved• 10 years ago

e: was wrong about everything

xlayn• 10 years ago

If you have to pull an army of cores to fight two the really impressive arch belongs to Apple.

OTOH I see this as the optimization of hardware to fix the cap that software, paradigms and devs mean.

dingo_bat3• 10 years ago

I guess it's naive to assume that Apple isn't also doing the same thing. And apple's cores are bigger so they generally do more work per core per cycle. The exynos in GS7 has eight cores. But the die area is roughly same as apple's dual core chip.

DannyBee2• 10 years ago

It's actually really common these days. :)

Search google for "hashed perceptron branch predictor"

mbell• 10 years ago

Intel (and probably AMD) have been using neural based approaches to branch prediction since the early 2000's. This isn't particularly new, it's just not a topic talked about much.

gpderetta4• 10 years ago

[citation needed]. As far as I know, Intel is suspected to be using TAGE or some variant of it in its last few generations of chips.

gcr• 10 years ago

Branch prediction strategies can be learned, just like any other model. I could certainly see something like this outperforming the huge bag of hand-tuned optimization strategies for branch prediction anyway. It could save some silicon.

Even a simple dot product can be called a "neural network," albeit a small uninteresting one. Your features could be (say) the state of cache, the number of jumps, the number of recent stalls / stack size / returns, and so on. Put them into a 10-dimensional vector or whatever, take the dot product between that and a set of 10 learned weights, and preload the branch if the result is above some threshold. Even a simple model could perhaps work really well here.

Of course it's not like they have some deep learning convolutional net running on the program input. I also bet the weights are set at the factory so it's not "learning" anything.

medains• 10 years ago

Nothing in the article suggests that it's learning - so I suspect that the weights are indeed set in the factory.

Learning on the fly would consume much more power as it back-propagates, so doesn't really make much sense as the potential performance gain would be marginal and would probably cause a performance hit when switching to a different process or thread.

corysama• 10 years ago

The article indicates that academia has been writing about using them for branch prediction for a while and than Intel and AMD have been doing just that for a while. They just don't talk about it much.

AMD's Zen architect Mike Clark confirmed to us his microarchitecture uses a hashed perceptron system in its branch prediction. "Maybe I should have called it a neural net," he added.

Neural networks can be very complicated, but they can also be very simple. What little I know about perceptrons is that they are very simple. If you google "hashed perceptron", many of the papers that come up mention branch prediction in their titles.

verandaguy• 10 years ago

Not necessarily. The cost of backtracking on a bad branch prediction on a device like a phone may be so great that implementing an ANN could be worth the effort to implement.

I'm not as familiar with mobile phone architecture as with that of PCs, but the number and types of operations influenced by the device's various sensors (light sensor, GPS, accelerometer, gyroscope) could conceivably be more than normal, naïve branch prediction can handle effectively.

gpderetta4• 10 years ago

Why would speculation invalidation be more expensive on a phone? If anything, many phone CPUs tend to have shorter pipelines and less expensive branch prediction misses.

Also I do not see how the number or quality of sensors could in any way affect the prediction rate.

flukus2• 10 years ago

First time I've heard of them in hardware. The JVM has been employing similar feats for many years now.

zigzigzag5• 10 years ago

Like where? Even the best JVMs do basic counted profiling of branches only, as far as I know.

hcs• 10 years ago

My Computer Architecture professor Daniel Jimenez worked on (invented?) something like this:

"Dynamic Branch Prediction with Perceptrons" (PDF)

http://hpca23.cse.tamu.edu/taco/pdfs/hpca7_dist.pdf

akash_m• 10 years ago

I read the paper description and immediately jumped to conclusion that you are Calvin's student :) We had to study this paper during one of my classes at UTCS

DannyBee2• 10 years ago

Calvin Lin is also no slouch :)

strait• 10 years ago

Incidentally, Daniel was my very first CS instructor about 20 years ago, when he taught my Intro to CS class at UT San Antonio. It was the vi editor and K&R book C programming on some ancient green monochrome terminals back then. I remember it as a fairly severe introduction, but he was always great at explaining things properly.

goombastic• 10 years ago

Why is it still slower than oneplus3 and htc 10?

Grazester3• 10 years ago

Samsung bloat?

mrfusion• 10 years ago

Hmm now I'm wondering if the brain does something like this? Might the brain need branch prediction? Interesting line of thought.

sbashyal• 10 years ago

I have a background in Machine Learning and it is my belief that the mind does use branch prediction. Take for example a scenario where you are climbing down a stair in your house. Your brain already predicts that you are about to land on the next step. Now if the step height was lowered than the usual, the prediction fails and you immediately focus back on the climbing-down process to see what has happened and prosecute whatever fall-back actions are needed to minimize disruption.

randomacct44• 10 years ago

Reminds me of what happens when you step onto an escalator that isn't working (the broken escalator phenomenon):

https://en.wikipedia.org/wiki/Broken_escalator_phenomenon

argonaut• 10 years ago

Nitpick: having a background in machine learning is only slightly relevant to belief on what the mind does. It's having a background in neuroscience that's important.

Sharlin• 10 years ago

An analogy is basically every time you subconsciously predict something, but something else happens. You get surprised and confused which is the brain's analog of flushing the pipeline and calibrating the predictor. Indeed, the execution of most mundane tasks only bubbles up to the conscious level when something unexpected happens and you need to update your model of the world.

Disclaimer: as I said, this is an analogy. Brains are not literally executing instructions in a pipeline.

Jerry2• 10 years ago

Even with all those neural net advantages, dual-core, 2GB RAM iPhone 6s beats octa-core, 4GB RAM Galaxy Note 7:

http://www.redmondpie.com/galaxy-note-7-vs-iphone-6s-real-wo...

Video (3:29 sec): https://www.youtube.com/watch?v=3-61FFoJFy0

PS: I highly recommend you check out the video... iPhone completely obliterated Note 7.

the_mitsuhiko• 10 years ago

That's not too surprising. There is a lot more that runs on an Android phone usually.

matt_wulfeck• 10 years ago

What more is running on an android phone then other OS? Seems like they have some fat to trim if there's more required to run their phone.

digi_owl• 10 years ago

Yeah my first thought went to how Android will background anything when you hit center button, while iOS will basically give it a few seconds to pack up state and then evict it. Been that way since day one, and has very little to do with the hardware.

coldtea3• 10 years ago

If you mean bloated Java runtimes, then we agree.

stephengillie• 10 years ago

It should be noted most Android phones use a "Dual-Quad" design, with a bank of 4x A53 for low-power operation, and a dual or quad set of more powerful processors (A57, M1, etc) for full-power operation.

Still, it's interesting to see the single A9 with coprocessor outperform dual M1s - like was said about the reduced core count in the Snapdragon 820:

“The most important thing to have is peak single-threaded performance, as most of the time only one or two cores are active." [0]

This is apparently true, with the iPhone outperforming despite its A9 running up to 400mhz slower than the Android cohort's Snapdragon.

[0] http://trustedreviews.com/opinions/snapdragon-820-vs-snapdra...

viraptor3• 10 years ago

That test is relevant for the end user, but it doesn't make sense as a hardware test. It's a different system, on different hardware, with different apps (that happen to share the same name), and essentially what you're comparing is resource loading strategies. iPhone is better at that - great.

But for a hardware test, I'd expect a single running app, executing shared native codebase, in no disruptions airplane mode, measuring specific part of the hardware. And that's not even close to measuring before/after improved branch prediction, which could be useful in both phones.

swinglock• 10 years ago

Why measure and therefore optimize for anything but how the product is being used?

BinaryIdiot3• 10 years ago

But for a hardware test, I'd expect a single running app, executing shared native codebase, in no disruptions airplane mode, measuring specific part of the hardware. And that's not even close to measuring before/after improved branch prediction, which could be useful in both phones.

I would argue that would be a completely useless test unless you're explicitly testing something for development purposes only. Users use phones. Nothing else matters but the user experience. If you can make your phone beat another phone in your limited, no disruptions, network traffic off example that doesn't mean anything.

I feel this is a fair test. You test the phones at what they're supposed to be able to do. If one does it better it's legit to point that out regardless of what the underlying structure looks like.

ddebernardy• 10 years ago

That test is relevant for the end user, but it doesn't make sense as a hardware test.

Err? It seems to me that the only tests that make any sense are those that count for end users. Everything else is pointless. It doesn't matter if it's the same system or code base or whatever other engineering fetish.

Users who compare their phones to see which goes faster look at how the same app opens and runs on two separate phones. Which incidentally is what this test is about. It's the only sensible test you can do.

coldtea3• 10 years ago

That test is relevant for the end user, but it doesn't make sense as a hardware test. It's a different system, on different hardware, with different apps (that happen to share the same name), and essentially what you're comparing is resource loading strategies. iPhone is better at that - great.

Anything besides the concrete end-user experience of a system doesn't matter at all.

(In theory it could matter for cpu/compiler designers etc -- but even what matters to them is inconsequential if in the end of the line it doesn't matter to the end-user).

kbart2• 10 years ago

"Even with all those neural net advantages"

On the other hand, without a neural net, it might be even slower. Android supports tons of different architectures and platforms, of course you can't optimize it like Apple can with a single platform.

BinaryIdiot3• 10 years ago

Android supports tons of different architectures and platforms, of course you can't optimize it like Apple can with a single platform.

Not really. Sure Android can run on multiple architectures but 99% of the time it's mostly the same. Why couldn't they provide optimizations for multiple architectures anyway? Whenever something can run on more than 1 architecture I usually see the argument about how it can't be as optimized on any specific architecture but I don't see why not. Optimize for the 80% use case and you're probably covered. Go beyond that to get more performance out of other platforms but it's unlikely necessary.

veeti• 10 years ago

This is because of Samsung's garbage software. Years and years go by and every single time they manage to make the best hardware and slap a laggy and bloated version of Android on top of it. I'm sure that a Note 7 with a more Nexus-like build of Android is a dream for many.

wutbrodo• 10 years ago

They drastically pared down the software differences in the last version or two. Me from five years ago would be shocked to hear me say it, but it's to the point that with the s6, I preferred the new stripped down TouchWiz by a long shot to stock android (TouchWiz has always had design advantages that stock android sometimes incorporates, e.g. Quick settings in the notification bar).

kayoone• 10 years ago

While i prefer stock Android, the Note arguably benefits a lot from a different software because it's tailored to the Stylus and round edges. Other than that they already toned it down a lot to where it was a few years ago, so i don't think the impact is that huge. I really love the Note 7 for the possibilities that the stylus brings, so the Samsung software is something i'd have to live with.

tribaal• 10 years ago

I've been a happy Samsung customer for the longest time, because I basically flashed their nice hardware with cyanogenmod.

My recently purchase S7 is the first ever cellphone I return to the store because: 1) It's way too hard to flash their exynos based hardware and 2) the default install comes with 14Gb of garbage included. That includes the full microsoft office and a slew of garbage apps I'll never use (I'd include the launcher in that, but I understand some people like it, so I'll it a pass)

Annecdata, but I have now become this guy who will tirelessly slap Samsung's garbage software "strategy" every time I have the opportunity.

zigzigzag5• 10 years ago

It'd be really interesting to see a performance analysis of what's going on there. Is it just a weaker CPU or are the apps less optimised as well? The two phones were comparable right up to the time they loaded a game and it just fell behind further and further from that point on. Why is loading that particular game so much slower? The developers didn't care to optimise it?

edit: The article points out that the resolution of the Samsung display is a lot higher, so it's not quite comparing apples and oranges here. I'm guessing that a much higher resolution is pure win when drawing mostly vector graphics i.e. apps but really kills you when loading games as presumably they're paging in much larger assets from disk.

simonh• 10 years ago

I doubt the assets are different, that would require the developer to include different assets for various different Android screen resolutions for different phones in the Android version of the app. Rather it's the dynamic drawing engine that will be doing more work generating the screen images at the higher fidelity because that will work at the screen's physical resolution.

brigade• 10 years ago

Well for starters, Anandtech measured the NAND performance of the 6s as significantly faster than the Note7; 1.65x faster sequential 256k and 1.15x faster random 4K.

andrepd• 10 years ago

Oh god it's one of those app opening videos isn't it?

Yep, it is. D: Well here is a tip that should be somewhat obvious: unscientific measurements of which phone loads apps faster when you click from the home screen isn't any basis to claim a phone "beats" the other, especially when we were talking about processor performance.

By the way, the iPhone beats the S7 on single core and the S7 wins on multi core performance. Which makes sense seeing as you are competing a dual core with an octa core.

gpderetta4• 10 years ago

Apple A9 is a quite sophisticate CPU, there is no reason to believe is not using a state of the art predictor. The Samsung CPU might not have any advantage at all on this area.

andrewpi• 10 years ago

US versions of the Note 7 use a Qualcomm processor, not a Exynos. (supposedly to support Verizon and Sprint's CDMA networks)

Grazester3• 10 years ago

Yeah the Note 7 is documented as being full of bloat effecting its performance. http://www.xda-developers.com/with-the-note-7-samsung-still-...

sehugg• 10 years ago

I don't see why branching is so hard -- it's two cycles to test the branch, one more if the branch is taken, one more if it crosses a page boundary! /6502joke

pawadu8• 10 years ago

well... for the size of its pipeline (1?) and the amount of speculative execution going on (0) the non-existing branch prediction of 6502 was basically optimal.

Someone• 10 years ago

The 6502 has a tiny bit of pipeline/speculative execution. The CPU fetches the next instruction before the current one has finished.

http://www.atarihq.com/danb/files/64doc.txt:

"If an instruction does not store data in memory on its last cycle, the processor can fetch the opcode of the next instruction while executing the last cycle."

The 6502 also sometimes does excess reads and writes. For example, "Read-modify-write instructions (like INC) read the original data, write it back, and then write the modified data." (http://forum.6502.org/viewtopic.php?t=2).

One could call that speculative execution, where the processor writes back the value, just in case adding one to it doesn't increase it, and then, having discovered that adding one changes the value, writes the right value :-)

ForFreedom• 10 years ago

iPhone excel because the OS and HW is designed by one company unlike Samsung where the HW and OS are from different companies, add to android Samsung bloatware.

The iOS works well with the iPhone simply because it is fine tuned for the product. On the androids side the OS is tuned to add bloatware of the respective HW manufactures. To add the so called speed the HW manufactures put in more RAM.

gambiting• 10 years ago

You do realise that samsung designs and makes most of Apple stuff, right?

eknkc• 10 years ago

What parts do Apple get exclusively from Samsung and depend on their expertise these days?

It seems like they go for redundancy, use two suppliers etc for common parts and chips. Which means these are built based on the designs of Apple. Samsung just manufactures them.

I don't follow closely though so I might be wrong.

AstralStorm2• 10 years ago

This is not new, AMD has been using perceptron-based predictors probably even before Bulldozer. They are good, but harder to optimise for than standard rule based heuristic ones.

sitkack• 10 years ago

They should expose the NN coefficients to the end user so that one can at least start from a known-better value. This problem would seem analogous to JIT warmup.

pepijndevos• 10 years ago

The latter half of the article makes me wonder to what extend the cycle count of instructions is specified, or up to a specific implementation.

Neural network spotted deep inside Samsung's Galaxy S7 silicon brain

Related Stories