Measuring the Entropy of the MOS 6502 CPU

http://www.pagetable.com/?p=401

by daniel02216 • 16 years ago

19 2 16 years ago

ced• 16 years ago

There's a bit of a conceptual problem with trying to measure entropy. That's because entropy is based on probabilities, and probabilities are subjective. What are "the" odds that my brother will pass his SAT exam? My brother can make the most confident prediction (lowest entropy[1]). But the reader of this comment can do no better [2] than take the generic SAT success rate (higher entropy). And some peasant in Tuvalu might as well say 1/2 (maximum entropy).

So entropy is subjective. It's a measure of how much new information you expect to get from a message. The more you know already, the lower the entropy.

EDIT: An awesome quote from Jaynes.

"Probability assignments are not assertions of any physical property; they are a description of a state of knowledge. It is therefore illogical to speak of "verifying" the probabilities by performing experiments on the physical world. That would be like trying to verify a boy's love for his dog by performing experiments on the dog"

[1] Not necessarily true, see Wiki or David MacKay's wonderful book for details. http://www.inference.phy.cam.ac.uk/mackay/itila/

[2] Also not strictly true! You might argue that his passing the SAT should correlate with my passing it, and estimate what my odds would be/were from my comment history.

phaedrus• 16 years ago

I wondered how it would compare if you measure this from the gate level rather than the transistor level. To check this, I looked online for a Verilog file describing a 6502 core. This site has one:

http://www.birdcomputer.ca/Projects/Prj6502/bc6502_page.html

The zipped size of that file is 38 kilobytes, which is actually larger than the approximately 7 kilobytes guessed in the article for transistor level information. That is probably due to the verilog code being more verbose; you would think zipping it would make up the balance but apparently not. However, even though 38K >> 7K, I think the fact that the two are at least in the same ballpark is encouraging.

Measuring the Entropy of the MOS 6502 CPU

Related Stories