diamondman

I found this attendant console in a thrift store in San Francisco for $4. It is covered with LEDs and buttons, and has a very high build quality. I thought about how hard it would be to make something similar today (with just the cost of LEDs and buttons), and decided to pick it up for a later project.

This repository is the result of my reverse engineering of the ATT26A hardware/firmware and includes:

  * Documentation of the hardware.
  * Documentation of the wire protocol.
  * Python library for driving the panel.
  * Example programs for using the python library.

New examples, patches, and new language bindings welcome.

Happy hacking!

I disagree that it is trivial. If we know what bits of the unencrypted bitstream do what, and have a cryptographically secure way of signing the stream, then GLHF with that. If it turns out that documenting the parts of the chip required for configuration makes the black box security not work, then it was crap security and provided no real security to anyone. As it stands there is no way to tell and we have to take the smiling sales guy's word that it is "SUPER secure, promise".

This is an interesting story. An unfortunate end. Most of these things sound like short term issues that get resolved as the tools are made. Alternately Xilinx could have just put the docs out there and said they only support ISE. That way if an open source developer makes a tool they have no responsibility, but can reconsider in a few years in case it gets wide acceptance. I am not a market strategist, but supporting everyone's custom tool sounds like a quick way to waste money and the wrong way to try and open source something. Similar to respecting every coupon people bring in, even hand written ones.

'get to the level' of ISE... I disagree. I do not want to build an IDE. I want an open toolchain, the thing that the ISE interface calls. There is some math in there that takes certain skills to implement, but I know several people who could do it if they had the layout of the chips. MAYBE the reason there are no open toolchains 'on the level' of ISE and competitors is not because Xilinx has some amazing secret sauce and is instead because the whole industry is caustically secretive and horribly anti competition.

This feels like circular logic to me. We do not want the hardware companies spending time 'open sourcing' their docs because they need to spend that time and money 'making more features' which they wouldn't have to do alone if it was open, but we can't have it open because spending the time and money 'open sourcing' their docs......

I think there is a big difference between 'users wanting tools to be closed source' and 'users not caring if their tools are closed source'. And I would argue that even if the primary user base does not care, we can do better than that. It is no excuse to use these fragile lumbering behemoths of bad design and super seeeeecret tricks that you can learn in a university or the internet. And history keeps showing the problems of big black boxes with the words 'trust us' written on the outside. All we need is the layout and a map of the bitstream and open developers will do most of the work for these companies.

As for spending their resources 'driving prices down', that is quite relative. And if you look at the pricing model of their software it is clearly not their goal to make that reasonable (at least Xilinx). And every version of the software stretches to provide arbitrary bullet points on the back of a box that either mean nothing, were tested in suspicious conditions, or conflate multiple optimized tests together and say 'we are better than everyone at everything. How many times will that be said by all competitors at the same time, and how many times will it be believed? Anyways, with all those features, somehow ISE is still unstable on windows.

When you say 'good, efficient software' i wonder if you are talking about Xilinx ISE/Vivado, or any of the tools from their competitors. I can not imagine you are.

Note that Altera's compiler for Open CL does not let you run your OpenCl stuff faster. It is intended so you can write your FPGA programs (that run outside of your computer usually) in OpenCL. Most of the imagined benefits kind of vanish, at least to me, when you realize that this is just a tool so if all you can hire if OpenCL developers, you can still get the job done.

You have to change out a certain DLL for POST windows 8 in order to get ISE to behave and not crash randomly. YAY! Xilinx said they didn't care but somewhere on their support forum there is some kind soul who said what dll to rename (they have two versions of the DLL in the install anyways but they did not use the win 8 one by default and refuse to patch).

Companies seem to be terrified of trying anything. That is why I was happy when Sony put The Interview up on the internet and theaters for simultaneous release. No one would try that first, and since it worked ok for them it is now an option other companies may consider. Even though both GPU companies and FPGA companies are in hardware, they do not see strategic experiments like opening documentation in the other to be sign of viability for themselves.

Those jobs also pay you to sell your ideas that you are never allowed to use again unless you continue working for the company you sold it to. And if you have the gall to use your own idea, lawyers!

Very well written.

I had no idea some companies went so far as directly making their debugger the software key. That is intense but makes a lot of sense.

In the case of the Xilinx Platform Cable I wrote firmware to, they have a CPLD that does the JTAG writing of data, and a Cypress FX2LP handling the USB information. They use a quad buffered USB endpoint to let the PC load as much crap as possible into USB and in my experience the buffer fills up fast and the jtag work takes the time. Of course this is only because multiple 'pages' were preloaded, otherwise the delay to request more pages would be insane.

Are you talking about the flash driver for loading external SPI flash? If so you likely know this but you have to load the FPGA with a program to load the flash with the program you want the FPGA to load on reboot. Each of these programs have to be compiled and tested per chip. While looking through OpenOCD I found they had drivers for the actual flash, and I am not sure if this is required for the fpga flash boot strapper I just described.

I will look over these links. I recently started messing with ARM chips and they are programmed very differently than FPGAs so I have some stuff to learn on that.

Amusingly the Xilinx license states that you are not allowed to use the bitstreams generated by their software in any device that they did not manufacture. This means if you implemented a xilinx fpga in an altera fpga, it would be against the agreement to program the inner xilinx fpga with the bitstream. The more you know!

OP here

I get there is a big difference between hardware and software, but I feel that certain parts of the gaps are closing. They will not CLOSE, but as they get closer we can learn from each other.

With that said, to address your point: Linux is one of the most dominant systems for servers. Google, Facebook, Twitter, etc use it for almost all of their backend systems. They use it not just because it is free but because it is proven and scalable. Open source databases and web servers also enjoy considerable (sometimes dominant) share. There are companies that provide support contracts for Linux (Suse, Redhat) and that model has worked to fund other open software.

So I think that establishes that a company will use open software.

As for being scary, you are correct that is a primary decision making motivation. It is why many professionals in tons of industries still run windows XP. Often they NEED XP because the software they rely on is so dependent on that specific ecosystem that huge rewrites are necessary in order for it to work on anything else, and this work (even if done by the original distributor) will take years for the tools to be stable again. This is not good engineering on the part of the tool designers. The crippling fear of NEVER TOUCHING ANYTHING is something that the open software alternatives seem to be alleviating in corporate environments since the late 90s.

The lifecycles of putting a phone together is much faster than Intel designing a processor, or Xilinx designing a new FPGA. building boards in phones and laptops is a complex game of Lego and placing them around so the timing diagrams work in all expected use cases. And if the timing does not work in some cases, whatever, people get a phone a year. Hell on my Galaxy S3 Verizon said 'oh yeah that is a known issue where sometimes the GPS just does not work because of loose wires, that was fixed in the S4, I think, which is only $$$.'

You are also correct about direct support from the company. In face make Xilinx enough money and they will share their secrets with you. But I am more targeting small to medium development. If open tools become popular and stable enough to get market share in hobbyist up to medium sized companies, it will only be so many years before bigger companies hire new engineers who grew up using open tools and have no patience for tools that have to be emulated in windows XP and have 35 nested sub menus for enabling a feature.

I do not feel attacked at all. You brought up interesting points. I hope i did them justice and did not ramble since there were a lot of points to hit.

OP here

This is a severe problem that saddens me greatly. For now I am just writing tools to make JTAG/SPI/etc loading of chips Ubiquitous to the developer writing software/netlists for them. I do not know enough about the place and route math and routines, nor have I seen enough details of real life chips to know the types of real challenges faced. But I will cross that bridge when I get there (maybe it will be someone else :)).

I personally have a huge issue letting anything fundamental I figure out be marked down as the property of a company to have and hold for 20+ years. But I understand the financial rewards and interesting problems are more than enough for many people, so I can not hold it against a student with a PHD worth of debt to need cash. Sigh.

Hopefully if a good enough extensible base exists, people will add pieces on (whatever they can get away with) over a long time. That is all I can really hope for.

OpenCL support for FPGAs is super limited and only done through vendor specific tools. The point of these tools is more to LET you write your FPGA program that runs outside of a desktop computer in OpenCL than it is to make an FPGA be able to speed up your OpenCL program running on a desktop. So it is not like 'I will pop in a new FPGA card so I can play better games.' Though that is what i want to happen in the next...... lets say 5-10 years.

Sadly it is private. It is a channel for reverse engineering silicon devices. If you are interested and you hang out in IRC channels for microcontrollers, FPGAs, etc, and ask the right questions you may get an invite.

I get the idea you are aiming for, but this will not work.

GPUs are vector processors. They are good at certain types of repetitive unrelated (parallel) math. They can be good at simulating physics since physics is described with matrix math.

You could simulate the gates in a chip, though this is a MASSIVE problem and will tax the GPU for meh results. If you decided to simulate a chip in a GPU, simulating an FPGA would be quite pointless since FPGAs are less than optimal so they can be reconfigured to implement (not simulate) different chips.

This would be like running a Sparc emulator in a Java VM implementation of an Intel VM... mabe throw javascript in there somewhere. Each level of emulation, simulation, or implementation robs resources.

Yes those architectures are more open but they do not do the same thing. More work with those setups so, say, Postgres can offload certain math to the GPU, is great and should be done. But this is somewhat different than FPGAs (though in the same spirit).

CPUs, GPUs, and FPGAs all solve different types of problems very well. But they do not run each other's problems well, and certainly do not implement each other super well. Well, with one exception: you can build a reasonable GPU with shaders in an FPGA, but this is not to out perform the FPGA it is implemented on in a task. If you hardwired the FPGA to run the equivalent operations of a shader in OpenCL running on a GPU implemented in the same FPGA, the FPGA would win hands down.

In the future if my tool for loading the chips works and is adopted, I hope to define an interface for using FPGAs directly in the computer similar to how OpenCL defines an interface for GPUs. Hopefully I would not follow the shit show that occasionally is the Chronos Group.

Some people will say that there are already pcie cards with fpgas on them and that this is not a new idea. But the issue is that existing cards like Digilent make are there for building pcie card designs. The FPGA (usually one) on the board has to implement the pcie fabric plus whatever it wants to do. Imagine if when you write an OpenGL/CL program you had to deal with all the DMA work and quirks of the underlying setup. Instead the grid of fpgas should have standard ways to access certain mapped regions of memory through a different chip that handles the PCIe transport layer stuff. If this was available then adding FPGA support into POSTGRES would not require bypassing the HAL of your kernel or worrying about timing details of something fundamental to modern computers (and thus software engineers) like PCIe.

Obviously the FPGA is still something that is hardware specific, but who knows what could be done with that. Even if we had to Synthesize from raw HDL every time, at least loading the chips would WORK. We could start using these chips to solve problems instead of thing slike a one off project by an EE major to build a grid of FPGAs that process bitcoin. The bitcoin work was impressive but it bothers me SO MUCH that they used reconfigurable devices to build a solid card with a fixed function.

I am replying to this comment instead of the multiple people under it.

You guys are right that FPGAs are not quite ready, or maybe will never be ready to be the primary radio for a phone because of the static power consumption. You could still have one available for certain burst operations that need to be highly accelerated. Good point about the timing attacks on crypto, but hardware crypto can be updated within an FPGA. This can be seen as a pro and a con to security.

Oh god i never noticed that! Maybe it does not install when running the native linux ISE.

Haha, not at all. I do not take it as an attack or anything :).

The UI is obvious. The inability to use most of the intermediate file formats for anything since they are secret formats is annoying. If I remember correctly it had dependencies on Java and Mono. The command line tools are archaic and very difficult to use even before realizing they are not documented out of the fear of giving something away about how anything works. The iMPACT for programming chips incorrectly loads the libusb.so file and has to be LD_PRELOADed on linux, but even then there is some weird race condition that makes it work 1 out of 4 times. In order to do anything you have to download and install 15 gigs of data and agree to aggressive licenses.

The compiler tool chain usually is a compiler and a linker. I could understand if you said that the place and route was more of a linker step, but people often call the full process 'compilation'. The fact that stitching the modules together and actualizing the equivalent of addresses and instructions (in CPU terms) is a physical fitting and box packing problem in an FPGA does not make it a fundamentally disparate step to me.

Wow, I did not know that about the ARM compilers. I must have got the wrong impression from forum posts on the use of GCC for ARM chips. That makes me pretty happy, particularly that ARM is helping with the open tools. I will say though that this feels like it is supporting my point because people are using the non vendor tools. If support is a concern, support is not something that only a big company can provide. Postgres provides support and adds features on auction. I will admit that the responsiveness of Oracle for their customers having a need is much bigger and more organized, but it better be for what people pay for that. With the Oracle example, I think I am leaning towards 'there will always be room for a proprietary solution to handle edge cases of a market' instead of open solutions will not work as well as proprietary ones and are unable to be the defacto standard.

Several friends complain about the arcane nature of verilog and vhdl and like the idea of better languages written by actual language designers. But since the steps of FPGA compiling first converts the HDL language into a netlist which is used for all other operations, once a tool chain exists we can just replace the verilog compiler with a VHDL or bluespec compiler and everything will still work.

OP here

You are right that my particular project is not the biggest piece, but it is the part that pisses me off. I can deal with Xilinx's crappy compiler if I can program my chips with ease..... or maybe I should say UNTIL I can program my chips with ease. Then my focus will change hehe.

Icarus works but it has been maintained by one 'eccentric' guy for quite a while and the code base is semi unapproachable. I believe this is why the Yosys guys started from scratch.

You are also right that the place and route, as well as verification of the LAYOUT in the chip are super big problems. It is in fact what my friends toying with the compiler side are dreading dealing with because we have no idea the delays of individual traces in the chip.

I disagree that people will stick with the vendors tools. Open compilers dominate most of the Intel CPU market besides on windows since Visual Studio is the only thing that deals with all the quirks reasonably well. But the windows case is more of a lack of interest.

ARM compilers are more interesting to me because most people use Keil. Keil... works. But it feels crazy retro paying for a compiler. I believe the only reason that Keil is being used for most embedded ARM projects is there simply are not enough people with compiler knowledge using ARM regularly yet to pour enough work to make a better alternative.

But it will come.

And one day we will have synthesis tools for FPGAs that are comparable to the big guys or better.

OP here

dominicgs had some good ideas since software defined radio is very powerful. One extreme example of something you could do with a reconfigurable radio signal processor in your machine is install things that the FCC would not let be produced and sold. For example if someone reversed the military high accuracy GPS stuff, and produced a verilog spec anonymously so he was not sent to prison forever, no one would be able to use that to produce chips in consumer devices. But if we found verilog and wanted military GPS on our phones, all we do is install software.

Another example some friends have been excited about is emulation of older hardware, including video games, but much more. I will describe old games since they are closer to people's hearts. All the old cartridge based games had the actual cartridge contain half of the hardware needed to make a full computer. This immediately complicated emulation. But the real issue comes from quirks in the actual hardware. This is an issue because developers had no API except the exact hardware specifications that every console had. No one worried about checking their clock frequency was correct and dividing it to the right point so they got a certain framerate... everything was constant. Developers would find and exploit undocumented glitches in the hardware to make things faster, or sometimes to make them run at all. Hell, some games used the difference in clock frequency between two internal chips as a source for PCM sound to make little bleeps so they could save code. For this last example this means that unless you are emulating to the clock cycle each of these chips and sometimes their exact internal structures, some game will not work or crash randomly. The CPU power to do this is insane: a 3GHz core was the minimum suggested processor for running a highly accurate NES emulator in real time.

FPGAs let us implement the actual system, bugs and all, in hardware and not worry about CPU usage. Give the fpga a blob of shared memory for writing video and PCM data to and the CPU just passes blobs of memory between devices.

Programs like Postgres could implement highly optimized methods of certain algorithms like hashing for looking up in an index.

The kernel could detect it has FPGA space and offload some of its work to the chip.

I hope these have sounded interesting.

Hmm, that is a way I did not think about. Documenting hardware limitations for compilers gives competitors leverage for saying theirs is better because it does not suffer from X.

I understand that open source is not particularly important to you, but I am a bit more skeptical about the verifiability of a product that is all secret sauce and promises than I am about something with open check-able code and test suites. Open source software very rarely tries to hide its flaws to prevent a PR issue and then lazily fixed in the future because it is 'low priority', instead they are fixed by whoever can, verified, etc. There are always counter examples, but I thing the verifiability of the tool is in the same world and of similar importance to the verifiability of the output.

You do make a point in the catastrophic cost of a screw up when casting an ASIC though.

I am sorry you suffer from the dongle collection woes too. The only reason I can see to buy expensive dongles are the ones that have a big blob of RAM on them for quickly reading debug output from the chip without slowing it down, and ones that can adapt to highly exotic voltages and pin outs automatically. But usually the software for these only work in windows, have secret drivers, and are NOT worth 2 thousand dollars.

Xilinx and Altera would not have to give up their custom algorithm. It is like building GCC instead of using Intel's compiler. We just want to know what the chip's instructions are, we will find a way to make it optimal ourselves.

I am OP

astrodust: That is exactly the problem :). Turns out ISE (Xilinx's tool) has several command line tools that are run by the GUI in order, but the arguments are not documented. I was able to make a Makefile that did the ISE compilation so I could edit in emacs, but that was way too much crap to be real. The new generation of Xilinx's tools, vivado, is written in Java and does not output any temporary files so during compile it can fill over 32 gigs of RAM and crash. Xilinx suggests that until max ram gets higher to use ISE for larger chips.

kornholi: I do not want to insult hardware engineers since they are very intelligent, they just often do not respect the same things software engineers do. I went to an Atmel event where they were demonstrating how to use their new ultra low power chips. It turns out that most of the people who went were software people driving down from SF. They started off by reminding us that they have a new IDE based on the powerful and versatile Visual Studios. Everyone in the audience groaned. Almost everyone in the audience asked for assistance to find where the Makefile was. The following exchange happened over and over. host: Oh, but you see, with AVR Studios, you do not have to _worry_ about the Makefile, it does it for you. guest: yeah but I do not want to use it. host: but... why would you not want to use the tool we provide, it works. It seems to me that most of the hardware industry will take whatever tools they can get, even if they have to have the company pay several thousand dollars. The ones I have met forget that they can write tools. When the tools need to be created they just pick whoever is the most comfortable in Java, or outsource it to anyone. This all sounds very critical of professional hardware engineers, but it is not exclusively their fault. It is also a culture thing of the companies and the industry. Everything feels to me like how software engineering was (as I am told) in the 70-80 where everyone is super paranoid about secrets, and rushing to be the first. There is reasonable concern for being the first. If you build the first flash ram chip, 40 years in the future when we have moved from flash to crystallized light or something, everyone who wants to compete will be using the same pinouts you picked for your first product so that boards never have to be redesigned. As for ISE and vivado, I hear it suffers from design by committee. Where every feature has to be checked off as working before it will ship so there is a new bullet point. Hell, they have a C to fpga compiler which you would suspect could do some crazy things and took thousands of engineering hours to make work. But instead it just implements a CPU in the FPGA with slightly accelerated operations for your setup, completely missing the point of FPGAs.

I am OP

Hmm, that is actually a good point. Might as well not draw any attention to yourself incase some ass hole has a patent on the concept of moving electrons down copper or something insane.

I am OP

You are correct some of the information required to build a computer for these chips are 'internal' details. However they already have strict patents on most of it, and knowing the configuration bits of a chip and the the delay of various paths is not going to give away their fabrication technique. They could potentially release just enough details to satisfy compiler writers instead of making us do it ourselves so we try to rip every details out of it.

My friend put it up on his blog until I get around to making mine again. http://curtis.io/others-work/open-tooling-for-fpgas