HN user
Regarding DNN inference FPGA can provide low latency AND higher throughput than GPUS.
If you want to compare apples-to-apples, we have done a comparison with realistic (and not synthetic) data regarding the performance of GPUs and FPGAs.
https://medium.com/@inaccel/faster-inference-real-benchmarks...
2 are the main challenges of the FPGA utilization:
- The first one is the FPGA programming. Now using OpenCL and HLS is much easier compared to VHDL/verilog to design your own accelerators.
- The second one is the FPGA deployment and integration. Until now it was very difficult to integrate your design with applications, to scale-out efficiently and to share it among multiple threads/users. The main reason was the lack of an OS_layer (or abstraction layer) that would enable to treat FPGAs as any other computing resource (CPU, GPU).
This is why at inaccel we developed a unique vendor-agnostic orchestrator for FPGAs. The orchestrator allows much easier integration, scaling and resource sharing of FPGAs.
That way we have managed to decouple the FPGA designer from the software developer. The FPGA designer creates the bitstream and the software developer just call the function that wants to accelerate. No need to define the bitstream file, no need to define the interface or the memory buffer allocation.
And the best part: It is vendor and platform agnostic. The FPGA designer creates multiple bitstream for different platform and the software developer couldn't care less. The developer just call the function and the inaccel FPGA orchestrator magically configure the right FPGA for the right function.
A product for you to build, ship and run FPGA accelerated applications
You are right that the FPGA vendors does not provide so far the right way of abstraction for Computing.
That's way at InAccel we developed the FPGA resource manager that allows to instantiate and deploy FPGAs in the same way as you invoke typical software functions. The FPGA manager takes care the scheduling, the resource management and the configuration of the FPGAs from a pool /marketplace of hardware accelerators.
That way is easier than ever to use FPGAs in the same way you use optimized libraries for CPUs/GPUs.
And we have a free community edition: More info at: https://www.inaccel.com/