FASTCUDA

In recent years, an observable trend in High Performance Computing (HPC) architectures has been the inclusion of accelerators, such as Graphical Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAs), to improve the performance of scientific applications. Several applications, ranging from graphics, to biological modeling, molecular dynamics, physics and others, have been successfully ported to GPUs, taking benefit of highly parallel hardware to accelerate their execution. Porting to GPUs, hard as it may be, requires only software skills to code the specific algorithm into parallel multi-threaded software. On the other hand, the path to FPGA development is notoriously more difficult since porting an algorithm to custom hardware is less straightforward, and the simulation-verification-debugging cycle can be many orders longer. For this reason, even though FPGAs’ custom hardware can potentially provide higher speedups at lower power consumption than GPUs, GPU-based solutions dominate the scientific world.

FASTCUDA aims to bridge this gap by taking advantage of the software parallelization effort that has gone into porting scientific applications to GPUs, and utilize it so as to implement FPGA-based systems. FASTCUDA focuses on CUDA, a GPU architecture and programming model initially developed by Nvidia for its line of GPUs, and provides a novel methodology, architecture and toolset to automatically port and run CUDA programs onto FPGA hardware.

Execution starts with the CUDA host program running single-threaded on the host CPU. Whenever a CUDA kernel is invoked, the host CPU dispatches the execution of the kernel to an accelerator (separate device) that supports parallel execution of multiple threads. Traditionally these are Nvidia’s GPUs or other multi-core platforms. However, we prove that even higher performance acceleration, as well as lower power and energy consumption, can be obtained if a computationally intensive CUDA kernel is synthesized into hardware and mapped onto an FPGA for execution. Therefore, FASTCUDA employs a hybrid approach: it uses an FPGA-based accelerator for executing the time critical CUDA kernels and a multi-core processor for executing the CUDA kernels that could not fit in the FPGA fabric.

FASTCUDA is a design methodology and accompanying toolset that allows CUDA programs to be executed efficiently on a shared memory, multi-core CPU communicating with an FPGA-based accelerator. A modern FPGA provides all required resources; multiple embedded micro-CPUs for the CUDA host program and the CUDA kernels that will be executed on the multi-core processor as well as large logic capacity for the CUDA kernels that will be accelerated in hardware. Toward this end FASTCUDA has not developed everything from scratch but it has joined numerous on-going efforts in industry and academia to create a unified efficient open-source framework.

The objectives of FASTCUDA were twofold:

1. create an innovative embedded system design flow by designing highly efficient components and by taking advantage of numerous open-source ongoing efforts in codesign of embedded systems, both at the academic and at the industrial level
2. enable an easier transition from research results to industrial exploitation, i.e. standardization of codesign usage

FASTCUDA has successfully defined the new design flow and has provided to the open-community the related toolset. The objectives have been achieved by defining, implementing and disseminating a publicly available platform that takes as input a description of the system in the CUDA programming model, and produces an efficient FPGA-based embedded design that executes certain CUDA kernels in software, while it implements the rest in hardware according to a hardware/software partitioning algorithm that has been developed throughout the project.

In order to fulfill the aforementioned objectives we have built the FASTCUDA platform which is comprised of the following sub-systems:

• A novel reconfigurable computing (RC) architecture composed of a multi-processor system, shared memory and reconfigurable fabric in order to run the multi-threaded CUDA applications.
• An advanced high-level synthesis tool which efficiently maps the coarse and fine grained parallelism exposed in CUDA kernels onto the reconfigurable fabric.
• A compiler framework in order to port the CUDA programming model to the FASTCUDA multi- processor environment.
• A design space exploration strategy based on profiling, user-driven block partitioning, and analysis by simulation, compilation and high-level synthesis of the quality of each point in the design space.
• A central on-chip processor that coordinates the execution of the CUDA kernels and executes the main code (referred as host code in the CUDA programming model) of the CUDA application.

The FASTCUDA platform is relatively easy to use through a graphical user interface (GUI) in order to gain wide acceptability by the embedded design community. Especially, as the tool targets the group of designers programming in a high-level and it is critical to speed-up their design time, the factor of having a tool that operates in a user friendly environment is of major importance. This can play an important role to the wide adoption of the tool.