Getting through FPGA - Cyclone V

Goals/Warning:

This tutorial approaches a superficial "first contact" with the design with SoC + FPGA, if you need more information about the peripherals, it's better to ask in the altera's forum where gently guys will try to help you with your idea. My goals with this tutorial it's not to explain all, but the hardest part of ever development, the kick-off to understand the development flow from zero knowledge about this devices, I'll try to minimize how difficult it was to learn from scratch with the very limited Intel documentation about the development
flow with kind of target. To resume, my goals with this article are:

  1. Brief explanation about the device (Cyclone V);
  2. Getting started with Quartus;
  3. Programming the FPGA in Quartus;
  4. Getting started with DS-5 Altera Edition;
  5. Debugging ARM-A9 through JTAG;
  6. Explanation about the boot flow and options;
  7. Generating files for SD/MMC auto boot;

Warning: If you don't have any idea about FPGA/ARM bare-metal, it'll be hard to understand some concepts and I suggest to review the references, where some links explain the concepts behind the terms.

When I decided to mess with it...

Some months ago I woke up in a friday and saw some information about the Innovate FPGA 2018 Contest, if I remember well it was one of the promotional digikey mails explaining about this worldwide contest. As I'm not fully working I decided to participate with a friend mine proposing a project using the DE10-Nano because I thought it was so nice to play with this board (so innocent to... - present Ânderson).
crios_present My project's name is CRIOS-Cruise speed controller with deep learning and now my team is one the semi-finalists, what means that I have to get this design working...so as the beginning of all, let's study this architecture.

Understanding the Cyclone V

The target in the DE10-Nano is a FPGA Cyclone V - 5CSEBA6U23I7, this is a different type of FPGA because besides the programmable logic you also have an ARM A9 Dual Core embedded into the SoC hardware in the same chipset. This target FPGA has a lot of things, it's not interesting to list all here, but in the DE10 we have a DDR3 memory of 1GB RAM that can be shared between registers with the FPGA portion and some external peripherals that help us to program the FPGA/ARM and debug the outputs (See the image above to understand). The Cyclone V itself has 110k of programmable logic elements, an ARM Cortex-A9 called as HPS (Hard processor System), 6 fractional PLL's, 5570 kbits of embedded memory (L1/L2 Cache and FPGA mem. structures) and can be programmed through serial configuration (EPCS64) and with an USB-Blaster II onboard device.
The above image reflects my feelings when I got this board in my hands for the first time... and the board it's like RAMBO (rambo II to be more precise), you can do a huge improved project but with the difference equivalent to get a .50 mm in my hands without a manual for someone that never touched in a gun. What means that I couldn't do nothing and maybe even burn the board, because some peripherals cannot be configured wrong (I'm not sure but it was the problem with some guy in the forum).

Getting Started with Quartus

Through this link you can get the Quartus 17.x (actual version) in lite edition, to install this program there isn't any magic, it's simple as all classic Windows programs (we're using windows because in Linux you'll take a long time to install the simulation tool / modelsim and other problems related to). Install the software and open it in the default home screen as in the image below:
Now we'll use a default project for the DE10-Nano that's referenced as a golden model design, such files are available in this link, the zip that must be downloaded it's DE10-Nano CD-ROM.
Once you have the zipped file, unzip it and open the project called GHRD (\DE10-Nano_ v.1.3.1_ HWrevC_ SystemCD\Demonstrations\SoC_ FPGA\DE10_ NANO_ SoC_ GHRD), the quartus project file has the extension .qpf. When opened the project, click in the third play button, called as Start Analysis & Synthesis this will build the design and generate a file with the extension .sof that means SRAM Object file, so to program the FPGA of Cyclone we will need this. Also, it's important to convert this file to another format called .rbf, to convert this file go to the output file folder (after synthesis - DE10-Nano_ v.1.3.1_ HWrevC_ SystemCD\Demonstrations\SoC_ FPGA\DE10_ NANO_ SoC_ GHRD\output_ files) and then convert with it:

C:\intelFPGA_lite\17.1\quartus\bin64\quartus_cpf.exe -c .\DE10_NANO_SoC_GHRD.sof soc_system.rbf  

Then soc system.rbf will be our design in the format RAW Binary format that can be programmed into the FPGA, to program the FPGA, open the programmer tool from Quartus menu, go to *Tools>Programmer* to see an image like below:
In the first time that you open this, it'll not list the hardware, so you need to go to *Hardware Setup > Add Hardware* and then click in the device listed, in my case DE10-Nano..., so now it'll be showed two IC's like the image above. The first is the HPS (ARM) side and the second (5CSEBA6...) is the FPGA itself. To program the FPGA, click on list *File* and then select the soc
system.rbf file, after check the checkbox defined as Program/Configure, once up it's done click on start to program the FPGA, ten later seconds you'll see 100% Successful in a green box in the right top corner box and your design will be available into the Cyclone V.

Qsys info

Another important tool to highlight in the development with Cyclone V is the tool called Qsys or in newer version Platform Designer, this tool is responsible to integrate HPS world and the FPGA design through a graphical user friendly interface. As I'm not expert on this tool yet, I'll just show the design imported from GHRD project and brief about what this tool does.
As you can see in the image, Qsys shows to us some devices connected through wires in other blocks, that represents a hardware connection between the modules that'll be generated when you finish to setup the tool. What means that after we define the connection of the HPS system with the external hardware (I2C, FPGA Connections, JTAG, Avalon..) we can generate Verilog or VHDL with the configuration between the interfaces automatically, turning the connection between devices so much easier, and if we're using SDRAM (DDR3 1GB/DE10-Nano) it's almost necessary to use Qsys to setup our SDRAM Controller because this memory has a lot of signals that're necessary to connect and configure.

In the end of Quartus minimal-setup, save your final design soc_system.rbf that we'll need in the boot flow of this article/tutorial.

Working with DS-5 Altera Edition

Here you can get the ultimate DS-5 Altera edition (download the standard because it supports Cyclone V), that's the development suite IDE for ARM related to Altera's boards. It's sad but we cannot do nothing with a free version, thus register yourself in the ARM website to get a full-pro trial license, that enables complete compilation with JTAG debugging in the DS-5. With the free version you can not even use JTAG to debug your hardware, what isn't interesting when we're developing a project with some ARM target. After install the tool, navigate to the installation folder in the directory C:\intelFPGA\17.1\embedded\examples\software to see some examples with Linux and Bare-metal application.

In this link there's a global tutorial of how to manage the DS-5 but in my opinion this don't cover all important details that we always "need to know" when starting a new development with this tool. First, once up installed, check in your Desktop environment if you have this Embedded_ Command_ Shell.bat shortcut, if you have, open it as admin user in Windows. Be aware that we'll always use this .bat as source for all tools henceforward such as eclipse, mkpimage and others that you'll see in front. Before open the DS-5 it's important always add the PATH for Quartus tools in the beginning of working with the embedded shell:

PATH=/cygdrive/c/intelFPGA_lite/17.1/quartus/sopc_builder/bin:$PATH  

To open the main tool to develop code for ARM type it in the terminal:

eclipse &  

Wait some instances to the tool open and show an IDE of Eclipse.

When you set some peripherals in Qsys you define a lot of address for the HPS use as a memory mapped structure and the HPS can know this devices through a header file (.h) that must be included in your ARM project. To generate this header, go to the folder of the GHRD project (Quartus project) and call the script called:

./generate_hps_qsys_header.sh

Then you'll see a message indicating that a hps_0.h file was created. For our sample application this will not be necessary but in future applications you'll needed it, so remember to generate and include in your project.

We'll use an example provided by Intel to make our first bare-metal application for the ARM-A9, to download it click on this link. The example is called as Altera-SoCFPGA-HardwareLib-Unhosted-CV-GNU, the unhosted term is referred to the fact that we'll put some information in UART Interface of the DE10-Nano and not in JTAG semihosting default. Once the Eclipse is open, go to File > Import > Existing projects into Workspace because the example also provides a Makefile to build it, now in the browse option point to the .tar.gz file and then Finish.
With an image like mine, try to compile clicking in the hammer icon or right-click in the project in build project. After click on Run > Debug Configurations then on "Altera-SoCFPGA-HardwareLib-Unhosted-CV-GNU-Debug". In the image below, see how you need to set the debug launcher for the DE-10 Nano. Clicking in the Browse box inside the Connections tab, you need to select the device connected through the USB. If you device is not listed, check if you connected the right USB cable and installed the USB-Blaster driver for Windows. Once done, click on close.

Now we'll debug the device and to make this work we need to load our bare-metal app with a preloader file. As we're working in a debug environment, DS-5 uses a file called debug script to manually load it through the app in the SoC. This file were imported to your project with the name of debug-unhosted.ds but if you try to launch the debug session you'll be stucked in the execution of the debug session because the preloader has an old path to the real spl.c functions for Altera/14.1 version as in the image.
So, what I decided to do is to point to the preloader generated by myself and that works! To download my preloader click on this repository but don't be afraid of don't understand about this because later on this post you'll generate your own preloader derived from your Qsys project. For now to launch the debug session with your target using my preloader files, to edit the debug script open the file called debug-unhosted.ds and modify it in the preloader step (loadfile) with an absolute path to the following directory:

<CYCLONEV_EXAMPLE-REPOSITORY>\sw\hps\preloader\uboot-socfpga\spl\u-boot-spl  

In the end your debug script will be like mine:

Then, to debug your target, click in the "bug" icon or Run > Debug and switch to the Debug workspace. Once up the preloader start the peripherals, the play button will be available to click, so when you start, you'll see and image like below. Remember to open a terminal(if you don't know what I'm talking about, see this) with the configurations 115200-8-N-1 in the UART interface of your board to see the output.

After you compiled the project, see the list of files generated and check if a file called hello.bin was created. This file is your application bare-metal in binary format and it'll be needed in the sequence of this tutorial where you'll create a bootable SD Card for DE10-Nano. Save this file with the name called baremetal.bin, the other binary (hello-mkimage.bin) is the same binary but with a CRC header needed by preloader if you want to start your code directly with the preloader and do not use bootloader (uboot), you'll understand more about the boot process in the following topics.

Boot flow and characteristics

Now that you arrived at this step, it's important to understand how the FPGA boot works, pay attention that this corresponds just to the HPS = ARM (who needs to be booted, the FPGA design is programmable the right term). According to this Intel boot guide the normal flow to boot is like the image below:
boot_flow Where we can run two kind of programs in the ARM, a Linux OS or a bare-metal application, by default the DE10 comes with a SD-Card with the Angstrom OS image with XFCE desktop environment and a design for the FPGA. The concept behind each step of the boot is simple, I'll just brief in a sequence way:

  1. bootROM - detect boot source from external keys (we'll talk later about this BSEL pins...), perform HPS hardware boot and load the next boot stage;
  2. preloader - bring the SDRAM, init some peripherals from HPS, load the next stage of boot (bare-metal or bootloader), loadup in the OCRAM;
  3. bootloader - loadup in SDRAM what means that we have more space available, bring up other peripherals from HPS and load the application;

The Cyclone V has different types of boot sources such as the list below:

  • SD/MMC (what we'll choose)
  • QSPI
  • NAND
  • FPGA (it's bizarre the way as the bootROM jumps to the FPGA Address)

I decided to choose SD Card as boot source because it's more easy to change files, make copies and define scripts for the bootloader. In the DE10-Nano the boot source is defined by the position of the DIP Switches below:
The boot flow also has more options, Intel informs that we can custom the flow and use just some parts as in the image below for the Cyclone V:
more_boot_flow In this approaches above the idea is to run a RTOS (Real-Time Operating System) or a bare-metal application in any part of the steps, btw to run an OS like Linux distro's you need to have a BOOTLOADER in your flow what isn't needed if you wanna run RTOS/Bare-metal. In this tutorial I'll choose the first option of this image with bootrom > preloader > bootloader but I will not use Linux, my idea is to use bare-metal application because there're lot of tutorials of generating a bootable Linux image with u-boot (u-boot is our bootloader). Intel also explains that if we want just boot a bare-metal app, we can just use preloader or even bootROM directly as explained in this guide developed by Intel. Then someone can ask me:

Why are you adding bootloader to the boot process Ânderson?

Then answer is so much simple, because we need to program the FPGA with our design. With bootloader we can run some commands and then load the design into the FPGA and after run our application, I just don't understand why Intel and any other tutorials in the Internet didn't mentioned this fact (in my mind it's like obvious that I want to program the FPGA too). If we're working with FPGA that has an ARM, I think that's important to program the FPGA in the boot because the idea behind the boot with the SD/MMC is to become a standalone device but okay, this is one of the my frustrations along the tutorials on the web.

  • Obs.: By default the boot process in ARM dual core just wake up only CPU0 and the CPU1 must be launched by the application from reset state.

Now that you understand all the boot concepts, let's create a flow to generate boot files inside a SD Card to program a design in our FPGA and run a bare-metal program in the ARM. The list of steps could be resumed by this:

  1. Prepare the SD Card;
  2. Generate the preloader;
  3. Generate the bootloader;
  4. Generate the bootloader script file;
  5. Copy all files to the SD Card;

Before continue, check if the files needed to create a bootable SD Card are present:

  • A design for the FPGA in the format .rbf (can be generated from a .sof with quartus_cpf tool, google this), this will be generated after place & route in Quartus. It was generated in the first part of this article;
  • Bare-metal application compiled in .bin format (generated in the DS-5 AE);
  • The HPS design files (.c,.h...) generated by Qsys. This files are present in the GHRD example that you compiled in the first part;

Considering that you have all the files above continue this tutorial, if isn't the case, review the first part with the Quartus/DS-5 to find the mismatch step.

Step 1 - Creating a bootable SD

For the first step, we need to prepare the SD Card to receive the files needed to boot, to be honest I don't know how to do it in Windows, so I'll just explain how to do in a Linux based OS, if you don't have any Linux available you can use a VMWARE. If you work on the right-side of the force, follow this commands (you need to have fdisk software installed):

  • Pay attention on this device ( /dev/sdx ) because if you get the wrong disk you'll do a big mess your system, usually it's /dev/sdb =)
sudo dd if=/dev/zero of=/dev/sdx bs=512 count=1 #Format the SD Card first  

After cleaned the SD, let's create the partitions, so type this commands "n","p"... in the sequence and press enter if some message appears referring about partition signature, just type yes. The below commands create a partition of 1MB in the format A2 and ID:3 (needed for preloader) and a partition of 32MB in the format FAT32 with ID:1, so follow this:

sudo fdisk /dev/sdx  
n p 3 <default> 4095 t a2  
n p 1 <default> +32M t 1 b  
w  
sudo mkfs.vfat /dev/sdx1 #Format the part. 1 to receive files  

After you'll have something like this where you'll put the correspondent files:

Don't worry if you do not understand right now what's each file because in the next steps we'll create those to put in your SD Card.

Step 2 - Generating the preloader

This is step is very dependent in the first part of the code generated in Qsys what means that if you want to change something in Qsys, it's recommended that you generate the preloader again, because it initializes the peripherals interfaces defined in the to Qsys. So, to generate the preloader follow this. Open the shell command through "EmbeddedCommandShell.bat" as admin in Windows an open the BSP editor:

bsp-editor.exe  

Then select <YOUR-QUARTUS-PROJECT>/hps_isw_handoff/soc_system_hps_0 as Preloader settings directory and in the BSP Target directory select an output directory for the preloader files as in the image below, then click in ok:
Then disable the option WATCHDOG ENABLE and enable FAT _ SUPPORT/BOOT_ FROM_ SDMMC also, check if the follow informations are according to this image:
Once up it's finished, click generate to generate the required files to compile the preloader through Makefile, when finished just click on exit. Now, go to BSP TARGET DIRECTORY that you've set before and type:

make  

It'll take some time to finish and you'll have the image of preloader ready to put on your SD. Separate the file preloader-mkpimage.bin because we'll use later.

Step 3 - Generating the bootloader

In the same directory as before type:

make uboot  

This will take more time than before and you can grab a cup of coffee to wait for this...

Step 4 - Generate the bootloader script file

For this step, open a text editor from your preference and then write this file below, that indicates for the bootloader (uboot) what to do in the first moment of boot:

 echo '====== Programming the FPGA ======'
 fatload mmc 0:1 $fpgadata soc_system.rbf;
 fpga load 0 $fpgadata $filesize;
 echo '====== Enable bridge FPGA <-> HPS ======'
 run bridge_enable_handoff;
 echo '====== Running bare-metal application ======'
 fatload mmc 0:1 0x00100040 baremetal.bin;
 go 0x00100040;

The first instruction indicates to the bootloader to load the FPGA design image and in the second line, program the device. In the run bridge enable handoff we enable the bridge hardware to communicate HPS and FPGA and then in the latest instructions we load our bare-metal application and then run with go command.

You can save as u-boot.script then open again Embedded_Command_Shell.bat and convert this to a bootable script file, indeed the command below just add a header for CRC and other mechanisms of bootloader accept this as a script file during the boot process. If you want to know more about this script file, go to this link. It's important to highlight that you cannot change the output file name (u-boot.scr) because this is the name that'll be searched by the u-boot during the boot process.

mkimage -T script -C none -n 'Script File' -d u-boot.script u-boot.scr  
Step 5 - Copy all files to the SD Card

Now that you arrived at this step, be sure that you have:

  1. soc_system.rbf - Image of FPGA design
  2. u-boot.img - Image of the bootloader
  3. u-boot.scr - Script file for the bootloader
  4. baremetal.bin - Binary file with the source code
  5. preloader-mkpimage.bin - Preloader generated from your design

Once up it's all with you copy the preloader-mkpimage.bin file to a Linux distribution, insert the SD Card and do the following:

sudo dd if=preloader-mkpimage.bin of=/dev/sdb3 bs=64k seek=0  
sync  

The other files you can grab and copy manually through the Windows system because it can mount FAT32 without problems. So, copy the soc_system.rbf, u-boot.img, u-boot.scr and baremetal.bin to the FAT32 partition of the SD Card, you could also copy with dd tool in linux but do as you prefer.

Final Output

After all the parts, if nothing gets wrong you'll endup with something like this on the USB/Serial interface from DE10-Nano:

Obs.: Do not care about the ASCII logo "CRIOS project", this is a modification that I've made to know which SD Card is mine and what is not.

If you liked the post please share and subscribe, or if you have any suggestions/comment post below. My personal e-mail is: anderson@aignacio.com

References:

Ânderson Ignacio da Silva

Electrical Engineer with electronic and protocols skills.

Rio Grande do Sul - Brasil http://aignacio.com

Subscribe to @aignacio's blog

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!