/ Cyclone V

FPGA/Linux co-design with Cyclone V - FPGA Side [1/3]

Beginning in the latest post about Cyclone V, I decided to investigate even more how we can create a co-design (hardware+software) to embedded into this target to create an interesting application.

Prerequisites Softwares

  1. SoC EDS Standard
  2. Quartus Lite
  3. arm-linux-gnueabihf
  4. arm-altera-eabi - Installed along the SoC EDS

Also, by an external demand, I decided to join the idea of learning more and add an I2C sensor reading in a linux application running into DE10-Nano board from Terasic. So, in this post I present a way to create a custom hardware with Quartus, Platform Designer (oldname Qsys) tools that runs a simple Linux distribution to access some peripherals and read an I2C sensor. This is a long post and maybe some details are not complete covered, so check the references in the footnotes to review if something does not makes clear.

The tutorial that I'm using as a reference was provided by RocketBoards.org, that is community Site for SoC FPGA Developers, if you enjoy my post go ahead and check that site because it has a lot of excellent material for beginners and advanced users.

Folder Structure

Before we begin as this tutorial has a lot of different type of files, we need to create a folder structure with all the files needed to make this co-design happen, I based my organization on the excellent tutorial provided by EPFL university. So, follow this commands below to create a structure of folders:

cd ~
mkdir -p ex_codesign/hw/ghrd_project
mkdir -p ex_codesign/sw/preloader
mkdir -p ex_codesign/sw/bootloader
mkdir -p ex_codesign/sw/linux
mkdir -p ex_codesign/sw/application
mkdir -p ex_codesign/sd/a2
mkdir -p ex_codesign/sd/fat32
mkdir -p ex_codesign/sd/ext3

You need to have this structure to continue, in the next sections you'll understand why it's important has this organized:
Screenshot-from-2018-08-04-23-12-47-1

The FPGA/Hardware side

The board that we're using it's a DE10-Nano by Terasic, and the main target of this board it's a Cyclone V 5CSEBA6U23I7 SoC with a hard-IP of ARM dual-core Cortex A9, follow the image below to check all the features of this board. Although we're using this target, you can user any board related that uses the Cyclone V as your target. The basic difference in other boards will be the pinup and the GHRD project, that you'll need to construct from scratch if your board does not be one of those provided by Terasic.
DE10-Nano_Board_Top
Although the price is not cheap from Brazil side, this board is fully useful with all the peripherals and connections for anyone that wants to start working with FPGA/embedded Linux both. I obtained mine by a contest provided by Terasic called Innovate FPGA and me and my friend get the 4th place in the contest with our design, see the latest post to understand exactly about what I'm talking about. To start with this target, switch all the MSE(SW10) switches to ON state, to unleash the HPS (ARM side) to program the FPGA through the bootloader (in our case u-boot).
b4912204-f643-4618-9e61-cd862ad88a34

Now separate a SD card with at least 4GB of size to start inserting files in the next sections, if you bought exactly this board, it comes with a 8GB SD. In our approach we'll use the Golden Hardware Rerefence Design (GHRD) provided by Terasic company to use as our hardware design project, as also the Platform Designer (Qsys oldname) project that has a model of using a lot of different hardware structures to the ARM side use. To download the reference project go to this site and download the DE10-Nano CD-ROM that has all the needed documents and files to start working with this target.Warn: Check the hardware revision of your board for the correct download.
dfsdf
Unzip the download and then copy to this folder:

unzip ~/Downloads/DE10-Nano_v.1.2.5_HWrevC_SystemCD.zip
cp -rf ~/Downloads/DE10-Nano_v.1.2.5_HWrevC_SystemCD/Demonstrations/SoC_FPGA/DE10_NANO_SoC_GHRD ~/ex_codesign/hw/ghrd_project

Now, open Quartus with this project and the Platform Designer (under Quartus at tools tab), once it's open, load the soc_system.qys project, also available at the same folder. Right-click on hps_0 and select edit, navigate to Peripheral Pins > I2C Controllers and change the select box of I2C1 pin to HPS I/O Set 0. This enables us to use I2C1.SDA/SCL pins to communicate with an I2C sensor in the next steps. You must have an image like this one if you scroll down at the end of the page, click on finish after:
Screenshot-from-2018-08-04-21-33-21
Before you finish the Platform Designer, generate the verilog files with Generate HDL... button at the bottom of the window to generate the hardware compatible with our minor modification. By the end you must have something like this one below.
dddxdsf
If you want to add another peripheral as a custom component device, check this tutorial by bitlog. As we have changed the internal muxes of the FPGA, we must re-generate the IP hardware, because of this we had clicked on generate. Now click on Start Analysis & Synthesis and wait to finish the process in Quartus.

When waiting the process of elaboration/synthesis, open the file DE10_NANO_SoC_GHRD.v, this is the top file of the design and it has the HPS structure of the SoC at (u0 - soc_system), in this module you'll see all the connections that needs to be wire up by the top to work with the structures that we have been set before.

wire hps_fpga_reset_n;
wire     [1: 0]     fpga_debounced_buttons;
wire     [6: 0]     fpga_led_internal;
wire     [2: 0]     hps_reset_req;
wire                hps_cold_reset;
wire                hps_warm_reset;
wire                hps_debug_reset;
wire     [27: 0]    stm_hw_events;
wire                fpga_clk_50;
// connection of internal logics
assign LED[7: 1] = fpga_led_internal;
assign fpga_clk_50 = FPGA_CLK1_50;
assign stm_hw_events = {{15{1'b0}}, SW, fpga_led_internal, fpga_debounced_buttons};
...
...
...
...
reg [25: 0] counter;
reg led_level;
always @(posedge fpga_clk_50 or negedge hps_fpga_reset_n) begin
    if (~hps_fpga_reset_n) begin
        counter <= 0;
        led_level <= 0;
    end

    else if (counter == 24999999) begin
        counter <= 0;
        led_level <= ~led_level;
    end
    else
        counter <= counter + 1'b1;
end

assign LED[0] = led_level;

Note that the wire fpga_led_internal it's connected to seven LEDs from the 2 to the 8 (7:1), and the first LED[0] it's used as a heartbeat for the FPGA side, in other words, it'll not be available by the ARM to control it. The blink speed of this LED it's configured to 2Hz approximately.

Note that after the process in Quartus tool you'll need to create the programming file (.sof format) and then convert it to .rbf that's the format that'll use in the future steps. To generate the file click on Assembler and then go to File > Converting Programming Files... in the sub-menu change the selection box to Raw Binary File (.rbf) and then add the DE10_NANO_SoC_GHRD.sof located in the output_files folder, after click on generate. If all the steps goes ok, you'll have a output_file.rbf inside the project's folder. Copy like this way to the correct folder:

cp ex_codesign/hw/ghrd_project/DE10_NANO_SoC_GHRD/output_file.rbf ex_codesign/sd/fat32/soc_system.rbf

Boot process

In my first post I explained in details, the boot flow of the Cyclone V so now I'll shroud this step with a short explanation. This image below provided by RocketBoards.org shows a good briefing of all the steps of the boot flow, so note that we'll use the same structure for our design.
igp_38917fd504aa0bd51b49e554a51c7833_3-bootflow
By each step in the image, I'll list all the files needed to has this flow working on our target.

  1. Boot ROM - as the name says, it's a ROM so we don't need any files because it exists in the board pre-programmed by factory;
  2. PreLoader - the preloader needs a file called preloader-mkpimage.bin that'll start our hardware, next file to be generated in this tutorial;
  3. U-boot - the bootloader of our system needs the programming file of the FPGA (soc_system.rbf we've been just created one step before), the u-boot image (u-boot.img), an u-boot script (u-boot.scr) that has the instruction of how to boot the device (program fpga, load device tree, load kernel image...), the device tree (soc_system.dtb we'll generate in the next steps) that has all the information for the Linux kernel knows which hardware we're loading up and by the end the Linux Kernel image (zImage);
  4. Linux - this is the root filesystem of our Linux that has all the files to complete the boot flow, initializing shell and memory units, in the next steps you'll understand what's this;

Generating the preloader

Navigate to the project folder and open the BSP editor that'll be responsible to generate the preloader files to us.

cd ~/ex_codesign/hw/ghrd_project/DE10_NANO_SoC_GHRD
bsp-editor &

Click on New HPS BSP and then select the path ~/ex_codesign/hw/ghrd_project/DE10_NANO_SoC_GHRD/hps_isw_handoff/soc_system_hps_0 as the Preloader settings directory. Then uncheck the use default locations and point the BSP target directory to the path /home/YOUR_USER/ex_codesign/sw/preloader, after click in the OK option.
Screenshot-from-2018-08-03-18-52-30
Also, in the next screen, check the FAT SUPPORT option to enable the preloader find for bootloader into the SD Card partition 1. After click on the generate option to start the generation of the SPL.
Screenshot-from-2018-08-03-19-48-06
Now we'll start the process of making the SPL with the following commands:

cd ~/ex_codesign/sw/preloader
export SOCEDS_DEST_ROOT="/home/YOUR_USER/intelFPGA/18.0/embedded/"
export QUARTUS_ROOTDIR="/home/YOUR_USER/intelFPGA_lite/18.0/quartus"
make

Note that we need to export an environment variable with the path of the SoC EDS software and Quartus, so that's why need set the export. Check where you have been installed both tools used in the tutorial (SoC EDS and Quartus). Some tools as mkpimage need to be called sometimes, thus find this tools in the intelFGPA folder that you've been installed with find command (find intelFPGA -name "mkpimage") and create the symbolic link in the /usr/bin. The cross compile toolchain used to compile the SPL is the arm-altera-eabi so you need also to create the links as I've done in the image below:
Screenshot-from-2018-08-03-20-13-04
With the SPL generated, copy it to the following folder:

cp preloader-mkpimage.bin ../../sd/a2/

Generating device tree

Now it's time to generate the device tree file, which is a data structure that defines the hardware inside of a system, this data is independent from the bootloader and kernel levels. It composes all the information about peripherals such as drivers, interrupts and clocks that are needed to execute the hardware by the kernel level. With this file, the same Linux image can be used with a different set of hardware devices communicating by different protocols. All the configuration of the hardware is loadup by runtime when the Linux kernel reads the device tree and boots. Two different files are used in this step, a device tree source (.dts), that's the device tree in human-readable format and the device tree binary (.dtb) that's the compiled version which Linux uses to load. As soon as the kernel is loaded, it brings the device tree, parse it and select which drivers to load/mount to be available to communicate with the hardware set.
igp_9a2d8af5579f025e2fd6b2de59e8740f_6-devicetreeflow
To generate the device tree we need to get some files from the Quartus project where the Qsys/Platform Design has been generated to us. In this link you have all the complete information about which is each file, so if you don't want to know exactly just go ahead with:

 cd ~/ex_codesign/hw/ghrd_project/DE10_NANO_SoC_GHRD/
 sopc2dts --input soc_system.sopcinfo --output soc_system.dts --type dts --board soc_system_board_info.xml  --board hps_common_board_info.xml --bridge-removal all --clocks

As in the previous section if you do not have this program sopc2dts, try to find it in the SoC EDS tool like me:
Screenshot-from-2018-08-03-22-36-18

After generate the .dts you'll need to convert it to .dtb format compiling it. The below command must do it:

sudo apt-get install device-tree-compiler -y #Install the compiler
dtc -I dts -O dtb -o soc_system.dtb soc_system.dts

Do not borrow with the warnings, reading back the log it seems that they appear for peripherals that aren't common for the Linux kernel such as PLL (Phase-Locked Loop).
Screenshot-from-2018-08-03-22-42-59
Now copy it to our folder structure:

cp soc_system.dtb ../../../sd/fat32/

Generating bootloader

When you change the peripherals of your hardware you must to regenerate the preloader and maybe the device tree too. The unique files that you'll maintain are the bootloader image and bootloader script that in our case will be the das u-boot.
preloader-uboot
The u-boot that act as our bootloader is responsible to initialize the remaining structures for the kernel assumes. This occurs by bringing the SD Card up and pointing in which folder will be our root filesystem along with the boot arguments that the kernel low-level needs to initialize the operating system. Also a very important feature of the u-boot is to program the FPGA with the .rbf image that we'll store in our SD card. To build the bootloader image we need to export another environment variable that'll point to the cross compiler toolchain needed.

export CROSS_COMPILE=arm-linux-gnueabihf-
cd ~/ex_codesign/sw/preloader/

You can generate the bootloader from the version pre-generated by the SPL because it comes with the latest SoC EDS that you've installed on your computer. Another option it's clone the bootloader from the official repository with a verified tag. So, follow the commands below to compile the from the official repository:

cd ~/ex_codesign/sw/bootloader
git clone https://github.com/altera-opensource/u-boot-socfpga.git
cd u-boot-socfpga
git checkout rel_socfpga_v2013.01.01_17.08.01_pr
make mrproper
make socfpga_cyclone5_config
make

We also need to create a boot script, where the bootloader will look up to run when it starts. In the same folder do the following.

nano bootloader.script

And paste this code:

echo -- Starting BOOTLOADER SCRIPT --

echo -- Programming FPGA --
fatload mmc 0:1 $fpgadata soc_system.rbf;
fpga load 0 $fpgadata $filesize;
run bridge_enable_handoff;

echo -- Setting Env Variables --
setenv fdtimage soc_system.dtb;
setenv mmcroot /dev/mmcblk0p2;
setenv mmcload 'mmc rescan;${mmcloadcmd} mmc 0:${mmcloadpart} ${loadaddr} ${bootimage};${mmcloadcmd} mmc 0:${mmcloadpart} ${fdtaddr} ${fdtimage};';
setenv mmcboot 'setenv bootargs console=ttyS0,115200 root=${mmcroot} rw rootwait; bootz ${loadaddr} - ${fdtaddr}';

run mmcload;
run mmcboot;

In this script you could check that we program the FPGA, load the device tree binary, indicate where is the rootfs partition in the SD card and set some kernel variable to bootz. Although it's easy to create the script, you need to add a header with the following command:

mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n "Boot Script DE10-Nano" -d bootloader.script u-boot.scr

Where the outputs will be copied to our folders:

cp u-boot.scr u-boot.img ../../../sd/fat32/

Other u-boot versions

If you decided to use other u-boot repository/versions you need first set the board with this commands:

make mrproper
make socfpga_de10_nano_defconfig
make

Then if you're using a different board that's based on the Cyclone V, just lookup for that in the configs/ folder as the image below.
Screenshot-from-2018-08-03-23-33-03
However if you want to create a custom bootloader with your target board print name and other options, this excellent tutorial explain how exactly you can do. The commit hash that I've used of the u-boot was a839c3641e4de98981695056eeeb2ec17ba1a4ab.

Now in the end of this post you must have this files in these folders, check it and if you have something missing, go back to previous steps:
Screenshot-from-2018-08-04-10-16-07

If it's all up and ok, grab you a coffee and check my blog for the part 2 of this huge tutorial =). If you like this post, please share/subscribe in the newsletter below to receive new updates.
29d594e8f2f3d5e655455e7cab079208

References
FPGA/Linux co-design with Cyclone V - FPGA Side [1/3]
Share this

Subscribe to @aignacio's