目的
FPGAでLinuxのブートシミュレーションがしたい。
Linuxブート動作を電子回路のレイヤーで覗いてみたい。
（Verilog HDLシミュレータで信号シミュレーションが出来るのか？はまだ未確認）
LiteXとは？
FPGA Linux simulationなどのキーワードでぐぐっていたら見つかった。
どうやらこれを使うとFPGA上にLinuxが動くSoC（System On Chip）回路を作ってくれて、さらにLinuxイメージのロードもしてシミュレーションやら、実機にダウンロードして動作させる事が出来る素晴らしい取り組み。
これが本家（かな？）
https://github.com/enjoy-digital/litex
今回スクリプト構築の参考にしたサイトはこちら
https://github.com/litex-hub/linux-on-litex-vexriscv
実験環境
ホストOS： Ubuntu 20.04
ホストCPU： INTEL
ホストメモリ： 32GB
ホストDISK： 512GB
スクリプトのポイント
・参考にした記事のGNU toolchainインストール手順は使ってるデータが古かったのでそこだけ別のサイトを参考にした。
https://github.com/litex-hub/linux-on-litex-vexriscv
・GNU toolchainビルドの時の以下で（自分の環境では）長い時間止まるので「何かおかしい？」と思ったが、そんな時は別ウィンドウで「watch -n 1 df」を実行し、少しづつだけど何かダウンロードされている事を見て、自分を安心させる。
INFO : (riscv-gnu-toolchain) make linux -------
cd /root/riscv/linux_on_riscv_arty_a7-100t/riscv-gnu-toolchain && \
flock `git rev-parse --git-dir`/config git submodule init /root/riscv/linux_on_riscv_arty_a7-100t/riscv-gnu-toolchain/gcc/ && \
flock `git rev-parse --git-dir`/config git submodule update /root/riscv/linux_on_riscv_arty_a7-100t/riscv-gnu-toolchain/gcc/
Submodule 'gcc' (https://gcc.gnu.org/git/gcc.git) registered for path 'gcc'
Cloning into '/root/riscv/linux_on_riscv_arty_a7-100t/riscv-gnu-toolchain/gcc'...
・Digilent社Arty A7 100Tボードはそこそこ高額でお財布がイタイが、別のボードを選んで余計なデバッグをしなくて済むように、そこはガマンした。もっと安い35Tの方でも問題ないと思われるが未確認。
事前準備
・別記事で書いたスクリプトを参考に、Vivado 2019.1をインストールしておく。
⇒ LiteXを記事の通り動かすには実行には、Vivado HLx Editionsが必要だった。
2015.1とかのバージョンだとVivado（無印）なのでLiteXコマンドでエラーになった模様。
Artix-7をサポートしてるバージョンとして2019.1を選択。
LiteX環境構築スクリプト
#!/bin/bash
# https://github.com/litex-hub/linux-on-litex-vexriscv
echo "INFO : add it to PATH in /etc/bash.bashrc : /root/.sdkman/contrib/completion/bash --------------------------------"; export PATH=/root/.sdkman/contrib/completion/bash:$PATH
echo "INFO : add it to PATH in /etc/bash.bashrc : /root/.sdkman/candidates/java/current/bin --------------------------------"; export PATH=/root/.sdkman/candidates/java/current/bin:$PATH
echo "INFO : add it to PATH in /etc/bash.bashrc : /opt/riscv/bin --------------------------------"; export PATH=/opt/riscv/bin:$PATH
echo "INFO : source /tools/Xilinx/Vivado/2019.1/settings64.sh --------------------------------"; source /tools/Xilinx/Vivado/2019.1/settings64.sh
DIR_BASE=/root/riscv/linux_on_riscv_arty_a7-100t
DIR_PJ=$DIR_BASE/linux-on-litex-vexriscv
echo "INFO : apt -y update -----------------------------------------"; apt -y update
echo "INFO : apt -y upgrade -----------------------------------------"; apt -y upgrade
echo "INFO : (Prerequisites) mkdir $DIR_BASE ------------------------"; mkdir -p $DIR_BASE
echo "INFO : (riscv-gnu-toolchain) cd $DIR_BASE ----------------------"; cd $DIR_BASE
echo "INFO : (riscv-gnu-toolchain) install autoconf ---------------"; apt-get -y install autoconf
echo "INFO : (riscv-gnu-toolchain) install automake ---------------"; apt-get -y install automake
echo "INFO : (riscv-gnu-toolchain) install autotools-dev ---------------"; apt-get -y install autotools-dev
echo "INFO : (riscv-gnu-toolchain) install curl ---------------"; apt-get -y install curl
echo "INFO : (riscv-gnu-toolchain) install python3 ---------------"; apt-get -y install python3
echo "INFO : (riscv-gnu-toolchain) install python3-pip ---------------"; apt-get -y install python3-pip
echo "INFO : (riscv-gnu-toolchain) install libmpc-dev ---------------"; apt-get -y install libmpc-dev
echo "INFO : (riscv-gnu-toolchain) install libmpfr-dev ---------------"; apt-get -y install libmpfr-dev
echo "INFO : (riscv-gnu-toolchain) install libgmp-dev ---------------"; apt-get -y install libgmp-dev
echo "INFO : (riscv-gnu-toolchain) install gawk ---------------"; apt-get -y install gawk
echo "INFO : (riscv-gnu-toolchain) install build-essential ---------------"; apt-get -y install build-essential
echo "INFO : (riscv-gnu-toolchain) install bison ---------------"; apt-get -y install bison
echo "INFO : (riscv-gnu-toolchain) install flex ---------------"; apt-get -y install flex
echo "INFO : (riscv-gnu-toolchain) install texinfo ---------------"; apt-get -y install texinfo
echo "INFO : (riscv-gnu-toolchain) install gperf ---------------"; apt-get -y install gperf
echo "INFO : (riscv-gnu-toolchain) install libtool ---------------"; apt-get -y install libtool
echo "INFO : (riscv-gnu-toolchain) install patchutils ---------------"; apt-get -y install patchutils
echo "INFO : (riscv-gnu-toolchain) install bc ---------------"; apt-get -y install bc
echo "INFO : (riscv-gnu-toolchain) install zlib1g-dev ---------------"; apt-get -y install zlib1g-dev
echo "INFO : (riscv-gnu-toolchain) install libexpat-dev ---------------"; apt-get -y install libexpat-dev
echo "INFO : (riscv-gnu-toolchain) install ninja-build ---------------"; apt-get -y install ninja-build
echo "INFO : (riscv-gnu-toolchain) install git ---------------"; apt-get -y install git
echo "INFO : (riscv-gnu-toolchain) install cmake ---------------"; apt-get -y install cmake
echo "INFO : (riscv-gnu-toolchain) install libglib2.0-dev ---------------"; apt-get -y install libglib2.0-dev
# https://stackoverflow.com/questions/72987674/git-pull-error-rpc-failed-curl-16-error-in-the-http2-framing-layer-fatal-exp
echo "INFO : (riscv-gnu-toolchain) git config for make linux error -------"; git config --global http.version HTTP/1.1
echo "INFO : (riscv-gnu-toolchain) git clone riscv-gnu-toolchain ---------"; git clone https://github.com/riscv/riscv-gnu-toolchain
echo "INFO : (riscv-gnu-toolchain) cd riscv-gnu-toolchain/ -------"; cd riscv-gnu-toolchain/
echo "INFO : (riscv-gnu-toolchain) ./configure --prefix=/opt/riscv -------"; ./configure --prefix=/opt/riscv
echo "INFO : (riscv-gnu-toolchain) make linux -------"; export GIT_SSL_NO_VERIFY=1; make linux
echo "INFO : (Prerequisites) cd $DIR_BASE ------------------------"; cd $DIR_BASE
echo "INFO : (Prerequisites) install build-essential -----------------"; apt -y install build-essential
echo "INFO : (Prerequisites) install device-tree-compiler ------------"; apt -y install device-tree-compiler wget git python3-setuptools
echo "INFO : (Prerequisites) install wget -----------"; apt -y install wget
echo "INFO : (Prerequisites) install git -----------"; apt -y install git
echo "INFO : (Prerequisites) install python3-setuptools -----------"; apt -y install python3-setuptools
echo "INFO : (Prerequisites) git clone linux-on-litex-vexriscv -------"; git clone https://github.com/litex-hub/linux-on-litex-vexriscv
echo "INFO : (Pre-built Bitstreams and Linux/OpenSBI images) cd linux-on-litex-vexriscv/images -------"; cd $DIR_PJ/images
echo "INFO : (Pre-built Bitstreams and Linux/OpenSBI images) wget linux + opensbi -------"; wget https://github.com/litex-hub/linux-on-litex-vexriscv/files/8331338/linux_2022_03_23.zip
echo "INFO : (Pre-built Bitstreams and Linux/OpenSBI images) unzip opensbi_2020_12_15.zip -------"; unzip -o linux_2022_03_23.zip
echo "INFO : (Installing LiteX) cd $DIR_BASE ---------------------"; cd $DIR_BASE
echo "INFO : (Installing LiteX) wget litex_setup.py -------"; wget https://raw.githubusercontent.com/enjoy-digital/litex/master/litex_setup.py
echo "INFO : (Installing LiteX) chmod litex_setup.py -------"; chmod +x litex_setup.py
echo "INFO : (Installing LiteX) install python3-pip -------"; apt-get -y install python3-pip
echo "INFO : (Installing LiteX) litex_setup.py init install -------"; ./litex_setup.py --init --install --user
echo "INFO : (Installing Verilator) install verilator ------------------------------"; apt -y install verilator
echo "INFO : (Installing Verilator) install libevent-dev ------------------------------"; apt -y install libevent-dev
echo "INFO : (Installing Verilator) install libjson-c-dev ------------------------------"; apt -y install libjson-c-dev
echo "INFO : cd $DIR_BASE ---------------------------------------------------------------"; cd $DIR_BASE
echo "INFO : (Installing OpenOCD) install libtool -----------------------------"; apt -y install libtool
echo "INFO : (Installing OpenOCD) install automake -----------------------------"; apt -y install automake
echo "INFO : (Installing OpenOCD) install pkg-config -----------------------------"; apt -y install pkg-config
echo "INFO : (Installing OpenOCD) install libusb-1.0-0-dev -----------------------------"; apt -y install libusb-1.0-0-dev
echo "INFO : (Installing OpenOCD) git clone openocd.git -----------------------------"; git clone https://github.com/ntfreak/openocd.git
echo "INFO : (Installing OpenOCD) cd openocd -----------------------------"; cd openocd
echo "INFO : (Installing OpenOCD) ./bootstrap -----------------------------"; ./bootstrap
echo "INFO : (Installing OpenOCD) ./configure --enable-ftdi -----------------------------"; ./configure --enable-ftdi
echo "INFO : (Installing OpenOCD) make -----------------------------"; make
echo "INFO : (Installing OpenOCD) make install -----------------------------"; make install
echo "INFO : cd $DIR_BASE ---------------------------------------------------------------"; cd $DIR_BASE
echo "INFO : (sbt for sim.py) tee /etc/apt/sources.list.d/sbt.list ------------------"; echo "deb https://repo.scala-sbt.org/scalasbt/debian all main" | sudo tee /etc/apt/sources.list.d/sbt.list
echo "INFO : (sbt for sim.py) tee /etc/apt/sources.list.d/sbt_old.list ------------------"; echo "deb https://repo.scala-sbt.org/scalasbt/debian /" | sudo tee /etc/apt/sources.list.d/sbt_old.list
echo "INFO : (sbt for sim.py) curl keyserver.ubuntu.com/pks & apt-key add --------------"; curl -sL "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x2EE0EA64E40A89B84B2DF73499E82A75642AC823" | sudo apt-key add
echo "INFO : (sbt for sim.py) update ----------------------------------------------"; apt-get -y update
echo "INFO : (sbt for sim.py) install sbt ----------------------------------------------"; apt-get -y install sbt
echo "INFO : cd $DIR_BASE ---------------------------------------------------------------"; cd $DIR_BASE
echo "INFO : (java by sdkman for sim.py) install sdkman --------------------------------"; curl -s "https://get.sdkman.io" | bash
echo "INFO : (java by sdkman for sim.py) chmod 744 sdk --------------------------------"; chmod 744 /root/.sdkman/contrib/completion/bash/sdk
echo "INFO : (java by sdkman for sim.py) source sdkman-init.sh --------------------------"; source /root/.sdkman/bin/sdkman-init.sh
echo "INFO : (java by sdkman for sim.py) install java --------------------------------"; sdk install java $(sdk list java | grep -o "\b8\.[0-9]*\.[0-9]*\-tem" | head -1)
echo "INFO : (meson for sim.py) upgrade pip ----------------------------------------"; python3 -m pip install --upgrade pip
echo "INFO : (meson for sim.py) install meson ----------------------------------------"; pip3 install meson
echo "INFO : cd $DIR_PJ --------------------"; cd $DIR_PJ
# to prevent from following error.
# [error] [launcher] error during sbt launcher: java.lang.OutOfMemoryError: GC overhead limit exceeded
# https://support.snyk.io/hc/en-us/articles/360003143417-Out-of-Memory-Error-when-testing-Scala-sbt-project
echo "INFO : (Build the FPGA bitstream) SBT limit setting --------------------"; export SBT_OPTS="-Xms1024M -Xmx4G -Xss2M -XX:MaxMetaspaceSize=2G"
echo "INFO : (Build the FPGA bitstream) build bitstream --------------------"; ./make.py --board=arty --variant=a7-100 --cpu-count=1 --build
echo "INFO : (Running the LiteX simulation) sim.py --------------------"; ./sim.py
# Arty A7ボードをPCとUSBケーブルで接続してるなら、以下も実行可能。
# echo "INFO : (Load the FPGA bitstream) load bitstream --------------------"; ./make.py --board=arty --variant=a7-100 --cpu-count=1 --load
# echo "INFO : (Load the Linux images over Serial) load linux via uart --------------------"; python3 litex_term.py --images=images/boot.json /dev/ttyUSB1
スクリプト実行ログ
最後に実行したsim.pyの方のログ。
：
[clocker] sys_clk: freq_hz=1000000, phase_deg=0
__ _ __ _ __
/ / (_) /____ | |/_/
/ /__/ / __/ -_)> <
/____/_/\__/\__/_/|_|
Build your hardware, easily!
(c) Copyright 2012-2023 Enjoy-Digital
(c) Copyright 2007-2015 M-Labs
BIOS built on Oct 15 2023 14:02:35
BIOS CRC passed (0aaad8a5)
LiteX git sha1: e499dd84
--=============== SoC ==================--
CPU: VexRiscv SMP-LINUX @ 100MHz
BUS: WISHBONE 32-bit @ 4GiB
CSR: 32-bit data
ROM: 64.0KiB
SRAM: 8.0KiB
SDRAM: 64.0MiB 32-bit @ 100MT/s (CL-2 CWL-2)
MAIN-RAM: 64.0MiB
--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
Executing booted program at 0x40f00000
--============= Liftoff! ===============--
OpenSBI v0.8-1-gecf7701
____ _____ ____ _____
/ __ \ / ____| _ \_ _|
| | | |_ __ ___ _ __ | (___ | |_) || |
| | | | '_ \ / _ \ '_ \ \___ \| _ < | |
| |__| | |_) | __/ | | |____) | |_) || |_
\____/| .__/ \___|_| |_|_____/|____/_____|
| |
|_|
Platform Name : LiteX / VexRiscv-SMP
Platform Features : timer,mfdeleg
Platform HART Count : 8
Boot HART ID : 0
Boot HART ISA : rv32imas
BOOT HART Features : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base : 0x40f00000
Firmware Size : 124 KB
Runtime SBI Version : 0.2
MIDELEG : 0x00000222
MEDELEG : 0x0000b109
[ 0.000000] Linux version 5.14.0 (florent@panda) (riscv32-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.08-381-g279167ee8d) 10.3.0, GNU ld (GNU Binutils) 2.36.1) #1 SMP Tue Sep 21 12:57:31 CEST 2021
[ 0.000000] earlycon: liteuart0 at I/O port 0x0 (options '')
[ 0.000000] Malformed early option 'console'
[ 0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '')
[ 0.000000] printk: bootconsole [liteuart0] enabled
[ 0.000000] Zone ranges:
[ 0.000000] Normal [mem 0x0000000040000000-0x0000000043ffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000040000000-0x0000000043ffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x0000000043ffffff]
[ 0.000000] SBI specification v0.2 detected
[ 0.000000] SBI implementation ID=0x1 Version=0x8
[ 0.000000] SBI TIME extension detected
[ 0.000000] SBI IPI extension detected
[ 0.000000] SBI RFENCE extension detected
[ 0.000000] SBI v0.2 HSM extension detected
[ 0.000000] riscv: ISA extensions aimp
[ 0.000000] riscv: ELF capabilities aim
[ 0.000000] percpu: Embedded 8 pages/cpu s11340 r0 d21428 u32768
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 16256
[ 0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0
[ 0.000000] Dentry cache hash table entries: 8192 (order: 3, 32768 bytes, linear)
[ 0.000000] Inode-cache hash table entries: 4096 (order: 2, 16384 bytes, linear)
[ 0.000000] Sorting __ex_table...
[ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.000000] Memory: 48648K/65536K available (5685K kernel code, 572K rwdata, 883K rodata, 209K init, 221K bss, 16888K reserved, 0K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] rcu: Hierarchical RCU implementation.
[ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] riscv-intc: 32 local interrupts mapped
[ 0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts.
[ 0.000000] random: get_random_bytes called from start_kernel+0x4ac/0x63c with crng_init=0
[ 0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x171024e7e0, max_idle_ns: 440795205315 ns
[ 0.000017] sched_clock: 64 bits at 100MHz, resolution 10ns, wraps every 4398046511100ns
[ 0.002154] Console: colour dummy device 80x25
[ 0.003085] Calibrating delay loop (skipped), value calculated using timer frequency.. 200.00 BogoMIPS (lpj=400000)
[ 0.004624] pid_max: default: 32768 minimum: 301
[ 0.008050] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[ 0.009087] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[ 0.029602] ASID allocator using 9 bits (512 entries)
[ 0.032202] rcu: Hierarchical SRCU implementation.
[ 0.037667] smp: Bringing up secondary CPUs ...
[ 0.038228] smp: Brought up 1 node, 1 CPU
[ 0.043635] devtmpfs: initialized
[ 0.066964] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.068287] futex hash table entries: 256 (order: 2, 16384 bytes, linear)
[ 0.075961] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.219089] pps_core: LinuxPPS API ver. 1 registered
[ 0.219624] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.221051] PTP clock support registered
[ 0.224211] FPGA manager framework
[ 0.237389] clocksource: Switched to clocksource riscv_clocksource
[ 0.388115] NET: Registered PF_INET protocol family
[ 0.390255] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
[ 0.397785] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
[ 0.399109] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
[ 0.400341] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear)
[ 0.401563] TCP: Hash tables configured (established 1024 bind 1024)
[ 0.402993] UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
[ 0.404087] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
[ 0.421832] Unpacking initramfs...
[ 0.457634] workingset: timestamp_bits=30 max_order=14 bucket_order=0
[ 0.666444] io scheduler mq-deadline registered
[ 0.667037] io scheduler kyber registered
[ 0.918171] LiteX SoC Controller driver initialized
[ 4.166205] Freeing initrd memory: 8192K
[ 4.659111] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart
[ 4.660393] printk: console [liteuart0] enabled
[ 4.660393] printk: console [liteuart0] enabled
[ 4.661316] printk: bootconsole [liteuart0] disabled
[ 4.661316] printk: bootconsole [liteuart0] disabled
[ 4.682040] i2c_dev: i2c /dev entries driver
[ 4.719192] NET: Registered PF_INET6 protocol family
[ 4.732729] Segment Routing with IPv6
[ 4.733825] In-situ OAM (IOAM) with IPv6
[ 4.735364] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 4.748263] NET: Registered PF_PACKET protocol family
[ 4.759811] Freeing unused kernel image (initmem) memory: 204K
[ 4.760589] Kernel memory protection not selected by kernel config.
[ 4.761872] Run /init as init process
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Saving random seed: [ 6.829806] random: dd: uninitialized urandom read (512 bytes read)
OK
Starting network: OK
Welcome to Buildroot
buildroot login: root ※ ここでrootと入力してENTER（シミュレーションなのでだいぶ遅い）
__ _
/ / (_)__ __ ____ __
/ /__/ / _ \/ // /\ \ /
/____/_/_//_/\_,_//_\_\
/ _ \/ _ \
__ _ __ _ _\___/_//_/ ___ _
/ / (_) /____ | |/_/__| | / /____ __ / _ \(_)__ _____ __
/ /__/ / __/ -_)> </___/ |/ / -_) \ // , _/ (_-</ __/ |/ /
/____/_/\__/\__/_/|_|____|___/\__/_\_\/_/|_/_/___/\__/|___/
/ __/ |/ / _ \
_\ \/ /|_/ / ___/
/___/_/ /_/_/
32-bit RISC-V Linux running on LiteX / VexRiscv-SMP.
login[70]: root login on 'console'
root@buildroot:~# pwd ※ 試しにpwdコマンド実行したら効いた
/root
root@buildroot:~#
これでVivadoを事前にインストールしておけば、スクリプト一発でLiteXがお試し出来た。