LoginSignup
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

@7of9

2021-12-28 「AXI CDMA Linux user space example on Zynq UltraScale+ RFSoC」をXSDK実装

Last updated at Posted at 2021-12-28

動作環境

  • Windows 10 Pro (v21H2)
  • Vivado v2019.1
  • FPGA基板: CORA Z7 Zynq－7000評価ボード（Z7－07S）

概要

AXI CDMA Linux user space example on Zynq UltraScale+ RFSoC
https://support.xilinx.com/s/article/1096735?language=ja

上記のサンプルはBRAM間をDMA転送するというシンプルなブロックであるため、DMAの勉強には良い資料だと思った。
一方で、Linux動作のため、XSDKのベアメタルですぐに試すことができない。

そのため、上記のLinux動作の実装をXSDK実装に組み換えすることにした。

AXI CDMA

PG034の詳細はまだ見ていないが、Memory MappedのSourceとDestination間をDMA転送する時に使うことができる。
（一方で、AXI DMA IPはAXI4-Stream間の転送になると理解している)

Block Design

参照元はZynq UltraScale+ RFSoC向けであるが、それと同等のものをZynq 7000向けに作り直した。

image.png (98.6 kB)

Address Editor

image.png (16.9 kB)

AXI CDMA設定

image.png (32.8 kB)

XSDK

Linux実装やsystem.mss生成のコードなど様々なものを参考に組み上げた。
Linuxではmmap()を使って仮想アドレスと物理アドレスを紐つけていたりしたが、そのあたりは別のXSDK向けの実装をもとにポインタ操作でできた。

helloworld.c
# include <stdio.h>
# include "platform.h"
# include "xil_printf.h"
# include "xparameters.h"
# include "xbram.h"
# include "xaxicdma.h"
# include "xil_io.h"

//#define BUFFER_BYTESIZE  (1024) // Length of the buffers for DMA transfer from DDR to BRAM
//#define MAP_SIZE (4096)

//#define DDR_MAP_SIZE (0x10000)
//#define DDR_MAP_MASK (DDR_MAP_SIZE - 1)


XBram Bram; // Bramドライバのインスタンス
# define BRAM_DEVICE_1_ID		(XPAR_AXI_BRAM_CTRL_1_DEVICE_ID)
# define CDMA_DEVICE_ID (XPAR_AXICDMA_0_DEVICE_ID)

# define BRAM_BASE  (XPAR_BRAM_0_BASEADDR)
# define PSDDR_BASE (XPAR_PS7_DDR_0_S_AXI_BASEADDR)
# define CDMA_BASE  (XPAR_AXICDMA_0_BASEADDR)

int main()
{
    init_platform();

//	XBram_Config *BramConfigPtr1;
//	XBram BramInst1; // インスタンス用

	XAxiCdma_Config *CdmaConfig;
	XAxiCdma CdmaInst; // インスタンス用

	int Status;

    printf("Hello 1117\r\n");


	// 1. Init ====================================

    // axi_bram_ctrl_0はAXI CDMAとつながっているため、PS側には見えていない？
    // そのためxparameters.hには関連するマクロ定義がないものと思われる。

    // そもそも以下のBRAMの定義や操作は必要ないか (メモリ読み出しのため不要?)

    //   BRAM
//	BramConfigPtr1 = XBram_LookupConfig(BRAM_DEVICE_1_ID);
//	if (BramConfigPtr1 == (XBram_Config *)NULL) {
//		printf("BRAM1 Config error\r\n");
//		return XST_FAILURE;
//	}
//	Status = XBram_CfgInitialize(&BramInst1, BramConfigPtr1, BramConfigPtr1->CtrlBaseAddress);
//	if (Status != XST_SUCCESS) {
//		printf("BRAM1 CfgInit error\r\n");
//		return XST_FAILURE;
//	}
//	printf("BRAM1 Init Pass\r\n");


    //   CDMA
	CdmaConfig = XAxiCdma_LookupConfig(CDMA_DEVICE_ID);
	if (CdmaConfig == (XAxiCdma_Config *)NULL) {
		printf("CDMA Config error\r\n");
		return XST_FAILURE;
	}
	Status = XAxiCdma_CfgInitialize(&CdmaInst, CdmaConfig, CDMA_BASE);
	if (Status != XST_SUCCESS) {
		printf("CDMA CfgInit error\r\n");
		return XST_FAILURE;
	}
	printf("CDMA Init Pass\r\n");


	// 2. Write to PS DDR ===========================

    u32 *psddrs = (u32 *)PSDDR_BASE; // 配列アクセス用
    u32 *brams = (u32 *)BRAM_BASE; // 配列アクセス用
	for(int idx = 0; idx < 100; idx++) {
		psddrs[idx] = idx + 10; // <======= 代入したい値 (任意の式を設定すればいい) ============

		printf("Input : value in PSDDR %ld : value in BRAM %ld\r\n", psddrs[idx], brams[idx]);
	}

	// *** データキャッシュがあることも考慮して、ここでFlash操作が必要 ***
	Xil_DCacheFlushRange((u32)psddrs, /*Length=*/100 * sizeof(u32));
//#ifdef __aarch64__
//	Xil_DCacheFlushRange((UINTPTR)&brams, /*Length=*/100);
//#endif


	// 3. DMA Transfer > 設定 =================================

	//   3.1 制御レジスタ、ステータスレジスタを見る
	unsigned int reg    = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_CR_OFFSET);
	unsigned int status = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_SR_OFFSET);
	printf("control reg:0x%08x\n", reg);
    printf("status reg:0x%08x\n", status);

    //   3.2 割り込みマスク
    printf("all interrupts masked...\n");
    XAxiCdma_WriteReg(CdmaConfig->BaseAddress, XAXICDMA_CR_OFFSET, XAXICDMA_XR_IRQ_SIMPLE_ALL_MASK);
	reg    = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_CR_OFFSET);
	status = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_SR_OFFSET);
    printf("control reg:0x%08x\n", reg);
    printf("status reg:0x%08x\n", status);

    //   3.3 送信元アドレス設定 (PS DDR)
    printf("Writing source address\n");
    XAxiCdma_WriteReg(CdmaConfig->BaseAddress, XAXICDMA_SRCADDR_OFFSET, PSDDR_BASE);
	reg    = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_SRCADDR_OFFSET);
	status = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_SR_OFFSET);
    printf("Source addr reg:0x%08x\n", reg);
    printf("status reg:0x%08x\n", status);

    //   3.4 送信先アドレス設定 (BRAM)
    printf("Writing destination address\n");
    XAxiCdma_WriteReg(CdmaConfig->BaseAddress, XAXICDMA_DSTADDR_OFFSET, BRAM_BASE);
	reg    = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_DSTADDR_OFFSET);
	status = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_SR_OFFSET);
    printf("Dest addr reg:0x%08x\n", reg);
    printf("status reg:0x%08x\n", status);

    //   3.5 送信長さ設定
    printf("Writing transfer length...\n");
    //dma_set(vadd_cdma, XAXICDMA_BTT_OFFSET, 0x190); // write length register
    XAxiCdma_WriteReg(CdmaConfig->BaseAddress, XAXICDMA_BTT_OFFSET, 0x190); // write length register (0x190 = 400)


	// 4. DMA Transfer > 転送 =================================

    while(1) {
       status = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_SR_OFFSET);
       if(status & (1 << 12)) {
          printf("DMA transfer is completed \n");
          break;
       }

    }

 	reg = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_BTT_OFFSET);
	status = XAxiCdma_ReadReg(CdmaConfig->BaseAddress, XAXICDMA_SR_OFFSET);
    printf("length reg:0x%08x\n", reg);
    printf("status reg:0x%08x\n", status);


    // *** キャッシュ中のデータが古いので、以下でDRAMから読みなおす ***
    Xil_DCacheInvalidateRange((u32)brams, 100 * sizeof(u32));


	// 5. Check PS DDR and BRAM ===========================

    int count = 0;
	for(int idx = 0; idx < 100; idx++) {
		printf("Result : value[%d] in PSDDR %ld : value in BRAM %ld\r\n", idx, psddrs[idx], brams[idx]);
		if (psddrs[idx] != brams[idx]) {
			count++;
		}
	}


	// 9. Cleanup =================================

    cleanup_platform();
    return 0;
}

実行

Hello 1117
CDMA Init Pass
Input : value in PSDDR 10 : value in BRAM 0
Input : value in PSDDR 11 : value in BRAM 2
Input : value in PSDDR 12 : value in BRAM 4
Input : value in PSDDR 13 : value in BRAM 6
Input : value in PSDDR 14 : value in BRAM 8
Input : value in PSDDR 15 : value in BRAM 10
Input : value in PSDDR 16 : value in BRAM 12
Input : value in PSDDR 17 : value in BRAM 14
Input : value in PSDDR 18 : value in BRAM 16
Input : value in PSDDR 19 : value in BRAM 18
Input : value in PSDDR 20 : value in BRAM 20
Input : value in PSDDR 21 : value in BRAM 22
Input : value in PSDDR 22 : value in BRAM 24
Input : value in PSDDR 23 : value in BRAM 26
Input : value in PSDDR 24 : value in BRAM 28
Input : value in PSDDR 25 : value in BRAM 30
Input : value in PSDDR 26 : value in BRAM 32
Input : value in PSDDR 27 : value in BRAM 34
Input : value in PSDDR 28 : value in BRAM 36
Input : value in PSDDR 29 : value in BRAM 38
Input : value in PSDDR 30 : value in BRAM 40
Input : value in PSDDR 31 : value in BRAM 42
Input : value in PSDDR 32 : value in BRAM 44
Input : value in PSDDR 33 : value in BRAM 46
Input : value in PSDDR 34 : value in BRAM 48
Input : value in PSDDR 35 : value in BRAM 50
Input : value in PSDDR 36 : value in BRAM 52
Input : value in PSDDR 37 : value in BRAM 54
Input : value in PSDDR 38 : value in BRAM 56
Input : value in PSDDR 39 : value in BRAM 58
Input : value in PSDDR 40 : value in BRAM 60
Input : value in PSDDR 41 : value in BRAM 62
Input : value in PSDDR 42 : value in BRAM 64
Input : value in PSDDR 43 : value in BRAM 66
Input : value in PSDDR 44 : value in BRAM 68
Input : value in PSDDR 45 : value in BRAM 70
Input : value in PSDDR 46 : value in BRAM 72
Input : value in PSDDR 47 : value in BRAM 74
Input : value in PSDDR 48 : value in BRAM 76
Input : value in PSDDR 49 : value in BRAM 78
Input : value in PSDDR 50 : value in BRAM 80
Input : value in PSDDR 51 : value in BRAM 82
Input : value in PSDDR 52 : value in BRAM 84
Input : value in PSDDR 53 : value in BRAM 86
Input : value in PSDDR 54 : value in BRAM 88
Input : value in PSDDR 55 : value in BRAM 90
Input : value in PSDDR 56 : value in BRAM 92
Input : value in PSDDR 57 : value in BRAM 94
Input : value in PSDDR 58 : value in BRAM 96
Input : value in PSDDR 59 : value in BRAM 98
Input : value in PSDDR 60 : value in BRAM 100
Input : value in PSDDR 61 : value in BRAM 102
Input : value in PSDDR 62 : value in BRAM 104
Input : value in PSDDR 63 : value in BRAM 106
Input : value in PSDDR 64 : value in BRAM 108
Input : value in PSDDR 65 : value in BRAM 110
Input : value in PSDDR 66 : value in BRAM 112
Input : value in PSDDR 67 : value in BRAM 114
Input : value in PSDDR 68 : value in BRAM 116
Input : value in PSDDR 69 : value in BRAM 118
Input : value in PSDDR 70 : value in BRAM 120
Input : value in PSDDR 71 : value in BRAM 122
Input : value in PSDDR 72 : value in BRAM 124
Input : value in PSDDR 73 : value in BRAM 126
Input : value in PSDDR 74 : value in BRAM 128
Input : value in PSDDR 75 : value in BRAM 130
Input : value in PSDDR 76 : value in BRAM 132
Input : value in PSDDR 77 : value in BRAM 134
Input : value in PSDDR 78 : value in BRAM 136
Input : value in PSDDR 79 : value in BRAM 138
Input : value in PSDDR 80 : value in BRAM 140
Input : value in PSDDR 81 : value in BRAM 142
Input : value in PSDDR 82 : value in BRAM 144
Input : value in PSDDR 83 : value in BRAM 146
Input : value in PSDDR 84 : value in BRAM 148
Input : value in PSDDR 85 : value in BRAM 150
Input : value in PSDDR 86 : value in BRAM 152
Input : value in PSDDR 87 : value in BRAM 154
Input : value in PSDDR 88 : value in BRAM 156
Input : value in PSDDR 89 : value in BRAM 158
Input : value in PSDDR 90 : value in BRAM 160
Input : value in PSDDR 91 : value in BRAM 162
Input : value in PSDDR 92 : value in BRAM 164
Input : value in PSDDR 93 : value in BRAM 166
Input : value in PSDDR 94 : value in BRAM 168
Input : value in PSDDR 95 : value in BRAM 170
Input : value in PSDDR 96 : value in BRAM 172
Input : value in PSDDR 97 : value in BRAM 174
Input : value in PSDDR 98 : value in BRAM 176
Input : value in PSDDR 99 : value in BRAM 178
Input : value in PSDDR 100 : value in BRAM 180
Input : value in PSDDR 101 : value in BRAM 182
Input : value in PSDDR 102 : value in BRAM 184
Input : value in PSDDR 103 : value in BRAM 186
Input : value in PSDDR 104 : value in BRAM 188
Input : value in PSDDR 105 : value in BRAM 190
Input : value in PSDDR 106 : value in BRAM 192
Input : value in PSDDR 107 : value in BRAM 194
Input : value in PSDDR 108 : value in BRAM 196
Input : value in PSDDR 109 : value in BRAM 198
control reg:0x00000000
status reg:0x00000002
all interrupts masked...
control reg:0x00005000
status reg:0x00000002
Writing source address
Source addr reg:0x00100000
status reg:0x00000002
Writing destination address
Dest addr reg:0x60000000
status reg:0x00000002
Writing transfer length...
DMA transfer is completed
length reg:0x00000190
status reg:0x00001002
Result : value[0] in PSDDR 10 : value in BRAM 10
Result : value[1] in PSDDR 11 : value in BRAM 11
Result : value[2] in PSDDR 12 : value in BRAM 12
Result : value[3] in PSDDR 13 : value in BRAM 13
Result : value[4] in PSDDR 14 : value in BRAM 14
Result : value[5] in PSDDR 15 : value in BRAM 15
Result : value[6] in PSDDR 16 : value in BRAM 16
Result : value[7] in PSDDR 17 : value in BRAM 17
Result : value[8] in PSDDR 18 : value in BRAM 18
Result : value[9] in PSDDR 19 : value in BRAM 19
Result : value[10] in PSDDR 20 : value in BRAM 20
Result : value[11] in PSDDR 21 : value in BRAM 21
Result : value[12] in PSDDR 22 : value in BRAM 22
Result : value[13] in PSDDR 23 : value in BRAM 23
Result : value[14] in PSDDR 24 : value in BRAM 24
Result : value[15] in PSDDR 25 : value in BRAM 25
Result : value[16] in PSDDR 26 : value in BRAM 26
Result : value[17] in PSDDR 27 : value in BRAM 27
Result : value[18] in PSDDR 28 : value in BRAM 28
Result : value[19] in PSDDR 29 : value in BRAM 29
Result : value[20] in PSDDR 30 : value in BRAM 30
Result : value[21] in PSDDR 31 : value in BRAM 31
Result : value[22] in PSDDR 32 : value in BRAM 32
Result : value[23] in PSDDR 33 : value in BRAM 33
Result : value[24] in PSDDR 34 : value in BRAM 34
Result : value[25] in PSDDR 35 : value in BRAM 35
Result : value[26] in PSDDR 36 : value in BRAM 36
Result : value[27] in PSDDR 37 : value in BRAM 37
Result : value[28] in PSDDR 38 : value in BRAM 38
Result : value[29] in PSDDR 39 : value in BRAM 39
Result : value[30] in PSDDR 40 : value in BRAM 40
Result : value[31] in PSDDR 41 : value in BRAM 41
Result : value[32] in PSDDR 42 : value in BRAM 42
Result : value[33] in PSDDR 43 : value in BRAM 43
Result : value[34] in PSDDR 44 : value in BRAM 44
Result : value[35] in PSDDR 45 : value in BRAM 45
Result : value[36] in PSDDR 46 : value in BRAM 46
Result : value[37] in PSDDR 47 : value in BRAM 47
Result : value[38] in PSDDR 48 : value in BRAM 48
Result : value[39] in PSDDR 49 : value in BRAM 49
Result : value[40] in PSDDR 50 : value in BRAM 50
Result : value[41] in PSDDR 51 : value in BRAM 51
Result : value[42] in PSDDR 52 : value in BRAM 52
Result : value[43] in PSDDR 53 : value in BRAM 53
Result : value[44] in PSDDR 54 : value in BRAM 54
Result : value[45] in PSDDR 55 : value in BRAM 55
Result : value[46] in PSDDR 56 : value in BRAM 56
Result : value[47] in PSDDR 57 : value in BRAM 57
Result : value[48] in PSDDR 58 : value in BRAM 58
Result : value[49] in PSDDR 59 : value in BRAM 59
Result : value[50] in PSDDR 60 : value in BRAM 60
Result : value[51] in PSDDR 61 : value in BRAM 61
Result : value[52] in PSDDR 62 : value in BRAM 62
Result : value[53] in PSDDR 63 : value in BRAM 63
Result : value[54] in PSDDR 64 : value in BRAM 64
Result : value[55] in PSDDR 65 : value in BRAM 65
Result : value[56] in PSDDR 66 : value in BRAM 66
Result : value[57] in PSDDR 67 : value in BRAM 67
Result : value[58] in PSDDR 68 : value in BRAM 68
Result : value[59] in PSDDR 69 : value in BRAM 69
Result : value[60] in PSDDR 70 : value in BRAM 70
Result : value[61] in PSDDR 71 : value in BRAM 71
Result : value[62] in PSDDR 72 : value in BRAM 72
Result : value[63] in PSDDR 73 : value in BRAM 73
Result : value[64] in PSDDR 74 : value in BRAM 74
Result : value[65] in PSDDR 75 : value in BRAM 75
Result : value[66] in PSDDR 76 : value in BRAM 76
Result : value[67] in PSDDR 77 : value in BRAM 77
Result : value[68] in PSDDR 78 : value in BRAM 78
Result : value[69] in PSDDR 79 : value in BRAM 79
Result : value[70] in PSDDR 80 : value in BRAM 80
Result : value[71] in PSDDR 81 : value in BRAM 81
Result : value[72] in PSDDR 82 : value in BRAM 82
Result : value[73] in PSDDR 83 : value in BRAM 83
Result : value[74] in PSDDR 84 : value in BRAM 84
Result : value[75] in PSDDR 85 : value in BRAM 85
Result : value[76] in PSDDR 86 : value in BRAM 86
Result : value[77] in PSDDR 87 : value in BRAM 87
Result : value[78] in PSDDR 88 : value in BRAM 88
Result : value[79] in PSDDR 89 : value in BRAM 89
Result : value[80] in PSDDR 90 : value in BRAM 90
Result : value[81] in PSDDR 91 : value in BRAM 91
Result : value[82] in PSDDR 92 : value in BRAM 92
Result : value[83] in PSDDR 93 : value in BRAM 93
Result : value[84] in PSDDR 94 : value in BRAM 94
Result : value[85] in PSDDR 95 : value in BRAM 95
Result : value[86] in PSDDR 96 : value in BRAM 96
Result : value[87] in PSDDR 97 : value in BRAM 97
Result : value[88] in PSDDR 98 : value in BRAM 98
Result : value[89] in PSDDR 99 : value in BRAM 99
Result : value[90] in PSDDR 100 : value in BRAM 100
Result : value[91] in PSDDR 101 : value in BRAM 101
Result : value[92] in PSDDR 102 : value in BRAM 102
Result : value[93] in PSDDR 103 : value in BRAM 103
Result : value[94] in PSDDR 104 : value in BRAM 104
Result : value[95] in PSDDR 105 : value in BRAM 105
Result : value[96] in PSDDR 106 : value in BRAM 106
Result : value[97] in PSDDR 107 : value in BRAM 107
Result : value[98] in PSDDR 108 : value in BRAM 108
Result : value[99] in PSDDR 109 : value in BRAM 109

PS DDRとBRAMの値が一致している（DMA転送が成功している）。
PS DDRへの代入式を変更しながら動作確認したが、繰返して動作に成功しているようだ。

ハマったところ

Xil_DCacheFlushRange()が必要

Xil_DCacheFlushRange()がなかったことでPS DDRに代入した値がDMA転送されてなかった

参考: https://thuruthurutoru.hatenablog.com/entry/2015/11/14/003305

重要なのが最後の Xil_DCacheFlushRange関数の実行．
この関数で明示的にキャッシュフラッシュをしないと，転送元のバッファにセットしたデータDRAMまで書き込まれず，DMA転送実行後の結果が意図した通りにならない．
最初はキャッシュの存在を意識していなかったため，データが化ける？？？？？としばらく悩んだが，キャッシュをフラッシュすることで解決．

Xil_DCacheInvalidateRange()が必要

Xil_DCacheInvalidateRange()がなかったことで、BRAMにはキャッシュの古いデータが残っていた?

参考: http://nahitafu.cocolog-nifty.com/nahitafu/2013/11/zynqfpgaplarmps.html

DMAで転送されたデータのCPUから読み出すには、Xil_DCacheInvalidateRangeという関数を使って、キャッシュの中のデータが古いからDRAMから読み直すようにという指示を与えなければなりません。そうしないと、せっかくDMAで更新されたデータではなく古いキャッシュの内容が読み出されてしまいます。

参考: https://dora.bk.tsukuba.ac.jp/~takeuchi/?%E9%9B%BB%E6%B0%97%E5%9B%9E%E8%B7%AF%2Fzynq%2FDMA%E5%87%A6%E7%90%86

Xilinx の XAxiDma ドライバのサンプルを読む
...

  1. Xil_DCacheInvalidateRange で受信バッファのキャッシュ内容を無効化

関連情報

上にあげたリンクも含めてここにまとめておく。

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?