LoginSignup
0
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@kabayan

ollamaでGPUメモリ余ってるのにCPU使ってしまい遅い件

Posted at
  • GPUメモリが余ってるのにCPUばかり使うときはnum_gpuを増やすとよい（かも）。
  • リニアに増えないことに注意。

例）

FROM MrScarySpaceCat/gemma3-tools:4b
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER top_k 40
# PARAMETER num_gpu 40 4831 6s
# PARAMETER num_gpu 30 2411 40s
# PARAMETER num_gpu 32 2507 40s
# PARAMETER num_gpu 34 2653 40s
# PARAMETER num_gpu 36 4831 6s
PARAMETER num_gpu 36
0
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?