Skip to content

Commit 7243604

Browse files
committed
Add LLM installer script
1 parent a3eb148 commit 7243604

1 file changed

Lines changed: 23 additions & 0 deletions

File tree

llm/install.sh

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#!/usr/bin/env sh
2+
3+
# Nix is required for llama.cpp
4+
if ! command -v nix &> /dev/null; then
5+
echo "Installing Nix."
6+
sh <(curl --proto '=https' --tlsv1.2 -L https://nixos.org/nix/install) --daemon
7+
fi
8+
9+
if ! command -v ollama &> /dev/null; then
10+
echo "Installing Ollama."
11+
curl -fsSL https://ollama.com/install.sh | sh
12+
fi
13+
14+
nix profile install nixpkgs#llama-cpp --extra-experimental-features nix-command --extra-experimental-features flakes
15+
16+
which llama-server
17+
llama-server --version
18+
19+
which llama-cli
20+
llama-cli -hf Qwen/Qwen2.5-7B-Instruct-GGUF
21+
22+
llama-bench --list-devices
23+
llama-bench --model ${HOME}/.cache/llama.cpp/Qwen_Qwen2.5-7B-Instruct-GGUF_qwen2.5-7b-instruct-q2_k.gguf

0 commit comments

Comments
 (0)