don
/
neon-sprawl
mirror of https://github.com/ViPro-Technologies/neon-sprawl
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

chore: document MLX model alternatives and PATH fix 

Expand model selection with Qwen3, Gemma, DeepSeek, and Llama options;
add ~/.local/bin troubleshooting; rename runtime alternatives section.

Co-authored-by: Cursor <cursoragent@cursor.com>
docs/local-mlx-cursor
Don 2026-05-27 20:20:51 -04:00
parent 294c994f20
commit e0b19a4367
1 changed files with 52 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
docs/dev/local-mlx-cursor.md
View File

@ -70,8 +70,18 @@ mlx_lm.server --help
which mlx_lm.server # note path for LaunchAgent below
```
If `which mlx_lm.server` prints **not found** after `uv tool install mlx-lm`, the binary is usually at `~/.local/bin/mlx_lm.server` but that directory is not on your `PATH`. Add to `~/.zshrc`:
```bash
export PATH="$HOME/.local/bin:$PATH"
```
Then `source ~/.zshrc`, or run `~/.local/bin/mlx_lm.server --help` directly.
## 2. Choose a model
### Defaults (recommended starting point)
| Role | Hugging Face repo id | Approx. RAM |
|------|----------------------|-------------|
| **Primary coder (recommended)** | `mlx-community/Qwen2.5-Coder-32B-Instruct-4bit` | ~18–22 GB |
@ -82,6 +92,43 @@ For Neon Sprawl (C# server tests, GDScript client, Bruno API collections), prefe
Models download on first use into the Hugging Face cache.
### Model alternatives
Qwen **Coder** models are a strong default for local codegen on Mac, but they are not the only worthwhile option. Rankings from public benchmarks (e.g. SWE-bench) change frequently; **try models on your own repo** before switching defaults.
| Family | Example MLX repo id | Best for | Tradeoff vs Qwen2.5-Coder-32B |
|--------|---------------------|----------|-------------------------------|
| **Qwen2.5 Coder** (default above) | `mlx-community/Qwen2.5-Coder-32B-Instruct-4bit` | Stable, proven Cursor + MLX path | Baseline — not always newest benchmark leader |
| **Qwen3 Coder** | `mlx-community/Qwen3-Coder-Next-mxfp4` | Newer coding benchmarks; MoE efficiency on large Macs | Less “battle-tested” in guides; confirm id via `/v1/models` |
| **Qwen3 Coder (MoE)** | Search HF for `Qwen3-Coder-30B-A3B-Instruct-MLX` (e.g. lmstudio-community builds) | Strong code quality with fewer active params than dense 30B | Repo id varies by publisher — verify on [Hugging Face](https://huggingface.co/models?search=mlx+qwen3+coder) |
| **Gemma 4** | Search HF for `Gemma-4` + `MLX` + `4bit` | Local assistant balance; Apache 2.0 | Some 2026 comparisons put Qwen slightly ahead on pure coding |
| **DeepSeek Coder V2** | `mlx-community/DeepSeek-Coder-V2-Lite-Instruct-4bit-mlx` | Alternative coder; permissive license | Smaller ecosystem of Cursor walkthroughs |
| **DeepSeek R1 distill** | Search HF for `DeepSeek-R1-Distill` + `mlx` | Step-by-step debugging, “why does this fail?” | Weaker as a primary codegen model |
| **Llama 3.3 8B** | `mlx-community/Meta-Llama-3.3-8B-Instruct-4bit` (or current 8B MLX build) | Fast general chat; OK light edits | Noticeably weaker on code than dedicated coders |
**Not practical as a full local Cursor daily driver** (API or huge weights): Kimi K2.x, GLM-5, full DeepSeek V3/V4 frontier models. Use cloud Cursor models for those workloads.
**How to try an alternative:** use the same `mlx_lm.server` / Cursor flow; only change `--model` and the model name in Cursor settings. Example:
```bash
mlx_lm.server --model mlx-community/Qwen3-Coder-Next-mxfp4 --host 127.0.0.1 --port 8080
```
On **128 GB** unified memory, sensible experiments after the default works:
1. **Qwen3 Coder** MLX build (likely best upgrade path in 2026).
2. Keep **Qwen2.5-Coder-7B** loaded or on a second port for quick questions.
3. Optional **R1 distill** when you want reasoning traces, not codegen.
**Pick criteria (more useful than benchmark tables):**
- MLX conversion exists under [mlx-community](https://huggingface.co/mlx-community) or a trusted MLX publisher.
- **Instruct** (or coder-instruct) weights, not base.
- Quality on *your* C#/GDScript files and test style in a 30-minute A/B.
- RAM at 4-bit fits comfortably alongside IDE + Godot + server.
Search for new conversions: `https://huggingface.co/models?search=mlx+coder`
## 3. Download and smoke test
```bash
@ -250,6 +297,7 @@ First response after cold start may be slow while weights load into memory.
| Symptom | What to try |
|---------|-------------|
| `mlx_lm.server` not found | `uv tool install mlx-lm`; add `$HOME/.local/bin` to `PATH` (see §1); or `~/.local/bin/mlx_lm.server` |
| Connection refused | Confirm `mlx_lm.server` is running; `curl http://127.0.0.1:8080/v1/models` |
| Wrong model / 404 | Model name must match `/v1/models` exactly |
| First request very slow | Normal — one-time load into unified memory |
@ -257,12 +305,14 @@ First response after cold start may be slow while weights load into memory.
| Cursor Verify fails | Tunnel with `cloudflared` or `ngrok`; use tunnel URL + `/v1` |
| Agent unreliable | Expected — use local for chat/edits only |
## Alternatives
## Alternative runtimes (not MLX)
Same Cursor setup (OpenAI-compatible `/v1`); different local stack:
| Tool | Role |
|------|------|
| **[Ollama](https://ollama.com)** | Easier onboarding; may not use MLX under the hood on Mac |
| **[LM Studio](https://lmstudio.ai)** | GUI; local server often at `http://localhost:1234/v1` |
| **[LM Studio](https://lmstudio.ai)** | GUI; local server often at `http://localhost:1234/v1`; many MLX builds |
| **`mlx-openai-server`** | OpenAI-compatible wrapper with extra options (short model aliases, multimodal) |
## References