neon-sprawl/docs/dev/local-mlx-cursor.md

Local MLX models with Cursor (macOS)

Status: Developer guide (not product architecture).
Audience: Neon Sprawl contributors on Apple Silicon who want private, offline-capable chat and inline edits in Cursor, backed by a local model.

Cursor does not talk to MLX directly. It needs an OpenAI-compatible HTTP API (/v1/chat/completions). This guide uses Apple’s MLX runtime via mlx-lm and its built-in server.

What works in Cursor

Feature	Local MLX
Chat (Cmd+L)	Yes
Inline edit (Cmd+K)	Usually yes
Composer / Agent	Limited — local models are weaker at multi-step tool use; prefer cloud models for large refactors
Tab autocomplete	No — Cursor tab completion uses separate proprietary models

Use local MLX for privacy, offline use, and cost. Keep cloud models for Agent mode, tab completion, and multi-file story work.

Architecture

flowchart LR
  subgraph cursor [Cursor IDE]
    Chat[Chat Cmd+L]
    Edit[Inline edit Cmd+K]
  end

  subgraph mac [Mac Apple Silicon]
    API["mlx_lm.server :8080/v1"]
    MLX[MLX runtime]
    Model["Qwen2.5-Coder-32B-Instruct-4bit"]
  end

  Chat --> API
  Edit --> API
  API --> MLX --> Model

Prerequisites

• macOS with Apple Silicon (MLX is not for Intel Macs).
• Python 3.12+ (Homebrew or uv).
• Enough disk for model weights (~18 GB for the recommended 32B 4-bit coder).
• Cursor with permission to override the OpenAI base URL (Settings → Models).

Rough RAM for the recommended model: ~18–22 GB active during inference. A Mac with 48 GB+ unified memory is comfortable; 128 GB can also run 8-bit variants for higher quality.

1. Install mlx-lm

Option A — uv (isolated tool install):

brew install uv
uv tool install mlx-lm

Option B — venv + pip:

brew install python@3.12
python3.12 -m venv ~/.local/venvs/mlx-lm
source ~/.local/venvs/mlx-lm/bin/activate
pip install mlx-lm

Verify:

mlx_lm.server --help
which mlx_lm.server   # note path for LaunchAgent below

If which mlx_lm.server prints not found after uv tool install mlx-lm, the binary is usually at ~/.local/bin/mlx_lm.server but that directory is not on your PATH. Add to ~/.zshrc:

export PATH="$HOME/.local/bin:$PATH"

Then source ~/.zshrc, or run ~/.local/bin/mlx_lm.server --help directly.

2. Choose a model

Defaults (recommended starting point)

Role	Hugging Face repo id	Approx. RAM
Primary coder (recommended)	mlx-community/Qwen2.5-Coder-32B-Instruct-4bit	~18–22 GB
Higher quality (optional)	mlx-community/Qwen2.5-Coder-32B-Instruct-8bit	~32–35 GB
Fast / small (optional)	mlx-community/Qwen2.5-Coder-7B-Instruct-4bit	~5 GB

For Neon Sprawl (C# server tests, GDScript client, Bruno API collections), prefer the Instruct coder variant — not the non-instruct base weights.

Models download on first use into the Hugging Face cache.

Model alternatives

Qwen Coder models are a strong default for local codegen on Mac, but they are not the only worthwhile option. Rankings from public benchmarks (e.g. SWE-bench) change frequently; try models on your own repo before switching defaults.

Family	Example MLX repo id	Best for	Tradeoff vs Qwen2.5-Coder-32B
Qwen2.5 Coder (default above)	mlx-community/Qwen2.5-Coder-32B-Instruct-4bit	Stable, proven Cursor + MLX path	Baseline — not always newest benchmark leader
Qwen3 Coder	mlx-community/Qwen3-Coder-Next-mxfp4	Newer coding benchmarks; MoE efficiency on large Macs	Less “battle-tested” in guides; confirm id via /v1/models
Qwen3 Coder (MoE)	Search HF for Qwen3-Coder-30B-A3B-Instruct-MLX (e.g. lmstudio-community builds)	Strong code quality with fewer active params than dense 30B	Repo id varies by publisher — verify on Hugging Face
Gemma 4	Search HF for Gemma-4 + MLX + 4bit	Local assistant balance; Apache 2.0	Some 2026 comparisons put Qwen slightly ahead on pure coding
DeepSeek Coder V2	mlx-community/DeepSeek-Coder-V2-Lite-Instruct-4bit-mlx	Alternative coder; permissive license	Smaller ecosystem of Cursor walkthroughs
DeepSeek R1 distill	Search HF for DeepSeek-R1-Distill + mlx	Step-by-step debugging, “why does this fail?”	Weaker as a primary codegen model
Llama 3.3 8B	mlx-community/Meta-Llama-3.3-8B-Instruct-4bit (or current 8B MLX build)	Fast general chat; OK light edits	Noticeably weaker on code than dedicated coders

Not practical as a full local Cursor daily driver (API or huge weights): Kimi K2.x, GLM-5, full DeepSeek V3/V4 frontier models. Use cloud Cursor models for those workloads.

How to try an alternative: use the same mlx_lm.server / Cursor flow; only change --model and the model name in Cursor settings. Example:

mlx_lm.server --model mlx-community/Qwen3-Coder-Next-mxfp4 --host 127.0.0.1 --port 8080

On 128 GB unified memory, sensible experiments after the default works:

1. Qwen3 Coder MLX build (likely best upgrade path in 2026).
2. Keep Qwen2.5-Coder-7B loaded or on a second port for quick questions.
3. Optional R1 distill when you want reasoning traces, not codegen.

Pick criteria (more useful than benchmark tables):

• MLX conversion exists under mlx-community or a trusted MLX publisher.
• Instruct (or coder-instruct) weights, not base.
• Quality on your C#/GDScript files and test style in a 30-minute A/B.
• RAM at 4-bit fits comfortably alongside IDE + Godot + server.

Search for new conversions: https://huggingface.co/models?search=mlx+coder

3. Download and smoke test

mlx_lm.generate \
  --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit \
  --max-tokens 64 \
  --prompt "Write a one-line C# xUnit test skeleton."

4. Start the OpenAI-compatible server

mlx_lm.server \
  --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit \
  --host 127.0.0.1 \
  --port 8080

Leave this process running while using Cursor.

Verify the API:

curl -s http://127.0.0.1:8080/v1/models | python3 -m json.tool

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Qwen2.5-Coder-32B-Instruct-4bit",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}],
    "max_tokens": 64
  }' | python3 -m json.tool

Use the exact "id" from /v1/models as the model name in Cursor.

Official server docs: mlx_lm/SERVER.md.

5. Configure Cursor

1. Open Cursor Settings (Cmd+,).
2. Go to Models.
3. Enable Override OpenAI Base URL (wording may vary: “use own API key”, custom OpenAI endpoint).
4. Set:
   • Base URL: http://127.0.0.1:8080/v1 (the /v1 suffix is required)
   • API key: any non-empty string (e.g. local)
5. Add model — see If Cursor says “Model name is not valid” below (often not the raw Hugging Face id).
6. Click Verify if available.
7. In Chat (Cmd+L), select that model from the dropdown.

If Verify fails on localhost

Some Cursor builds restrict 127.0.0.1. Expose the same server through a tunnel:

# Server still on 8080; in another terminal:
brew install cloudflared
cloudflared tunnel --url http://127.0.0.1:8080

Set Cursor’s base URL to the tunnel URL with /v1 appended (e.g. https://….trycloudflare.com/v1).

If Cursor says “Model name is not valid”

This is not an MLX problem. Your server can return the correct id in /v1/models (e.g. mlx-community/Qwen2.5-Coder-32B-Instruct-8bit) while Cursor still refuses to add it — recent Cursor builds validate custom model names against an internal catalog before any request is sent. Slashes, mlx-community/…, and most Hugging Face repo ids fail with:

Model name is not valid: "mlx-community/Qwen2.5-Coder-32B-Instruct-8bit"

Workaround (recommended): run mlx-openai-server with a short alias Cursor already recognizes, while loading your real weights in the background.

Requires Python 3.11+ (separate from the Python 3.9 env uv tool install mlx-lm uses):

uv tool install mlx-openai-server --python 3.12

Stop mlx_lm.server if it is on port 8080, then:

mlx-openai-server launch \
  --model-type lm \
  --model-path mlx-community/Qwen2.5-Coder-32B-Instruct-8bit \
  --served-model-name gpt-4o-mini \
  --host 127.0.0.1 \
  --port 8080

In Cursor → Models:

• Base URL: http://127.0.0.1:8080/v1
• Add model: gpt-4o-mini (the alias, not the Hugging Face path)

Verify with curl (should answer using your Qwen weights):

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Reply with exactly: ok"}],
    "max_tokens": 16
  }'

mlx_lm.server alone treats the request model field as a Hugging Face repo id, so gpt-4o-mini fails there (404 / “Repository Not Found”). Use mlx-openai-server when you need a Cursor-friendly alias.

Other names to try in Cursor if gpt-4o-mini is blocked on your build: gpt-4o, gpt-4o-mini-2024-07-18, or lowercase variants (some forum reports: casing matters). The alias must match what you pass in chat and what --served-model-name exposes on /v1/models.

Ollama equivalent: ollama create gpt-4o-mini -f Modelfile with FROM qwen2.5-coder:… — same idea, different runtime (DEV walkthrough).

6. Optional: start server on login

Replace MLX_LM_SERVER with the output of which mlx_lm.server.

Save as ~/Library/LaunchAgents/com.neon-sprawl.mlx-coder.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.neon-sprawl.mlx-coder</string>
  <key>ProgramArguments</key>
  <array>
    <string>MLX_LM_SERVER</string>
    <string>--model</string>
    <string>mlx-community/Qwen2.5-Coder-32B-Instruct-4bit</string>
    <string>--host</string>
    <string>127.0.0.1</string>
    <string>--port</string>
    <string>8080</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
  <key>KeepAlive</key>
  <true/>
  <key>StandardOutPath</key>
  <string>/tmp/mlx-coder-server.log</string>
  <key>StandardErrorPath</key>
  <string>/tmp/mlx-coder-server.err</string>
</dict>
</plist>

launchctl load ~/Library/LaunchAgents/com.neon-sprawl.mlx-coder.plist

7. Day-one helper script

Save as ~/bin/start-mlx-coder.sh (or anywhere on your PATH):

#!/usr/bin/env bash
set -euo pipefail
MODEL="mlx-community/Qwen2.5-Coder-32B-Instruct-4bit"
PORT=8080

echo "Warming $MODEL (first run downloads ~18GB)..."
mlx_lm.generate --model "$MODEL" --max-tokens 1 --prompt "ok" >/dev/null

echo "Starting OpenAI-compatible server on http://127.0.0.1:$PORT/v1"
exec mlx_lm.server --model "$MODEL" --host 127.0.0.1 --port "$PORT"

chmod +x ~/bin/start-mlx-coder.sh
~/bin/start-mlx-coder.sh

Then point Cursor at http://127.0.0.1:8080/v1.

8. Neon Sprawl usage patterns

Good fits for local MLX

• Explain a server type, test, or Bruno request.
• Draft a small GdUnit skeleton or C# AAA test outline.
• Answer “why might this flake?” on an open file.

Prefer cloud Cursor models

• Agent mode across many files.
• Story-sized implementation (Linear NEO-* branches).
• Tab completion.

Optional Cursor rule

If local models hallucinate edits, add a project rule (e.g. .cursor/rules/local-mlx.md):

- Prefer small, file-scoped changes.
- For edits, show a unified diff or exact replacement block.
- Do not claim tools ran or tests passed unless shown.
- Match existing C# AAA test layout and GdUnit `# Arrange` / `# Act` / `# Assert`.

9. Sanity-check prompts in Cursor

With the MLX model selected in Chat:

1. “Summarize the open implementation plan in three bullets.”
2. “Suggest only the Assert section for this xUnit test — no other changes.”
3. “Minimal GdUnit test for a node that emits health_changed.”

First response after cold start may be slow while weights load into memory.

10. Troubleshooting

Symptom	What to try
mlx_lm.server not found	uv tool install mlx-lm; add $HOME/.local/bin to PATH (see §1); or ~/.local/bin/mlx_lm.server
unrecognized arguments: --mlx-community/...	Use --model mlx-community/..., not --mlx-community/...
Connection refused	Confirm mlx_lm.server is running; curl http://127.0.0.1:8080/v1/models
Wrong model / 404	Model name must match /v1/models exactly
First request very slow	Normal — one-time load into unified memory
Poor code quality	Use Instruct variant; try 8-bit if you have RAM headroom
Model name is not valid	Cursor catalog validation — use mlx-openai-server + --served-model-name gpt-4o-mini (§5)
Cursor Verify fails	Tunnel with cloudflared or ngrok; use tunnel URL + /v1
Agent unreliable	Expected — use local for chat/edits only

Alternative runtimes (not MLX)

Same Cursor setup (OpenAI-compatible /v1); different local stack:

Tool	Role
Ollama	Easier onboarding; may not use MLX under the hood on Mac
LM Studio	GUI; local server often at http://localhost:1234/v1; many MLX builds
mlx-openai-server	OpenAI-compatible wrapper with extra options (short model aliases, multimodal)

References

• MLX LM server: github.com/ml-explore/mlx-lm — mlx_lm/SERVER.md
• Recommended weights: mlx-community/Qwen2.5-Coder-32B-Instruct-4bit
• Repo stack context: docs/architecture/tech_stack.md