go-farther-and-farther/dsh-tool-eyes

README excerpt

dsh-tool-eyes Local vision "eyes" for DeepSeek Harness (DSH). Give your text-only agent eyes with two model-facing tools : - screen — capture the screen (or describe an existing image file) through a local OpenAI-compatible vision endpoint (llama.cpp with --mmproj , LM Studio, Ollama, ...) and return the vision model's text description. - ocr — extract ALL text verbatim with the Windows built-in OCR engine : zero model, zero GPU, zero cloud, milliseconds. Why DeepSeek's chat-completions line is text-only. Instead of switching your whole conversation to a vision model, keep the text brain and add eyes as tools: - Private by default — point screen at a local endpoint and images never leave your machine. - Cheap — a 0.8B–4B local VLM is plenty for describing screens; ocr costs nothing at all. - Honest by default — the screen prompt tells the VLM to describe only what is visible and never guess app/game/character names unless confirmed by on-screen text (this measurably cuts small-model name hallucination). Requirements - Windows 10/11 (the ocr tool uses WinRT OCR; screen uses .NET for capture) - Node.js = 22.19, DeepSeek Harness = 0.1.0-rc.6 - screen additionally needs any OpenAI-comp…

Category