haifeiWu/dsh-computer-use

README excerpt

dsh-computer-use dsh-computer-use is a plugin for DSH (DeepSeek Harness) that gives your agents local macOS computer use capabilities — see the screen (screenshot), drive mouse/keyboard/scroll, and control the browser at DOM level, with an OCR fallback for models without vision. - Tools: screenshot / screen ocr (macOS Vision) / desktop mouse move click drag scroll (cliclick) / keyboard type key / browser open click type scroll extract close (Playwright-core + system Chrome) - Enforces an act → verify loop: screenshot before and after every operation; text-only models read the screen via screen ocr / browser extract - Safety by design: on-screen coordinate guards, keyboard shortcut whitelist, TCC permission boundaries (never bypassed) - Mounts via profile cordis.patch.yml ; does not modify the official dsh repository 以下为中文文档 / Chinese documentation below: 为 dsh（DeepSeek Harness） 提供本地 macOS computer use 能力：模型可以看屏幕（截图）、操作屏幕（鼠标/键盘/滚动）、控制浏览器（DOM 级），并在模型无视觉能力时用 OCR 文本兜底。 独立插件包，通过 profile cordis.patch.yml 挂载，不改动 dsh 官方仓库。 能力总览 场景 工具 实现 看屏幕 screenshot （全屏/区域/窗口 → image block） /usr/sbin/screencapture + attachment 服务 看屏幕（无视觉模型） screen ocr （文本 + 像素坐标） macOS Vision framework（swift） 桌面键鼠 deskto…

Category