nanbbb/dsh-computer-use-windows

README excerpt

dsh-computer-use Windows Computer Use tools for DeepSeek Harness. The plugin combines: - Windows window discovery and foreground input; - UI Automation element discovery and focus state; - window screenshots; - ModLens-compatible 3x3 visual grounding; - observation-bound element/coordinate actions; - DeepSeek Harness one-shot approval prompts for risky actions. Why the grid exists ModLens deliberately returns semantic evidence rather than invented bounding boxes. computer observe therefore produces an annotated A1-C3 screenshot. For a custom UI, call modlens read image on grid screenshot path , select the reported cell with computer zoom , and repeat until the region is small enough to click. Standard controls should use the UI Automation element index path. Apps without an accessibility tree (WeChat, Electron, …) Self-drawn UIs such as WeChat expose an empty accessibility tree, so element index navigation is unavailable. Use the screenshot path instead: 1. computer observe returns a grid screenshot path . If a window renders blank on first capture (it was still activating or was occluded), the native helper re-activates it and retries before falling back to a screen copy; a still-…

Category