web-interface
Operate and safely adapt the MedRAX Gradio chat interface around an initialized agent, including image and DICOM uploads, chat threads, tool-result images, and server configuration.
- License 许可
- Apache-2.0
- 最近更新
- 2026-10-08
- Stars
- ★ 11
- Forks
- 3
01 前言 Front Matter
- name
- web-interface
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Operate and safely adapt the MedRAX Gradio chat interface around an initialized agent, including image and DICOM uploads, chat threads, tool-result images, and server configuration.
- disable-model-invocation
- true
02 资源文件 3
- 参考文档references/configuration.md4.5 KB
- 参考文档references/troubleshooting.md5.0 KB
- 参考文档references/workflows.md5.7 KB
03 SKILL.md 正文
MedRAX web interface
Use this skill when a researcher needs to run or modify the browser UI around an already initialized MedRAX agent. This is an interface adapter, not a tool implementation or an agent-graph tutorial.
Operating contract
- Input: an initialized
agentexposingworkflow.stream(...), plus atools_dictmapping the selected tool names to initialized tool instances. - Output: a Gradio
Blocksdemo that accepts a user message and optional image, streams assistant/tool events, and returns(chat_history, image_path, text)for the bound outputs. - Keep the original uploaded path for tools and keep a separately viewable path for the UI. Never silently replace a DICOM path with its converted PNG when a downstream tool needs the DICOM.
- The interface is a convenience for analysis, not a diagnostic device. Do not present model output as a clinical conclusion; retain a visible safety notice appropriate to the deployment.
For the detailed event and upload sequence, read workflows.md. For launch and resource settings, read configuration.md. For failure diagnosis, read troubleshooting.md.
Entrypoint and lifecycle
- Load the medical system prompt and call the application initializer. Its
configuration includes
tools_to_use,model_dir,temp_dir,device, model name, temperature, top-p, and OpenAI-compatible keyword arguments. - Select only tools that are installed and affordable. The initializer's
utility choices include
ImageVisualizerToolandDicomProcessorTool; its model-backed choices include classification, segmentation, visual QA, report generation, grounding, LLaVA-Med, and generation. A selected tool must be present in both the agent tool list andtools_dict. - Call
create_demo(agent, tools_dict). It constructsChatInterface, agr.Blockslayout, a messages-styleChatbot, text input, image display, upload buttons, and Clear Chat/New Thread controls. - Bind text submission as two stages:
add_messagefirst appends the user image and text and disables the textbox;process_messagethen streams agent events and updates chatbot, display image, and textbox. Re-enable the textbox after completion, including failure. - Launch with explicit, reviewed server settings. A safe local default is
loopback and
share=False; do not inherit the example's broad bind and public share settings without a deployment decision.
ChatInterface currently stores current_thread_id, original_file_path, and
display_file_path on one Python object. That is adequate for a single-user
demo only. For concurrent users, put these values in per-session gr.State (or
a server-side session store), use a collision-resistant thread identifier, and
never let one browser's upload or thread be reused by another browser.
Upload and path rules
UploadButtonis configured separately for ordinary images and DICOM. The current DICOM button accepts a generic file, so validate the lower-cased suffix before processing rather than trusting the widget.- Copy uploads into a controlled temporary directory using a generated name. Preserve the original suffix; do not use the client filename as a path. Prefer a UUID plus suffix over a seconds-only timestamp to avoid collisions. Apply size, count, and suffix allowlists and clean old files deliberately.
- For an ordinary image, the copied path can serve both purposes after basic
validation. For
.dcm, call the selectedDicomProcessorTooland read the returned display mapping'simage_pathfor the UI. - Keep
original_file_pathas the path sent in the textualimage_path:message. Keepdisplay_file_pathas the PNG or other viewable conversion. If a multimodal payload is also built, encode the display path with its real MIME type; never label raw DICOM bytes as JPEG. This distinction is required even though the current implementation uses one fallback path for both. - If DICOM is enabled, fail early with a clear configuration error when
tools_dict["DicomProcessorTool"]is missing. Do not catch that as an unexplained chat failure.
Message and event handling
Build the current request in the shape expected by the agent:
- Optional path message:
{"role": "user", "content": "image_path: <path>"}. - Optional multimodal message: a user content list containing an
image_urldata payload for the viewable image. - Optional text message: a user content list containing
{"type": "text", "text": <message>}.
The current LangGraph call is conceptually:
agent.workflow.stream(
{"messages": messages},
{"configurable": {"thread_id": thread_id}},
)
The checkpointer and thread ID carry graph state; the visible Gradio history is not itself the complete graph conversation. New threads must receive a new ID. Clear Chat should also decide whether it is merely a visual reset or a new conversation; make that choice explicit instead of leaving stale graph state.
Handle the stream defensively:
- For a
processevent, take the latest message content, format it as text, and append an assistantChatMessage. - For an
executeevent, iterate tool messages, use the emitted tool name, and display textual results plus any validated image result. The current image branch recognizes the emitted nameimage_visualizer; do not assume it is the same spelling/case asImageVisualizerToolintools_dict. - Never use
evalon tool output. Prefer structured tool returns; for a legacy serialized value, use a constrained parser and validate the result shape and path before rendering it. - Every normal and error path must yield the same number and order of outputs:
(history, display_path, ""). Do not expose raw temporary paths in assistant text, and do not put an unredacted provider exception or credential in chat.
Reset, rendering, and assets
- Clear Chat should clear visible messages and the displayed image, remove or rotate the active session's upload references, and—if conversation reset is intended—rotate the thread ID too.
- New Thread should rotate only the conversation ID unless the product explicitly promises to remove the selected image. Document the behavior.
- Tool-generated images should be rendered only after checking that the path is an expected local result, not an arbitrary path returned by a model or tool.
- Resolve the avatar/logo from a packaged resource at runtime. If it is absent, use a no-avatar configuration instead of making startup depend on the process's current working directory.
Resource-constrained adaptation
For a CPU-only utility demonstration, initialize only
ImageVisualizerTool and DicomProcessorTool with device="cpu" where the
initializer requires a device. This avoids local vision-model weight loading,
but the normal process_message still calls the configured chat model. A full
chat demo therefore still needs an accessible OpenAI-compatible model endpoint,
or a test/stub agent whose workflow.stream produces deterministic events.
Do not claim a no-weight, no-model setup can answer medical questions.
Use configuration.md for a minimal selection matrix and troubleshooting.md when an optional model-backed tool prevents startup.
Scope boundaries
Route tool signatures, image processing, DICOM conversion implementation, and
result schemas to chest-xray-analysis or image-data-utilities. Route
LangGraph agent construction, prompts, checkpointers, and tool binding to
agent-orchestration. Route benchmark execution and scoring to
benchmark-evaluation. This skill only adapts their initialized outputs to a
safe Gradio lifecycle.
04 同仓库更多 Skills
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
python-api
Programmatic Object-Detection-Metrics API guidance for BoundingBox, BoundingBoxes, Evaluator, enums, coordinate conversion, AP/mAP semantics, and source-style import diagnostics.
file-evaluation
Folder-based PASCAL VOC AP/mAP evaluation from ground-truth and detection text files.
object-detection-metrics
Guides legacy Object-Detection-Metrics PASCAL VOC AP/mAP evaluation from detection text folders and direct Python API classes.
nx-workspace-patterns
Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.