sandbox-tfjs
Classify and embed images, detect objects, and answer from a passage in a Code node or CodeAct action, with TensorFlow.js running on the host
- License 许可
- AGPL-3.0
- 最近更新
- 2026-08-18
- Stars
- ★ 1
- Forks
- 0
01 前言 Front Matter
- name
- sandbox-tfjs
- description
- Classify and embed images, detect objects, and answer from a passage in a Code node or CodeAct action, with TensorFlow.js running on the host
02 资源文件 1
- 参考文档package.json949 B
03 SKILL.md 正文
TensorFlow.js models in the sandbox
Specifier: @nodetool-ai/sandbox-tfjs. Declare it in the node's packages
property and import it at the top of the body.
The models are tens of megabytes of weights, fetched over the network and kept between calls. The guest has neither the heap for them nor a way to keep anything alive past a run, so this pack is a host module: the import resolves to a generated facade over NodeTool's own implementation, and each model loads once per process.
These models are small, local, and free — that is the whole reason to reach for
them. For anything a description or a caption would answer better, use a vision
model through nodetool.media instead.
classify — ImageNet labels
import { classify } from "@nodetool-ai/sandbox-tfjs";
const labels = await classify(await workspace.readBytes("photo.jpg"), { topK: 3 });
return { best: labels[0].className, probability: labels[0].probability };
MobileNet v2 over the 1000 ImageNet classes, most likely first. topK defaults
to 5.
embed — a feature vector for an image
import { embed } from "@nodetool-ai/sandbox-tfjs";
const vector = await embed(bytes); // 1280 numbers
The MobileNet penultimate layer. Compare two vectors by cosine similarity for near-duplicate detection or nearest-neighbour search.
detect — objects with boxes
import { detect } from "@nodetool-ai/sandbox-tfjs";
const objects = await detect(bytes, { maxBoxes: 10, minScore: 0.6 });
// [{ className: "person", score: 0.91, bbox: { x, y, width, height } }, …]
COCO-SSD over the 80 COCO classes. The box is in image pixels. maxBoxes
defaults to 20 and minScore to 0.5.
answer — extractive question answering
import { answer } from "@nodetool-ai/sandbox-tfjs";
const spans = await answer("When was it founded?", inputs.article);
return { answer: spans[0]?.text ?? null };
BERT QnA quotes the passage — it never writes prose. Each span carries a
score and its startIndex/endIndex in the passage.
Gotchas
- Every export is async, and the first call to each model waits for its weights to download.
- Images go in as encoded bytes (PNG/JPEG/WebP), decoded through the
sandbox's own
imagebackend — so the same pixel capsimage.decodeenforces apply, and 10 MB is the input limit. - The models see pixels, nothing else.
classifyon a screenshot of text answers with an ImageNet class, not with the text; that is@nodetool-ai/sandbox-ocr.
04 同仓库更多 Skills
sandbox-zip
Read and write zip archives in a Code node or CodeAct action, with fflate running on the host under a 50 MB inflation cap
sandbox-yaml
Read and write YAML inside a Code node or CodeAct action, with js-yaml running in the guest
sandbox-xml
Parse XML feeds and sitemaps in a Code node or CodeAct action, with fast-xml-parser running on the host
sandbox-xlsx
Read and write Excel workbooks in a Code node or CodeAct action, with exceljs running on the host
sandbox-twilio
Send SMS and WhatsApp messages and look up numbers from a Code node or CodeAct action, with NodeTool's Twilio request builder running on the host
sandbox-supabase
Read and write Supabase tables from a Code node or CodeAct action, with NodeTool's PostgREST request builder running on the host