kuzco-recognition
Recognize and locate instances of an object in an image using a local vision
language model, exposed as an ellmer tool by the
kuzco R package.
When to use
The user wants to find a specific kind of object in an image: faces, cars, dogs,
text regions. Counting instances and describing/locating each one. For general
"what is in this image?" questions, use kuzco-classification instead.
Setup
library(kuzco)
library(ellmer)
chat <- chat_ollama(model = "qwen2.5vl")
chat$register_tool(tool_llm_image_recognition())
Tool arguments (LLM-controlled)
image (required): local file path to a jpeg, jpg, or png image.
recognize_object (optional): the object to recognize, e.g. "face", "car", "dog".
Defaults to "face".
additional_prompt (optional): extra instructions, e.g. "only count adults".
language (optional): output language, e.g. "English".
Result
Compact JSON text with object_recognized, object_count, object_description,
and object_location. Report the count first, then describe/locate instances.
If the count is zero, say so plainly rather than hedging.
Example
User: "How many faces are in img/group.jpg and where?"
- Call
kuzco_image_recognition with image = "img/group.jpg", recognize_object = "face".
- Report the count and describe each location.