Produce speech and clone voices with Fish Audio — its hosted TTS API (S2.1-Pro / S2-Pro / S1 model lineup, REST + WebSocket streaming, instant and persistent voice cloning) and its open-weight OpenAudio S1-mini / Fish-Speech models for self-hosting. Use this skill when an agent must generate narration or dialogue through Fish Audio, choose between Fish Audio's hosted models and open weights, clone a voice from reference audio, author emotion/tone/special markers for expressive delivery, estimate cost from UTF-8 bytes, wire real-time streaming for a voice agent, decide whether self-hosting beats the API, or review Fish Audio TTS output for production. Do not use it to pick a different provider — it covers Fish Audio specifically.
インストールする前に、エージェントに与えられる指示の中身を確認できます。
Fish Audio is a TTS and voice-cloning provider with two distinct product surfaces that share a lineage but differ in licensing and operation:
api.fish.audio) — the commercial service. Current lineup is branded S2.1-Pro / S2-Pro / S1. Paid per UTF-8 byte, streaming, cloning, and a free tier.The single most important fact to get right: the hosted API and the open weights are not the same models, and their licenses differ. The full flagship weights are hosted-only; only smaller distilled weights are openly published, and those carry a non-commercial license. Never assume "Fish is open source, so I can self-host it commercially" — verify which artifact and which license apply. See Open weights and self-hosting below.
All model names, prices, endpoints, and limits below are volatile. Verification date for every dated claim in this document: 2026-07-10. Re-verify against docs.fish.audio before quoting these to a user as current.
Labels used throughout:
Reach for Fish Audio when:
Prefer something else when:
This skill does not choose between providers. If the user is still deciding whether to use Fish Audio at all versus ElevenLabs, Cartesia, etc., surface the tradeoffs but recognize a provider-comparison skill governs that decision.
POST https://api.fish.audio/v1/ttswss://api.fish.audio/v1/tts/livePOST https://api.fish.audio/modeltranscribe-1 model (separate endpoint; $0.36 per audio hour).Required headers on TTS calls: Authorization: Bearer <FISH_API_KEY>, model: <model-name>, and Content-Type: application/json or application/msgpack. The model header — not a body field — selects the model. [Doc]
| Model header value | Role | Marker syntax | Notes |
|---|---|---|---|
s2.1-pro | Recommended for production | square-bracket tags [...] | Improved quality, latency, throughput over S2-Pro. [Claim] |
s2.1-pro-free | Free tier, same weights as s2.1-pro | [...] | $0. No time-to-first-audio (TTFA) or DPA guarantees. For dev/prototyping/smaller use. [Doc] |
s2-pro | Previous-generation default | [...] | Multi-speaker, natural-language expression control. [Doc] |
s1 | Earlier generation | parenthesis tags (...) | Emotion markers via parentheses. [Doc] |
Critical marker-syntax difference [Heuristic, grounded in Doc]: S1 uses parentheses — (excited), (whispering). The S2 family uses square brackets — [excited], [whispers]. Markers written for the wrong model are often read aloud literally or ignored. Match marker syntax to the model header.
"OpenAudio S1" ≠ the hosted s1 model. OpenAudio S1 is the open-weight release brand (below). The hosted s1 is the corresponding hosted endpoint. Keep the two mentally separate when a user says "S1."
| Parameter | Type / range | Purpose |
|---|---|---|
text | string | Content to synthesize. In WebSocket mode, usually empty at start and streamed via text events. |
reference_id | string | ID of a saved voice (library or a cloned model). |
references | array | Instant cloning: raw reference audio + transcript inline, no persistent model. |
format | mp3 (default), wav, pcm, opus | Output container. |
mp3_bitrate | 64 / 128 / 192 | MP3 quality. |
sample_rate | Hz | For WAV output. |
speed | 0.5–2.0 | Speech rate. |
latency | normal (default) / balanced | balanced trades a little stability for lower latency. |
chunk_length | 100–300 (default 200) | Text batching size per synthesis chunk. |
normalize | boolean | Expands numbers/dates/abbreviations for natural reading. |
temperature | float | Lower = more deterministic/stable. |
top_p | float | Sampling diversity. |
repetition_penalty | >1.0 | Suppresses looping/repeated artifacts. |
max_new_tokens | int | Caps audio length per chunk. |
Format selection [Heuristic]: wav or pcm for downstream editing and telephony pipelines; opus for streaming to browsers/mobile; mp3 when you just need a portable file. pcm avoids container/decoder latency in real-time loops.
Three generation modes in the Python SDK [Doc]:
tts.convert() — returns complete audio bytes (simplest; use for files/batch).tts.stream() — iterator of audio chunks (memory-efficient long-form).tts.stream_websocket() — real-time bidirectional streaming for agents.WebSocket protocol [Doc, 2026-07-10] at /v1/tts/live, MessagePack-serialized:
start (config: format, chunk_length, reference_id, latency), text (append text as it's produced upstream, e.g. token-by-token from an LLM), flush (force immediate synthesis of buffered text — the key lever for low latency in interactive apps), stop (end of stream).audio (a chunk of bytes in the requested format — concatenate all chunks for the full clip), finish (session end, with a reason).Real-time heuristic [Heuristic]: stream the LLM's tokens into text events and issue flush at sentence or clause boundaries. Waiting for a whole paragraph before the first flush defeats the point of streaming. Fish's own material describes sub-500 ms end-to-end and ~450 ms perceived first-audio when streaming instead of waiting for the full file. [Claim]
s2.1-pro-free is $0.len(text.encode("utf-8")) / 1e6 * 15 USD, not from character count.transcribe-1: $0.36 per audio hour. Voice design voice-design-1: $0.01 per successful request.s2.1-pro-free) carries no TTFA or DPA guarantees. [Doc]Two paths [Doc, 2026-07-10]:
POST /model with type=tts, title, voices (one or more reference audio files), visibility (private/public), optional description. Returns a voice id; pass it as reference_id in TTS. Models move through created → trained; a "fast" train mode returns a usable voice immediately. enhance_audio_quality (default on) denoises and normalizes the reference.references field of a TTS call. Best for one-off voices.Reference-audio requirements [Doc + Heuristic]:
.wav, .mp3, .m4a, .opus. [Doc]Fish Audio's differentiator is inline expressive control. Markers are written into the text itself. [Doc]
S1 (parentheses) [Doc, 2026-07-10]: 40+ markers across three groups.
(angry), (sad), (excited), (surprised), (sarcastic), (joyful), (empathetic), and more.(in a hurry tone), (shouting), (screaming), (whispering), (soft tone).(laughing), (chuckling), (sobbing), (sighing), (panting), (crowd laughing).S2 family (square brackets) [Doc, 2026-07-10]: a much larger open tag vocabulary (Fish cites 15,000+ tags) plus free-form natural-language descriptions. Documented examples include [pause], [short pause], [emphasis], [laughing], [chuckle], [sigh], [inhale], [exhale], [whisper], [shouting], [screaming], [loud], [low volume], [volume up], [volume down], [excited], [angry], [sad], [surprised], [singing], [echo], [with strong accent], [clearing throat], [audience laughter], and free-form forms like [whisper in a small voice]. [Doc/Claim — the 15,000 figure is a first-party claim.]
Marker heuristics [Heuristic]:
(angry)(sad)(excited) in one clause produces incoherent delivery.(laughing) / [laughing] insert a non-verbal event — don't also spell out "haha" unless you want both.temperature.model header (see the S1-vs-S2 warning above). This is the most common expressive-TTS failure with Fish.normalize ([Doc]) so "$1,200", "Dr.", "3/4", and "2026" are expanded, or pre-expand them yourself for full control. Un-normalized numerics are a frequent source of wrong readings.[pause] for ordinary prose. Reserve pause markers for deliberate dramatic beats.chunk_length batch them; extremely long single chunks raise the odds of drift, repetition, or a runaway (laughing).reference_ids per speaker rather than trying to shift one voice with markers alone.Fish Audio publishes open weights under the OpenAudio brand (successor branding to Fish-Speech). This is a separate track from the hosted API. [Doc, 2026-07-10]
fishaudio/openaudio-s1-mini). Released alongside S1 on June 3, 2025.fishaudio/fish-speech) also describes a larger S2-Pro (4B) open weight in its README. [Doc] Confirm exactly which weight file and license you are downloading before relying on it.fishaudio/fish-speech repository's README states code and weights under the Fish Audio Research License (with an explicit "we will act against violations" notice). [Doc]ffmpeg, libsox, portaudio); install via conda or uv; vLLM / SGLang backends for throughput. [Doc]Treat these as vendor and third-party claims with stated methods, not as neutral ground truth. All dated 2026-07-10 as read.
How to use benchmarks in production advice [Heuristic]: low WER/CER means intelligible, not expressive or preferred. Arena ELO reflects listener preference on the arena's prompt mix, which may not match your domain (long-form audiobook vs. short agent turns). Always validate on your own scripts, voices, and languages before committing.
Non-negotiable, and the agent should enforce these when a user asks to clone a voice:
| Symptom | Likely cause | Fix |
|---|---|---|
| Markers read aloud literally ("open paren excited close paren") | Wrong marker syntax for the model | Match (...) to s1, [...] to S2 family |
| Robotic / low-similarity clone | Noisy, short, or multi-speaker reference | Re-record clean mono ≥30 s; keep enhance_audio_quality on |
| Repetition / looping / runaway laughter | Long chunk, high temperature, or stacked markers | Shorten chunks, raise repetition_penalty, lower temperature, one marker per span |
| Numbers/dates/acronyms misread | Normalization off | Set normalize=true or pre-expand text |
| Bill far higher than expected | Non-Latin script billed at 3–4 bytes/char | Estimate from utf-8 byte length, not char count |
| High first-audio latency in an agent | Waiting for full text before synthesis | Use WebSocket, stream text, flush at clause boundaries, format=pcm, latency=balanced |
| Commercial/legal exposure from self-host | Used CC-BY-NC-SA weights commercially | Move to hosted API or obtain a commercial license |
Production review checklist [Heuristic]: intelligibility (no dropped/garbled words), correct pronunciation of names/numbers, appropriate and non-literal marker rendering, consistent voice identity across chunks, natural pacing/breaths, no repetition or clipping at chunk seams, target loudness, and — for clones — documented consent on file.
Intent: a 2-sentence dramatic voiceover with a whispered aside, delivered as a WAV for editing; estimate cost first.
Model: s2.1-pro (square-bracket markers).
TEXT='The vault was empty. [whisper] Someone had been here first.'
# Cost estimate: bytes / 1e6 * $15
python -c "print(len(open('/dev/stdin').read().encode())/1e6*15)" <<< "$TEXT"
curl -s https://api.fish.audio/v1/tts \
-H "Authorization: Bearer $FISH_API_KEY" \
-H "model: s2.1-pro" \
-H "Content-Type: application/json" \
-d '{
"text": "The vault was empty. [whisper] Someone had been here first.",
"reference_id": "<your-voice-id>",
"format": "wav",
"sample_rate": 44100,
"normalize": true,
"temperature": 0.7,
"repetition_penalty": 1.2
}' --output vault.wav
Why structured this way: brackets match the S2 model; the whisper marker precedes only the aside; normalize guards stray numerics; modest temperature with a repetition penalty keeps the dramatic line stable. Likely failure: if the voice were an s1 clone you'd need (whisper) instead. Variation: switch to s2.1-pro-free for a zero-cost test render before spending on the paid model.
Intent: clone a consented narrator once, reuse across many lines.
from fishaudio import FishAudio
client = FishAudio()
# 1) Create the model from a clean 45-second mono sample (consent on file).
voice = client.voices.create(
title="Narrator - Priya (consented 2026-07-10)",
voices=[open("priya_45s_clean.wav", "rb").read()],
description="Studio sample, single speaker, no music",
visibility="private",
)
# 2) Reuse by reference_id.
audio = client.tts.convert(
text="Chapter one. (soft tone) It began on a grey morning.",
reference_id=voice.id, # this clone was trained on s1-style pipeline; use (..) markers
)
open("ch1.mp3", "wb").write(audio)
Why: the reference is long, clean, mono, and consented; the clone is private; markers match the model family. Failure: a public reference sample with background music degrades every downstream render. Variation: for a single throwaway line, use references=[...] instant cloning instead of creating a persistent model.
Intent: speak an LLM's tokens as they arrive with minimal first-audio latency.
Pattern (pseudocode over wss://api.fish.audio/v1/tts/live, MessagePack):
send start { format:"pcm", latency:"balanced", chunk_length:200, reference_id:"..." }
for token in llm_stream:
send text { text: token }
if token ends a clause: send flush {}
send stop {}
# concatenate every server `audio` event's bytes until `finish`
Why: pcm avoids decode latency; flush at clause boundaries emits audio without waiting for the full reply; balanced shaves latency. Failure: flushing only at the end reintroduces the ~1 s+ full-file wait. Variation: raise chunk_length for smoother prosody on long, non-interactive narration where latency doesn't matter.
Intent: user has 40 hours/month of internal, non-commercial research narration and one RTX 4090. Decision: OpenAudio S1-mini self-hosted is reasonable — fits the 4090, non-commercial use satisfies CC-BY-NC-SA, and steady volume amortizes the GPU. [Heuristic] But if that narration ships in a paid product, self-hosting the NC weights is a license violation → use the hosted API instead. Confirm the actual downloaded weight's license before deploying.
Reference audio for cloning is biometric-adjacent personal data. Keep clone models private, store consent records with the voice, avoid uploading third-party voices you cannot substantiate rights for, and delete clones and source audio when the project ends. Do not send user PII inside TTS text to a third-party API without the user's awareness.
まだレビューはありません。使ってみた感想をお寄せください。
概要と使いどころ
Production guidance for reusable layered 2D character rigs in browser-rendered and video media. Use for SVG hierarchies, pivots and constraints, FK/IK planning, pose and facial libraries, acting, cycles, deterministic handoff, crop variants, and rig QA; not for character concept continuity, 3D armatures, or motion-capture solving.
日本語の概要は準備中です。原文の説明を表示しています。
Use this skill to turn generated, captured, scanned, or modeled 3D output into production-ready standalone assets for DCC, real-time engine, web, or interchange delivery. It covers requirements, topology repair, scale and axes, UVs, PBR baking, LODs, static and skeletal readiness, rig handoff, optimization, glTF/USD/FBX delivery, validation, QA, and rights/provenance. Do not use it for provider-specific generation prompts, full scene/world building, mocap operation, or exact CAD/manufacturing work.
日本語の概要は準備中です。原文の説明を表示しています。
Use ACE-Step and ACE-Step 1.5 for local or hosted AI music generation, including text-to-music, lyrics-to-song, instrumental beds, covers, repainting, stem/track extraction, track completion, LoRA personalization, REST/Python/Gradio workflows, rights review, and music integration for video, ads, games, and social content.
日本語の概要は準備中です。原文の説明を表示しています。
Use Adobe Firefly Services to generate, edit, expand, fill, match, composite, and upscale still images through the current REST APIs. Apply when selecting Firefly Image 5 versus Image 3/4 or custom models, implementing authenticated asynchronous image workflows, using style or structure references, building product composites, handling provenance and enterprise rights, or diagnosing Firefly image jobs in production. Exclude Firefly video, audio, and unrelated Creative Cloud APIs.
日本語の概要は準備中です。原文の説明を表示しています。
Plan, prompt, call, edit, iterate, and productionize Alibaba Cloud Model Studio image generation with current Wan 2.7 Image, Qwen-Image 2.0, and Z-Image Turbo models. Use for Alibaba/DashScope text-to-image, multi-reference generation, instruction editing, character-consistent image sets, typography and layout work, regional authentication, model selection, API integration, cost/rate-limit planning, troubleshooting, safety, rights, privacy, and output QA.
日本語の概要は準備中です。原文の説明を表示しています。
Plan, implement, and review Alibaba Wan video generation using Alibaba Cloud Model Studio/DashScope hosted APIs or official Wan 2.1/2.2 open-weight checkpoints. Use for Wan text-to-video, image-to-video, first/last-frame or continuation, reference-to-video, video editing, speech-driven video, character animation, regional deployment, billing, licensing, and production safety.
日本語の概要は準備中です。原文の説明を表示しています。