TTS Fine-Tuning Infrastructure and GPU Platforms¶

GPU rental platform comparison and deployment patterns for fine-tuning and serving 2B-4B TTS models. Based on April 2026 pricing; rates change frequently — verify before committing.

GPU Rental Platform Comparison¶

On-Demand Pods¶

Serverless / Scale-to-Zero¶

Selection Guide¶

Fine-Tune Cost Estimates¶

VoxCPM2 (2B, tokenizer-free diffusion autoregressive)¶

LoRA adds <1% parameters. GPU requirements: ≥8GB VRAM minimum, 24GB for normal batch sizes.

Dataset scale: 60-120 min clean speech = solid LoRA tier (2-4× recommended minimum of 5-30 min). Full fine-tune is overkill for single-speaker.

Fish-Speech S2 Pro (4B)¶

+50-80% time vs VoxCPM2. H100 80GB recommended for LoRA; A100 80GB with gradient accumulation.

Inference Real-Time Factor (RTF)¶

RTF = inference time / audio duration. RTF < 1 = faster than real-time.

Cost per 1M characters (rough, ~18.5 hours of audio):

With batching and LoRA hot-swap, effective cost 2-3× lower.

Multi-LoRA Serving¶

vLLM Multi-LoRA¶

For TTS models with compatible architectures:

# Enable runtime LoRA loading
VLLM_ALLOW_RUNTIME_LORA_UPDATING=True python -m vllm.entrypoints.openai.api_server \
    --model base_tts_model \
    --enable-lora \
    --max-lora-rank 32

# Load adapter per request via lora_request parameter

Limitation: vLLM targets LLM architectures. Diffusion AR (VoxCPM2) and dual-AR (Fish S2 Pro) may not be compatible out of the box — verify per model.

PEFT Manual Switching¶

For non-vLLM-compatible TTS architectures:

from peft import PeftModel
import time

base_model = load_base_model()  # loaded once, stays in VRAM
adapter_cache = {}

def get_model_for_teacher(teacher_id: str):
    if teacher_id not in adapter_cache:
        path = f"adapters/{teacher_id}"
        adapter_cache[teacher_id] = path

    base_model.load_adapter(adapter_cache[teacher_id], adapter_name=teacher_id)
    base_model.set_adapter(teacher_id)
    return base_model
    # Switch time: ~50-200ms depending on adapter size

Storage per Adapter¶

LoRA rank 16-32 for TTS → 20-100 MB per adapter (compressible). 100 adapters ≈ 5 GB on network volume ≈ $0.35/month on RunPod. Negligible at scale.

Deployment Patterns¶

Pattern A: Persistent Pod (first production version)¶

# RunPod Pod with custom FastAPI inference server
# Persistent network volume: $0.07/GB/mo
# Structure:
# /volume/
#   base_model/          # Base TTS weights (~10-20 GB)
#   adapters/            # LoRA adapters (100 × 50MB = 5 GB)
#   datasets/            # Training data (optional)

# HTTP endpoint via pod's public URL
# One H100 handles 10-30 parallel audio generation streams

Pattern B: Serverless Flex (scale-to-zero)¶

# RunPod Serverless: custom Docker image
# - Base model + adapters mounted from network volume
# - Handler returns base64-encoded audio or file URL
# - FlashBoot: 1-2s cold start with cached image
# - Active workers: add N always-on workers when traffic is regular

Pattern C: Modal¶

import modal

app = modal.App("tts-inference")
volume = modal.Volume.from_name("tts-models")

@app.function(
    gpu="A100",
    volumes={"/models": volume},
    timeout=120,
)
def synthesize(text: str, lora_id: str) -> bytes:
    model = load_with_adapter(f"/models/adapters/{lora_id}")
    return model.synthesize(text)

Voice Safety Infrastructure¶

Non-optional controls for voice cloning systems:

Consent Gate¶

# At dataset upload time:
# 1. Server generates random phrase with 60s TTL
# 2. User reads phrase aloud (microphone, not file upload)
# 3. ASR verifies phrase content
# 4. Speaker verification confirms same voice as training data
# 5. Log consent event with timestamp, user ID, audio hash

Audio Watermarking¶

Embed imperceptible watermark in every synthesized audio output: - AudioSeal (Meta, open-source): neural watermark, verifiable via API - Resemble Neural Watermarking: commercial, traceable to account - ElevenLabs embeds watermarks in all generation by default

Embed at vocoder output stage — zero perceptible quality degradation.

Audit Trail¶

# Every synthesis request:
audit_log.append({
    "timestamp": datetime.utcnow(),
    "account_id": user_id,
    "lora_id": adapter_id,
    "text_hash": sha256(text),
    "audio_hash": sha256(audio_output),
    "duration_seconds": len(audio) / sample_rate,
})
# Retain for 30-90 days for forensic attribution

Gotchas¶

• Vast.ai spot interruption risk: interruptible instances lose all non-checkpointed work without warning. Never start a 6-hour fine-tune on spot without checkpoint-every-N-steps configured. Use Vast.ai only after the pipeline is validated on non-interruptible hardware.
• vLLM incompatibility with non-LLM TTS: vLLM's Multi-LoRA is designed for decoder-only transformer LLMs. VoxCPM2 (tokenizer-free diffusion AR) and Fish-Speech S2 Pro (dual-AR) architectures may require custom serving code. Test explicitly before assuming vLLM will work.
• 60-120 min clean speech = LoRA, not full fine-tune: with this dataset size, full fine-tune overfits and does not improve over LoRA. Full fine-tune requires 1-5 hours of clean single-speaker audio. Using full fine-tune on a 60-min dataset wastes 2-4× compute with worse results.
• ConTree microVMs have no confirmed GPU access: Nebius ConTree is a code execution sandbox for agents, not a GPU compute service. The separate Nebius cloud has H100 and L40S instances. Do not confuse the two when planning compute budget.

See Also¶

• tts models - TTS architecture families, model comparison
• voice cloning - zero-shot cloning vs. fine-tuning approaches
• lora fine tuning for editing models - LoRA training patterns (image domain, partially applicable)
• agent deployment - infrastructure patterns for agent-serving backends