AI Video Pipeline

Surflink's AI analysis runs on an external SurfVision backend (the model/server is not in this repo). The pipeline is YOLO26 (detection, tracking, pose) + Meta SAM 3.1 (instance segmentation). The app talks to it over HTTP and runs dual-backend: a self-hosted GPU primary with automatic Modal cloud failover.

Backend resolution (primary + failover)

src/lib/modal.js resolves the backend server-side at request time:

If SURFVISION_PRIMARY_URL is set and its root (/) responds within ~3s, the primary (self-hosted GPU) is used.
Otherwise it falls back to NEXT_PUBLIC_MODAL_API_URL (Modal cloud).

/api/clips/analyze persists the resolved backend as session_clips.api_url, so client polling and the store route always query the backend the job actually ran on.

SSRF guard: the primary URL is server-only (never NEXT_PUBLIC), and the store routes only honor a recorded api_url if it matches an allowlist of host suffixes (*.modal.run, *.surfvision.ai, *.surflink.ai, *.trycloudflare.com). A client-supplied URL is honored only when NODE_ENV !== 'production' (local-GPU dev).

Endpoints called

Endpoint	Purpose
`POST {base}/analyze`	Submit a clip (cloud: a signed `video_url`; local: a multipart `file`). Returns `{ job_id }`.
`GET {base}/status/{job_id}`	Poll processing status/progress.
`GET {base}/result/{job_id}/video`	Download the processed video (server-side, then cached).
`GET {base}/result/{job_id}/stats`	Download the stats JSON.
`GET {base}/`	Health/root probe.

Earlier docs referenced a /result/{job_id}/progress endpoint -- that is incorrect. Progress is polled via GET {base}/status/{job_id}.

End-to-end flow

Upload. The browser uploads each raw clip to the raw-uploads Supabase Storage bucket via resumable TUS (max 2 concurrent).
Analyze. POST /api/clips/analyze mints a 1-hour signed URL for the clip and forwards video_url to {base}/analyze (55s timeout; route maxDuration=60), then persists job_id and status processing on the session_clips row.
Poll. The client polls {base}/status/{job_id} every 3 seconds, in parallel across clips with Promise.allSettled, handling statuses complete/done/failed/error and progress payloads.
Store (CDN cache). On completion the clip is pushed to a single global FIFO queue (concurrency 2). POST /api/clips/store downloads {base}/result/{job_id}/video (with retry/backoff; maxDuration=300), uploads it to session-videos, and saves video_storage_url + stats_json.
Stream. Playback goes through the HMAC-gated /api/video/stream proxy. Local-GPU clips can play directly from the local server until the CDN copy is ready.

Coaching feedback (Gemini)

Separately from detection/tracking, structured coaching feedback is generated by Google Gemini:

POST /api/sessions/coaching -- per-clip and batch analysis (strengths, improvements, per-wave notes, key moments, summaries).
POST /api/sessions/coaching/progress -- cross-session progress analysis.
GET /api/sessions/coaching/export -- HTML/JSON coaching report.

These require GOOGLE_AI_API_KEY (and use GEMINI_MODEL, default gemini-3-flash-preview).

Database & Migrations

Auth & Middleware

AI Video Pipeline

Backend resolution (primary + failover)

src/lib/modal.js resolves the backend server-side at request time:

If SURFVISION_PRIMARY_URL is set and its root (/) responds within ~3s, the primary (self-hosted GPU) is used.

Otherwise it falls back to NEXT_PUBLIC_MODAL_API_URL (Modal cloud).

/api/clips/analyze persists the resolved backend as session_clips.api_url, so client polling and the store route always query the backend the job actually ran on.

SSRF guard: the primary URL is server-only (never NEXT_PUBLIC), and the store routes only honor a recorded api_url if it matches an allowlist of host suffixes (*.modal.run, *.surfvision.ai, *.surflink.ai, *.trycloudflare.com). A client-supplied URL is honored only when NODE_ENV !== 'production' (local-GPU dev).

Endpoints called

Endpoint	Purpose
`POST {base}/analyze`	Submit a clip (cloud: a signed `video_url`; local: a multipart `file`). Returns `{ job_id }`.
`GET {base}/status/{job_id}`	Poll processing status/progress.
`GET {base}/result/{job_id}/video`	Download the processed video (server-side, then cached).
`GET {base}/result/{job_id}/stats`	Download the stats JSON.
`GET {base}/`	Health/root probe.

Earlier docs referenced a /result/{job_id}/progress endpoint -- that is incorrect. Progress is polled via GET {base}/status/{job_id}.

End-to-end flow

Upload. The browser uploads each raw clip to the raw-uploads Supabase Storage bucket via resumable TUS (max 2 concurrent).

Analyze. POST /api/clips/analyze mints a 1-hour signed URL for the clip and forwards video_url to {base}/analyze (55s timeout; route maxDuration=60), then persists job_id and status processing on the session_clips row.

Poll. The client polls {base}/status/{job_id} every 3 seconds, in parallel across clips with Promise.allSettled, handling statuses complete/done/failed/error and progress payloads.

Store (CDN cache). On completion the clip is pushed to a single global FIFO queue (concurrency 2). POST /api/clips/store downloads {base}/result/{job_id}/video (with retry/backoff; maxDuration=300), uploads it to session-videos, and saves video_storage_url + stats_json.

Stream. Playback goes through the HMAC-gated /api/video/stream proxy. Local-GPU clips can play directly from the local server until the CDN copy is ready.

Coaching feedback (Gemini)

Separately from detection/tracking, structured coaching feedback is generated by Google Gemini:

POST /api/sessions/coaching -- per-clip and batch analysis (strengths, improvements, per-wave notes, key moments, summaries).

POST /api/sessions/coaching/progress -- cross-session progress analysis.

GET /api/sessions/coaching/export -- HTML/JSON coaching report.

These require GOOGLE_AI_API_KEY (and use GEMINI_MODEL, default gemini-3-flash-preview).