feat: upload current kino project

This commit is contained in:
Botomir
2026-07-10 07:59:12 +02:00
commit bb75ef2150
56 changed files with 5176 additions and 0 deletions

12
.dockerignore Normal file
View File

@@ -0,0 +1,12 @@
.venv/
**/.venv/
__pycache__/
**/__pycache__/
.pytest_cache/
**/.pytest_cache/
*.pyc
*.pyo
backend/*.egg-info/
backend/**/*.egg-info/
plugin.video.xstream/
script.module.xstreamscraper/

13
.env.example Normal file
View File

@@ -0,0 +1,13 @@
KINOPROJEKT_DB=/data/kino.sqlite3
KINOPROJEKT_MEDIA_ROOT=/jellyfin
KINOPROJEKT_TMP=/data/tmp
JELLYFIN_URL=http://jellyfin:8096
# JELLYFIN_API_KEY belongs in the protected target-host .env or Vaultwarden, never in Nextcloud notes.
JELLYFIN_API_KEY=
DEFAULT_MAX_HEIGHT=1080
MAX_DOWNLOAD_BYTES=15000000000
KINOPROJEKT_ADMIN_USER=admin
# Generate with: python -c "from app.services.auth import hash_password; import getpass; print(hash_password(getpass.getpass()))"
KINOPROJEKT_ADMIN_PASSWORD_HASH=
# Generate with: python -c "import secrets; print(secrets.token_urlsafe(48))"
KINOPROJEKT_SESSION_SECRET=

37
.gitignore vendored Normal file
View File

@@ -0,0 +1,37 @@
# Python
__pycache__/
*.py[cod]
*$py.class
.pytest_cache/
.mypy_cache/
.ruff_cache/
.coverage
htmlcov/
*.egg-info/
.venv/
venv/
# Runtime/local data
.env
.env.*
!.env.example
data/
backend/data/
*.sqlite
*.sqlite3
*.db
# Media/download staging
backend/app/web/downloads/*
!backend/app/web/downloads/.gitkeep
# Node/build artifacts
node_modules/
dist/
build/
# OS/editor
.DS_Store
.idea/
.vscode/
*.swp

530
Botomir-Status.md Normal file
View File

@@ -0,0 +1,530 @@
# Botomir-Status
Aktualisiert: 2026-06-28 20:39 CEST
## Update 2026-06-28 20:39 CEST
- Vor Umbau lokales Fallback-Backup erstellt:
```text
/root/botomir-work/backups/kino-projekt-current-before-extension-20260628-202959.tgz
sha256: fe623994047e59e6db886a3637098244342ffa16e813c1b0d8c1d81228222255
```
- Neue generische Browser-Extension `Kino Capture` implementiert, orientiert an der Architektur von Video DownloadHelper:
- Manifest V3
- beobachtet `webRequest`/Response-Header im aktiven Tab
- sammelt `.m3u8`, `.mpd`, `.mp4`, `.webm`, `.mkv`, `.mov`, `.m4v` und `video/*`/MPEGURL/DASH Content-Types
- ergänzt DOM-/Performance-Scan per Content-Script
- Popup zeigt Kandidaten und öffnet die Kino-App mit `?capture=...`
- App-URL ist im Popup konfigurierbar, Default `http://192.168.178.222:8099/`
- keine hoster-spezifischen Bypässe/DRM-/CAPTCHA-/Token-Umgehung
- Kino-App-UI um Download-Link und Installationshinweise ergänzt.
- Extension-ZIP gebaut und statisch ausgeliefert:
```text
/static/downloads/kino-capture-extension.zip
sha256 lokal: 6c7bbceac8c8b85cc144fdd9f1d7de68284fd0eee7ecb2e5d6b60c5e7b62f3a9
```
- Checks:
```text
node --check service_worker.js/content.js/popup.js
python3 -m json.tool manifest.json
pytest -q
# 38 passed, 1 warning in 0.75s
```
- Deployment auf `192.168.178.222` durchgeführt.
- Verifikation:
```text
GET http://127.0.0.1:8099/health
# {"status":"ok"}
GET http://192.168.178.222:8099/static/downloads/kino-capture-extension.zip
# HTTP/1.1 200 OK
# content-type: application/zip
# content-length: 6535
GET http://192.168.178.222:8099/
# enthält: Kino-Capture Extension herunterladen
```
## Update 2026-06-28 20:02 CEST
- Browser-Capture-Bookmarklet robuster gemacht:
- wenn `window.open(...)` durch Popup-Blocker/Browser-Schutz nicht öffnet, navigiert es jetzt als Fallback den aktuellen Tab zur Kino-App
- wenn im aktuellen Tab keine Medien-URL gefunden wurde, erscheint ein Hinweis/Confirm statt scheinbar „nichts passiert“
- Regex akzeptiert nun Query **und Fragment** nach Medien-Endungen (`.m3u8?…`, `.mp4#…`)
- Tests weiterhin grün:
```text
pytest -q
# 38 passed, 1 warning in 0.75s
```
- Deployment auf `192.168.178.222` durchgeführt.
- Verifikation:
```text
GET http://127.0.0.1:8099/health
# {"status":"ok"}
GET /static/app.js
# enthält: if(!opened)location.href=dest
# enthält: Keine Medien-URL
```
## Update 2026-06-28 19:47 CEST
- Analyzer generisch erweitert, ohne hoster-/schutzspezifische Umgehungen einzubauen:
- erkennt jetzt auch JS-escaped Medien-URLs wie `https:\/\/...\/master.m3u8`
- erkennt unquoted absolute Medien-URLs in Inline-JS/Config-Objekten
- normalisiert HTML-Entities und escaped slashes vor der Kandidatenprüfung
- Fehlertext verbessert: Wenn eine HTML-Seite erreichbar ist, aber automatisch kein importierbarer Medienkandidat gefunden wird, kommt jetzt ein Hinweis auf `Seite analysieren` bzw. Browser-Capture statt nur `text/html`.
- Regressionstest ergänzt für HTML/JS-Extraktion.
```text
cd /root/botomir-work/kino-projekt-current/backend
. .venv/bin/activate
pytest -q
# Ergebnis: 38 passed, 1 warning in 0.76s
```
- Deployment auf Jellyfin-LXC `192.168.178.222` durchgeführt.
- Zielhost-Verifikation:
```text
extract_media_urls(...)
# ['https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8?token=1',
# 'https://cdn.example.org/movie/manifest.mpd?x=1']
GET http://127.0.0.1:8099/health
# {"status":"ok"}
POST /api/probe mit öffentlicher HLS-Test-URL
# provider=mediathek, title=x36xhzz.m3u8, allowed=true
```
## Update 2026-06-28 19:31 CEST
- Fehler `URL is reachable, but content type/extension is not accepted: text/html` behoben.
- Ursache: Beim Import/Probe wurde teilweise noch die HTML-Playerseite selbst an den Import-Provider gegeben. HTML-Seiten sind keine Mediendatei; der Provider lehnte sie korrekt ab, statt den zuvor gefundenen `.m3u8`/Medienkandidaten zu nutzen.
- Fix: `/api/probe` und `/api/imports` versuchen bei `text/html` jetzt automatisch die Seitenanalyse, wählen den besten erlaubten Kandidaten (`direct_video`, `hls_manifest`, `dash_manifest`) und speichern/importieren dessen Medien-URL. Die ursprüngliche Seiten-URL wird in den Job-Metadaten als `source_page_url` erhalten.
- Regressionstest ergänzt: HTML-Seite mit erkanntem HLS-Kandidaten wird zu dessen `.m3u8`-URL aufgelöst.
```text
cd /root/botomir-work/kino-projekt-current/backend
. .venv/bin/activate
pytest -q
# Ergebnis: 37 passed, 1 warning in 0.75s
```
- Deployment auf Jellyfin-LXC `192.168.178.222` durchgeführt und Dienst geprüft:
```text
importable_kinds ['dash_manifest', 'direct_video', 'hls_manifest']
systemctl is-active kino-projekt.service
# active
GET http://127.0.0.1:8099/health
# {"status":"ok"}
POST /api/probe mit öffentlicher HLS-Test-URL
# Ergebnis: provider=mediathek, title=x36xhzz.m3u8, allowed=true
```
## Update 2026-06-28 19:20 CEST
- Fehler beim Import von HLS/DASH-Kandidaten behoben: `.m3u8`/`.mpd` sind nur Manifeste/Playlists und dürfen nicht per HTTP direkt als Datei nach Jellyfin kopiert werden.
- `MediathekProvider.download()` erkennt jetzt HLS/DASH-Manifeste und lädt sie über `yt-dlp` + ffmpeg, mit `--remux-video mp4`, statt die wenige-KB-Manifestdatei zu speichern.
- Direkte Medienlinks wie `.mp4` bleiben beim bisherigen HTTP-Streaming-Pfad.
- Regressionstests ergänzt:
```text
cd /root/botomir-work/kino-projekt-current/backend
. .venv/bin/activate
pytest -q
# Ergebnis: 36 passed, 1 warning in 0.75s
```
- Deployment auf Jellyfin-LXC `192.168.178.222` durchgeführt. Verifikation auf dem Ziel:
```text
yt-dlp command: /opt/kino-projekt/venv/bin/python -m yt_dlp
ffmpeg: vorhanden
m3u8_detected: True
systemctl is-active kino-projekt.service
# active
GET http://127.0.0.1:8099/health
# {"status":"ok"}
/opt/kino-projekt/venv/bin/python -m yt_dlp --simulate ... x36xhzz.m3u8
# m3u8_native mp4
```
## Update 2026-06-28 19:08 CEST
- Jellyfin-API-Key aus dem bestehenden Vaultwarden-Item `Kino-Projekt App-Login` Feld `jellyfin-api` gelesen, ohne den Wert auszugeben.
- `/etc/kino-projekt.env` auf Jellyfin-LXC `192.168.178.222` ergänzt/aktualisiert:
- `JELLYFIN_URL=http://127.0.0.1:8096`
- `JELLYFIN_API_KEY=[REDACTED]`
- Vorheriges Env-Backup angelegt: `/etc/kino-projekt.env.bak-jellyfin-*`.
- Jellyfin Refresh API direkt mit dem Key geprüft:
```text
POST http://127.0.0.1:8096/Library/Refresh
# Ergebnis: HTTP 204
```
- `kino-projekt.service` neugestartet und geprüft:
```text
systemctl is-active kino-projekt.service
# active
GET http://127.0.0.1:8099/health
# {"status":"ok"}
```
- Systemd nutzt weiterhin `EnvironmentFile=/etc/kino-projekt.env`; vorhandene Env-Keys wurden geprüft, ohne Werte auszugeben.
## Update 2026-06-28 18:56 CEST
- Login-Problem untersucht: der Login über `https://kino.dasposchi.de` funktionierte, aber beim direkten LAN-Zugriff `http://192.168.178.222:8099` setzte die App immer ein `Secure`-Cookie. Browser speichern/senden `Secure`-Cookies über HTTP nicht zuverlässig; dadurch wirkte der Login im LAN technisch defekt.
- Fix umgesetzt: Session-Cookie ist nur noch `Secure`, wenn der Request selbst HTTPS nutzt oder `X-Forwarded-Proto: https` gesetzt ist. Direkter LAN-HTTP-Zugriff bekommt ein nicht-`Secure` HttpOnly/SameSite=Lax-Cookie, Mesh/HTTPS bleibt `Secure`.
- Tests ergänzt und ausgeführt:
```text
cd /root/botomir-work/kino-projekt-current/backend
. .venv/bin/activate
pytest -q
# Ergebnis: 34 passed, 1 warning in 0.73s
```
- Deployment auf Jellyfin-LXC `192.168.178.222` durchgeführt, Dienst neugestartet und geprüft.
### Verifikation
```text
GET http://192.168.178.222:8099/health
# Ergebnis: HTTP 200 {"status":"ok"}
Login über http://192.168.178.222:8099/api/auth/login
# Ergebnis: 204; Set-Cookie ohne Secure; /api/auth/status danach authenticated=true
Login über https://kino.dasposchi.de/api/auth/login via Mesh
# Ergebnis: 204; Set-Cookie mit Secure; /api/auth/status danach authenticated=true
```
## Update 2026-06-28 18:41 CEST
- Analyzer-Code aus dem separaten lokalen Stand in den aktuellen, deployed Provider-/Import-Stand gemergt, ohne die bestehende Auth-/Import-Architektur zurückzurollen.
- Neu im deployed Dienst:
- `POST /api/analyze` kombiniert HTML-Scan, `yt-dlp`-Analyse und Headless-Browser-Netzwerkbeobachtung.
- `POST /api/browser-capture` validiert Kandidaten aus dem Bookmarklet/echten Browser.
- WebUI enthält jetzt „Seite analysieren“, Browser-Capture-Bookmarklet und Kandidatenliste mit Dateigrößenanzeige.
- SSRF-Schutz für Seitenanalyse/Downloads wurde ergänzt; private/interne Hosts werden geblockt.
- uBlock-Origin-Pfadlogik ist aktiv; Browser-Analyse meldete `ublock_origin=true` im Smoke-Test.
- Deployment auf Jellyfin-LXC `192.168.178.222` abgeschlossen. Vor dem Austausch wurde ein Backup unter `/opt/kino-projekt/backups/deploy-20260628-*/app-src-before-analyzer.tgz` angelegt.
### Verifikation in diesem Lauf
```text
cd /root/botomir-work/kino-projekt-current/backend
. .venv/bin/activate
pytest -q
# Ergebnis: 32 passed, 1 warning in 0.48s
GET http://192.168.178.222:8099/health
# Ergebnis: HTTP 200 {"status":"ok"}
POST http://192.168.178.222:8099/api/browser-capture
# Payload: filesamples MP4-Kandidat, Login-Cookie aus lokaler Passwortdatei
# Ergebnis: HTTP 200, 1 Kandidat, source=browser-capture, kind=direct_video
POST http://192.168.178.222:8099/api/analyze
# Payload: https://filesamples.com/samples/video/mp4/sample_640x360.mp4
# Ergebnis: HTTP 200, 1 Kandidat, source=browser, kind=direct_video, ublock_origin=true
curl --interface tailscale0 --resolve kino.dasposchi.de:443:100.64.0.9 https://kino.dasposchi.de/health
# Ergebnis: HTTP 200 {"status":"ok"}
POST https://kino.dasposchi.de/api/analyze über Mesh
# Ergebnis: Login 204; Analyze 1 Kandidat, source=browser, kind=direct_video, ublock_origin=true
```
### Hinweise
- Remote-Produktionsvenv hat keine Test-Extras (`pytest`) installiert; Tests wurden lokal im Arbeitsstand ausgeführt und der deployte Dienst per HTTP-Smoke-Test verifiziert.
- Ein echter Import in `/jellyfin` wurde weiterhin nicht ausgelöst; verifiziert wurden Analyse, Kandidatenvalidierung, Auth und Dienstgesundheit.
## Update 2026-06-25 06:20 CEST
- Reverse-Proxy-Blocker für `kino.dasposchi.de` behoben: Nginx Proxy Manager auf dem Public-VPS hat jetzt einen eigenen Proxy-Host `kino.dasposchi.de` mit Lets-Encrypt-Zertifikat und Headscale-Mesh-only Access List.
- Für den Upstream wurde ein dedizierter Reverse-SSH-Tunnel vom Jellyfin-LXC zum VPS eingerichtet:
- `kino-vps-tunnel.service` auf CT `100` (`jellyfin`) ist `active`.
- VPS/NPM-Container `headscale-tunnel-sshd:8892` leitet auf `127.0.0.1:8099` im Jellyfin-LXC weiter.
- NPM-Datenbank/Configs wurden vor Änderung gesichert: `/opt/infra/npm/backups/botomir-kino-20260625-060320/`.
- Neues Zertifikat: `/etc/letsencrypt/live/npm-29/`, SAN `DNS:kino.dasposchi.de`, gültig bis `2026-09-23 03:05:22 UTC`.
- Secret-Hygiene: App-Passwort wurde nicht ausgegeben; lokale Passwortdatei ist jetzt `0600`.
### Verifikation in diesem Lauf
```text
# Public ohne Mesh: TLS/SNI repariert, aber bewusst blockiert
GET http://kino.dasposchi.de/
# Ergebnis: HTTP 301 -> https://kino.dasposchi.de/
GET https://kino.dasposchi.de/
# Ergebnis: HTTP 403 openresty (Headscale-Mesh-only Access List greift)
# Mesh-Zugriff über public Hostname und VPS-Mesh-IP
curl --interface tailscale0 --resolve kino.dasposchi.de:443:100.64.0.9 https://kino.dasposchi.de/
# Ergebnis: HTTP 200, 2217 bytes, WebUI erreichbar
curl --interface tailscale0 --resolve kino.dasposchi.de:443:100.64.0.9 https://kino.dasposchi.de/health
# Ergebnis: HTTP 200 {"status":"ok"}
curl --interface tailscale0 --resolve kino.dasposchi.de:443:100.64.0.9 https://kino.dasposchi.de/api/auth/status
# Ergebnis: HTTP 200 {"enabled":true,"authenticated":false,"username":null}
# App-Login ohne Secret-Ausgabe
POST https://kino.dasposchi.de/api/auth/login (über Mesh, Passwort aus root-only lokaler Datei)
# Ergebnis: HTTP 204
GET https://kino.dasposchi.de/api/auth/status (mit Session-Cookie)
# Ergebnis: HTTP 200 {"enabled":true,"authenticated":true,"username":"admin"}
POST https://kino.dasposchi.de/api/probe {"url":"https://archive.org/details/BigBuckBunny_328"} (mit Session-Cookie)
# Ergebnis: HTTP 200, provider=internet_archive, title="Big Buck Bunny", external_id="BigBuckBunny_328", allowed=True
# Zielhost lokal
systemctl is-active kino-projekt.service
# Ergebnis: active
systemctl is-active kino-vps-tunnel.service
# Ergebnis: active
curl http://127.0.0.1:8099/health
# Ergebnis: HTTP 200 {"status":"ok"}
# VPS/NPM
nginx -t im npm-server-app Container
# Ergebnis: syntax is ok; test is successful
proxy_host DB-Zeile:
# id=15, domain=["kino.dasposchi.de"], upstream=headscale-tunnel-sshd:8892, access_list_id=4, certificate_id=29, ssl_forced=1, enabled=1
```
### Aktueller Stand
- Akzeptanzkriterien `public blockiert`, `Mesh erreichbar`, `App-Login aktiv`, `Probe legaler Quelle`, `Jellyfin-Dienst gesund` sind erfüllt.
- Nicht durchgeführt: echter Download/Import in die Jellyfin-Bibliothek. Grund: Das würde reale Mediendateien in `/jellyfin` schreiben; ohne explizite Auswahl/Bestätigung wurde nur eine Probe gegen eine legale Quelle durchgeführt.
## Update 2026-06-25 02:00 CEST
- Zielhost-Deployment auf dem Jellyfin-LXC `jellyfin` (`192.168.178.222`) durchgeführt: bestehendes `/opt/kino-projekt/app-src` wurde gesichert und durch den aktuellen legal-source FastAPI-Stand ersetzt.
- Vor dem Austausch wurde ein Backup erstellt: `/opt/kino-projekt/backups/deploy-20260625-015600/` mit `app-src.tar.gz` und der geschützten `/etc/kino-projekt.env`.
- Der bestehende systemd-Dienst `kino-projekt.service` wurde neu gestartet und läuft weiter auf Port `8099`.
- `/jellyfin/YouTube-Mediathek` ist vorhanden und passend gesetzt: `2775 kino:media`.
- Kompatibilität ergänzt: `backend/app/config.py` akzeptiert jetzt das bereits auf dem Zielhost vorhandene Legacy-Setting `KINOPROJEKT_DB_URL=sqlite:////...`, ohne neue Secrets oder `.env`-Werte in Nextcloud abzulegen.
### Verifikation in diesem Lauf
```text
cd /root/botomir-work/kino-projekt-current/backend
. .venv/bin/activate
pytest -q
# Ergebnis: 27 passed, 1 warning in 0.45s
cd /root/botomir-work/kino-projekt-current
docker build -t kino-projekt-botomir:cron-20260625-deploy .
# Ergebnis: Image gebaut: sha256:e78ca77fb8434ea10ff4622489f4543a31f2d682c688ff72f71f4c429b832669
Container-Smoke-Test:
GET http://127.0.0.1:18085/health
# Ergebnis: HTTP 200 {"status":"ok"}
GET http://127.0.0.1:18085/api/auth/status
# Ergebnis: HTTP 200 {"enabled":false,"authenticated":true,"username":null}
POST http://127.0.0.1:18085/api/probe {"url":"http://localhost/video.mp4"}
# Ergebnis: HTTP 400, private/loopback Ziele werden abgelehnt
Zielhost:
systemctl is-active kino-projekt.service
# Ergebnis: active
GET http://192.168.178.222:8099/health
# Ergebnis: HTTP 200 {"status":"ok"}
GET http://192.168.178.222:8099/api/auth/status
# Ergebnis: HTTP 200 {"enabled":true,"authenticated":false,"username":null}
GET http://192.168.178.222:8099/
# Ergebnis: HTTP 200, 2217 bytes, neue WebUI erreichbar
POST http://192.168.178.222:8099/api/probe ohne Login
# Ergebnis: HTTP 401 {"detail":"Login required"}
stat /jellyfin/YouTube-Mediathek
# Ergebnis: 2775 kino:media /jellyfin/YouTube-Mediathek
Jellyfin Health: GET http://192.168.178.222:8096/health
# Ergebnis: HTTP 200 Healthy
```
### Noch offen / Blocker
- Öffentliches `https://kino.dasposchi.de/` ist noch nicht korrekt auf den Dienst geroutet: aktueller Probe-Fehler `TLS connect error ... tlsv1 unrecognized name`/HTTP `000`. Der lokale Dienst auf dem Jellyfin-LXC läuft; als nächstes muss der Nginx-Proxy-Manager-/Edge-VHost für `kino.dasposchi.de` auf `192.168.178.222:8099` mit Mesh-only Access List eingerichtet bzw. repariert werden.
- Ein echter Import mit App-Login/Jellyfin-Refresh wurde noch nicht ausgelöst, weil kein Klartext-App-Passwort in Nextcloud liegt und Secrets nicht ausgegeben wurden. Die Auth-Schicht ist aktiv (`enabled=true`) und blockiert unauthentifizierte API-Aufrufe korrekt.
### Dateien mit Änderungen in diesem Lauf
- `backend/app/config.py` — Legacy-Kompatibilität für `KINOPROJEKT_DB_URL` ergänzt.
- Zielhost `/opt/kino-projekt/app-src/` — aktueller App-Stand deployed.
- `Botomir-Status.md` — diese Statusnotiz aktualisiert.
## Update 2026-06-24 21:41 CEST
- Jellyfin-Zielprofile umgesetzt: `film`, `serie` und `clip` erzeugen jetzt serverseitig die zur verifizierten Bibliotheksstruktur passenden Zielordner unter `/jellyfin/Filme`, `/jellyfin/Serien/.../Staffel<N>` und `/jellyfin/YouTube-Mediathek`.
- WebUI erweitert: Zielprofil-Radios sowie optionale Felder für Titel, Jahr, Serientitel, Staffel, Episode und Episodentitel werden beim Import an `POST /api/imports` gesendet.
- Import-Request-Metadaten werden in SQLite mit der Source gespeichert, damit der Hintergrundjob die Zielpfad-Korrekturen ohne zusätzliche Secrets/Nextcloud-Daten verwenden kann.
- README und Deployment-Doku beschreiben die Zielprofile und den anzulegenden `/jellyfin/YouTube-Mediathek`-Ordner.
### Verifikation in diesem Lauf
```text
cd /root/botomir-work/kino-projekt-current/backend
. .venv/bin/activate
pytest -q
# Ergebnis: 27 passed, 1 warning in 0.49s
cd /root/botomir-work/kino-projekt-current
docker build -t kino-projekt-botomir:cron-20260624-profiles .
# Ergebnis: Image gebaut: sha256:574c9013385541f774a160eb99b14ab139cfd99c8a7e0aec4119daf41b4a18f8
# Build-Kontext: 26.66 kB
docker run -d --rm -p 127.0.0.1:18084:8080 -e KINOPROJEKT_DB=/tmp/kino.sqlite3 -e KINOPROJEKT_MEDIA_ROOT=/tmp/jellyfin kino-projekt-botomir:cron-20260624-profiles
curl http://127.0.0.1:18084/health
# Ergebnis: HTTP 200 {"status":"ok"}
GET /
# Ergebnis: HTTP 200, 2217 bytes, Zielprofil-Steuerelemente vorhanden
GET /static/app.js
# Ergebnis: HTTP 200, 4973 bytes, `currentImportPayload` sendet `target_profile`
```
### Dateien mit Änderungen in diesem Lauf
- `backend/app/services/paths.py` — Jellyfin-Zielprofil-Pfade (`film`, `serie`, `clip`) ergänzt.
- `backend/app/schemas.py` — ImportRequest um Zielprofil- und Korrekturfelder erweitert.
- `backend/app/main.py` — Import-Request-Metadaten werden in `Source.metadata_json` persistiert; Jobs verwenden das Zielprofil.
- `backend/app/services/downloader.py` — Hintergrundimport nutzt die Zielprofil-Pfade statt freier Bibliotheksnamen.
- `backend/app/web/index.html`, `backend/app/web/app.js`, `backend/app/web/style.css` — WebUI-Zielprofil-Auswahl und optionale Korrekturfelder.
- `backend/tests/test_path_safety.py` — Regressionstests für Film-/Serien-/Clip-Pfade.
- `README.md`, `docs/deployment.md`, `Botomir-Status.md` — Doku/Status aktualisiert.
### Offene nächste Schritte
1. Zielhost-Deployment vorbereiten: geschützte `.env`/Vaultwarden-Werte für App-Login und Jellyfin setzen; keine Secrets in Nextcloud ablegen.
2. Auf dem Zielhost `/jellyfin/YouTube-Mediathek` mit Owner/Gruppe analog zu `/jellyfin/Filme` und `/jellyfin/Serien` anlegen.
3. Mesh-only Veröffentlichung `kino.dasposchi.de` und danach echten legalen Testimport plus Jellyfin-Refresh verifizieren.
## Update 2026-06-24 17:34 CEST
- Internet-Archive-Provider ergänzt: `archive.org/details/<identifier>`, `archive.org/download/<identifier>/...` und `archive.org/metadata/<identifier>` werden jetzt vor dem generischen Direktlink-Provider erkannt.
- Metadaten werden über die öffentliche Archive-Metadata-API abgefragt; Downloads wählen nur plausible öffentliche Medien-Dateien und laufen danach durch den bestehenden SSRF-geschützten Downloadpfad.
- Provider-Policy und README wurden um Internet Archive als erlaubte Quelle erweitert.
### Verifikation in diesem Lauf
```text
cd /root/botomir-work/kino-projekt-current/backend
. .venv/bin/activate
pytest -q
# Ergebnis: 23 passed, 1 warning in 0.43s
python - <<'PY'
import asyncio
from app.providers.internet_archive import InternetArchiveProvider
async def main():
meta = await InternetArchiveProvider().probe('https://archive.org/details/BigBuckBunny_328')
print(meta.model_dump())
asyncio.run(main())
PY
# Ergebnis: provider=internet_archive, title='Big Buck Bunny', external_id='BigBuckBunny_328'
docker build -t kino-projekt-botomir:cron-20260624-ia .
# Ergebnis: Image gebaut: sha256:4c45d54599dcde1a5d212213c637190000cd2086e9b0671e6499b55ffe723135
# Build-Kontext: 19.41 kB
docker run -d --rm -p 127.0.0.1:18083:8080 -e KINOPROJEKT_DB=/tmp/kino.sqlite3 kino-projekt-botomir:cron-20260624-ia
curl http://127.0.0.1:18083/health
# Ergebnis: HTTP 200 {"status":"ok"}
curl -H 'content-type: application/json' -d '{"url":"https://archive.org/details/BigBuckBunny_328"}' http://127.0.0.1:18083/api/probe
# Ergebnis: provider=internet_archive, title='Big Buck Bunny', external_id='BigBuckBunny_328', allowed=True
```
### Dateien mit Änderungen in diesem Lauf
- `backend/app/providers/internet_archive.py` — neuer Provider für öffentliche Internet-Archive-Items.
- `backend/app/main.py` — Provider registriert, vor dem generischen Direktlink-Provider.
- `backend/app/providers/mediathek.py` — Archive-URLs vom generischen Direktlink-Provider ausgeschlossen.
- `backend/tests/test_internet_archive_provider.py` — Unit-Tests für Erkennung, Identifier-Parsing, Medienauswahl und Fehlfall ohne Medien.
- `backend/tests/test_provider_detection.py` — Detection-Regression für Internet Archive ergänzt.
- `README.md`, `docs/provider-policy.md`, `Botomir-Status.md` — Dokumentation aktualisiert.
## Aktueller Stand
- Projekt wurde aus `Projekte/ToDo/` nach `Projekte/Angefangen/` verschoben.
- Initialer, rechtlich sauberer MVP-Code wurde ergänzt. Der hochgeladene xstream-Code bleibt nur Analyse-Anhang; Piracy-Scraper/Hoster werden nicht implementiert.
- Umgesetzt: FastAPI-Healthcheck, statische Minimal-WebUI, Provider-Abstraktion, YouTube-Erkennung/Probe über `yt-dlp` ohne Cookies/Umgehungsoptionen, konservative Mediathek-/Direktlink-Prüfung mit SSRF-Schutz, SQLite-Modelle, Pfadsicherheitshelfer, Dockerfile, Compose-Datei, `.dockerignore` und Doku.
- Import-Job-MVP: `POST /api/imports`, `GET /api/imports`, `GET /api/imports/{id}`, `GET /api/sources`, Hintergrund-Runner, sichere Zielpfad-Kopie, YouTube-/Direktlink-Download-Wrapper, optionaler Jellyfin-Library-Refresh über `JELLYFIN_API_KEY`, UI-Button zum Starten eines Imports und Downloader-Test.
- App-Login: Argon2-Passwort-Hash, signiertes Session-Cookie (`HttpOnly`, `Secure`, `SameSite=Lax`), `POST /api/auth/login`, `POST /api/auth/logout`, `GET /api/auth/status`, UI-Login-Formular, `.env.example` und Tests. API-Routen sind geschützt, sobald `KINOPROJEKT_ADMIN_PASSWORD_HASH` und `KINOPROJEKT_SESSION_SECRET` gesetzt sind.
- Neu in diesem Lauf: Import-Fortschritt verbessert. `yt-dlp` läuft mit `--newline`; Download-Prozentzeilen werden geparst und in den Job-Fortschritt gemappt. Direkte Mediathek-/HTTP-Downloads melden Fortschritt über `content-length`, wenn verfügbar. Die WebUI pollt `GET /api/imports/{id}` alle 2 Sekunden und zeigt Status, Prozent, Zielpfad sowie Hinweise/Fehler live an.
- Secrets wurden nicht in Nextcloud abgelegt. `.env.example` enthält nur leere Platzhalter und Befehle zum lokalen Erzeugen von Hash/Secret.
## Verifikation lokal im Hermes-Container
```text
cd /root/botomir-work/kino-projekt-current/backend
. .venv/bin/activate
pytest -q
# Ergebnis: 17 passed, 1 warning in 0.41s
```
Docker-Build:
```text
cd /root/botomir-work/kino-projekt-current
docker build -t kino-projekt-botomir:cron-20260624-progress .
# Ergebnis: Image gebaut: sha256:f6b1fc960f939d2cda658aefc1df7e0c29f5d73bc2a4ca04a2e27fbbea897b0b
# Build-Kontext: 21.22 kB; lokale .venv/__pycache__/egg-info-Artefakte wurden durch .dockerignore ausgeschlossen.
```
Container-Smoke-Test ohne Zielhost-Secrets:
```text
docker run -d --rm -p 127.0.0.1:18082:8080 \
-e KINOPROJEKT_DB=/tmp/kino.sqlite3 \
kino-projekt-botomir:cron-20260624-progress
curl http://127.0.0.1:18082/health
# Ergebnis: HTTP 200 {"status":"ok"}
curl http://127.0.0.1:18082/api/auth/status
# Ergebnis: HTTP 200 {"enabled":false,"authenticated":true,"username":null}
curl -o /tmp/kino_index.html -w '%{http_code} bytes=%{size_download}\n' http://127.0.0.1:18082/
# Ergebnis: 200 bytes=1212
curl http://127.0.0.1:18082/static/app.js | grep -q 'pollJob'
# Ergebnis: yes
```
## Dateien mit Änderungen in diesem Lauf
- `backend/app/providers/base.py` — Provider-Download-Signatur um optionalen Fortschritts-Callback erweitert.
- `backend/app/providers/youtube.py``yt-dlp --newline`, Fortschrittsparser und laufendes Stream-Auslesen statt nur finalem `communicate()`.
- `backend/app/providers/mediathek.py` — direkter HTTP-Download meldet Fortschritt anhand `content-length`.
- `backend/app/services/downloader.py` — Provider-Fortschritt wird auf `progress=0.15..0.75` gemappt und in SQLite gespeichert.
- `backend/app/web/app.js` — Job-Status wird live gepollt und gerendert.
- `backend/tests/test_downloader.py` — Regressionstest für `yt-dlp`-Fortschrittsparser ergänzt.
- `Botomir-Status.md` — diese Statusnotiz aktualisiert.
## Offene nächste Schritte
1. Zielhost-Deployment vorbereiten: `KINOPROJEKT_ADMIN_PASSWORD_HASH`, `KINOPROJEKT_SESSION_SECRET`, `JELLYFIN_API_KEY` und Zielpfade aus Vaultwarden/geschützter `.env` setzen; keine Secrets in Nextcloud ablegen.
2. Mesh-only Deployment von `kino.dasposchi.de` vorbereiten/umsetzen, sobald Zielhost und Reverse-Proxy-Pfad eindeutig feststehen.
3. Danach echten Import gegen einen legalen Testlink sowie Jellyfin-Refresh auf dem Zielhost verifizieren.

7
Dockerfile Normal file
View File

@@ -0,0 +1,7 @@
FROM python:3.12-slim
WORKDIR /app/backend
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg ca-certificates && rm -rf /var/lib/apt/lists/*
COPY backend/ ./
RUN pip install --no-cache-dir .[test] yt-dlp
EXPOSE 8080
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]

632
Plan-Botomir.md Normal file
View File

@@ -0,0 +1,632 @@
# Kino-Projekt Implementation Plan
> **For Hermes:** Use subagent-driven-development skill to implement this plan task-by-task.
**Goal:** Eine interne WebUI namens „Kino-Projekt“, mit der YouTube-Videos sowie Links aus legalen Mediatheken/öffentlich erlaubten Quellen geprüft, heruntergeladen/importiert, sauber für Jellyfin abgelegt und anschließend in Jellyfin aktualisiert werden.
**Architecture:** Das System besteht aus einer Mesh-only WebUI, einem FastAPI-Backend, einer SQLite-Datenbank, einer Job-Queue für Imports und einem Provider-System. Provider sind strikt getrennt: YouTube, Mediathek-/Direktlink-Import, Internet Archive und später weitere legale Quellen. Downloads laufen nur über geprüfte Provider und definierte Zielpfade; Jellyfin wird danach per API aktualisiert.
**Tech Stack:** Python 3.12+, FastAPI, SQLite, SQLModel oder SQLAlchemy, yt-dlp, httpx, Docker Compose, Jellyfin API, Nginx Proxy Manager/Headscale-Mesh.
---
## Scope
Der Projektname bleibt **Kino-Projekt**.
Der initiale, rechtlich saubere Scope ist:
1. YouTube-Videos, Playlists und Kanäle importieren, soweit öffentlich erreichbar und nicht DRM-/Login-/Paywall-geschützt.
2. Links aus legalen Mediatheken oder erlaubten Quellen hinzufügen, z. B. öffentlich verfügbare ARD/ZDF/Arte/3sat-/Mediathek-URLs, Internet Archive, eigene Download-URLs oder andere Quellen mit klar erlaubter Nutzung.
3. Downloads in Jellyfin-kompatibler Struktur ablegen.
4. Jellyfin-Library-Refresh auslösen.
5. UI zeigt Status, Quelle, Zielpfad und Fehler transparent an.
## Non-Goals
Nicht implementieren:
- Scraping/Download von Piracy-Seiten wie Kinox/Kinoger/xstream-Hostern.
- Umgehung von DRM, Paywalls, Logins, Geoblocking, Altersprüfungen oder Zugriffsbeschränkungen.
- Cookie-Import oder Browser-Session-Nutzung als Standardfunktion.
- Beliebige Shell-Parameter aus der UI.
- Freie Schreibpfade außerhalb der konfigurierten Medienordner.
---
## Ziel-UX
Die WebUI soll sich wie ein internes „Kino-Dashboard“ anfühlen:
- URL oder Suchbegriff eingeben.
- Quelle erkennen: YouTube, Mediathek, Direktlink, Internet Archive.
- Metadaten anzeigen: Titel, Kanal/Sender, Dauer, Beschreibung, Thumbnail, Quelle.
- Qualität/Zielbibliothek auswählen.
- Import starten.
- Fortschritt beobachten.
- Nach Abschluss Link zu Jellyfin bzw. Zielpfad anzeigen.
Beispiel-Ziel-URL:
```text
https://kino.dasposchi.de
```
Die WebUI wird nur über das Headscale-Mesh erreichbar gemacht.
---
## Datenmodell
### Tabelle: `sources`
```python
class Source(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
kind: str # youtube, mediathek, direct_url, internet_archive
url: str
title: str | None = None
provider: str | None = None
external_id: str | None = None
metadata_json: str = "{}"
created_at: datetime
```
### Tabelle: `import_jobs`
```python
class ImportJob(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
source_id: int = Field(foreign_key="source.id")
status: str # queued, probing, downloading, postprocessing, refreshing, done, failed
target_library: str
target_path: str | None = None
progress: float = 0.0
error: str | None = None
created_at: datetime
updated_at: datetime
```
---
## Provider-Konzept
Alle Quellen implementieren ein gemeinsames Interface.
```python
class Provider(Protocol):
name: str
def can_handle(self, url: str) -> bool: ...
async def probe(self, url: str) -> MediaMetadata: ...
async def download(self, url: str, target_dir: Path, options: DownloadOptions) -> DownloadResult: ...
```
### Provider 1: YouTube
- Erkennung über `youtube.com`, `youtu.be`, Playlist-/Channel-URLs.
- Metadaten via `yt-dlp --dump-json`.
- Download via `yt-dlp`, aber ohne Cookies und ohne Umgehungsoptionen.
- Qualitätsprofil: maximal 1080p als Default.
### Provider 2: Mediathek/Direktlink
- Für legale Mediathek-Links und direkte Mediendateien.
- Keine feste Domain-Allowlist, weil diese zu aufwändig zu pflegen wäre.
- Stattdessen konservative Sicherheitsprüfung pro URL:
- nur `http`/`https`
- DNS-Auflösung und Redirect-Kette prüfen
- keine privaten, loopback-, link-local- oder sonstigen internen Ziel-IP-Ranges
- Content-Type muss Video/Audio oder ein bekannter öffentlicher Stream-Typ sein
- Dateiendung/Container plausibilisieren
- Maximalgröße, Timeout und Rate-Limits erzwingen
- keine Cookies, keine Login-Flows, keine Umgehungsflags
- Wenn `yt-dlp` eine Mediathek-URL direkt unterstützt, darf derselbe sichere yt-dlp Wrapper genutzt werden.
- Keine DRM-/Paywall-/Login-/Geoblocking-Umgehung.
### Provider 3: Internet Archive
- API-/URL-basierter Import öffentlich verfügbarer Inhalte.
- Später als eigener Provider, wenn MVP stabil ist.
---
## Konfiguration
Datei: `.env`
```env
KINOPROJEKT_DB=/data/kino.sqlite3
KINOPROJEKT_MEDIA_ROOT=/jellyfin
KINOPROJEKT_TMP=/data/tmp
JELLYFIN_URL=http://jellyfin:8096
JELLYFIN_API_KEY=change-me
DEFAULT_MAX_HEIGHT=1080
MAX_DOWNLOAD_BYTES=15000000000
```
Secrets wie `JELLYFIN_API_KEY` kommen nicht nach Nextcloud, sondern in Vaultwarden oder eine geschützte `.env` auf dem Zielhost.
---
## Ziel-Dateistruktur
```text
kino-projekt/
backend/
app/
main.py
config.py
db.py
models.py
schemas.py
providers/
base.py
youtube.py
mediathek.py
internet_archive.py
services/
downloader.py
jellyfin.py
paths.py
metadata.py
web/
index.html
app.js
style.css
tests/
test_provider_detection.py
test_path_safety.py
test_youtube_probe.py
test_mediathek_probe.py
pyproject.toml
docker-compose.yml
README.md
docs/
provider-policy.md
deployment.md
```
---
# Tasks
### Task 1: Projektgerüst anlegen
**Objective:** Repository-Struktur, Python-Projekt und minimale FastAPI-App erstellen.
**Files:**
- Create: `backend/pyproject.toml`
- Create: `backend/app/main.py`
- Create: `backend/app/config.py`
- Create: `backend/tests/test_health.py`
**Steps:**
1. FastAPI-Abhängigkeiten eintragen.
2. `/health` Endpoint erstellen.
3. Test für `/health` schreiben.
4. `pytest` ausführen.
**Verification:**
```bash
cd backend
pytest -q
uvicorn app.main:app --host 0.0.0.0 --port 8080
curl http://localhost:8080/health
```
Expected:
```json
{"status":"ok"}
```
---
### Task 2: Konfiguration und Pfadsicherheit
**Objective:** Medien-/Temp-Pfade sicher konfigurieren und Path Traversal verhindern.
**Files:**
- Create: `backend/app/services/paths.py`
- Modify: `backend/app/config.py`
- Create: `backend/tests/test_path_safety.py`
**Rules:**
- Alle Downloads nur unter `KINOPROJEKT_TMP`.
- Alle finalen Dateien nur unter `KINOPROJEKT_MEDIA_ROOT`.
- Keine `../`-Escape-Möglichkeiten.
- Titel/Kanalnamen werden für Dateisysteme normalisiert.
**Verification:**
```bash
pytest backend/tests/test_path_safety.py -q
```
---
### Task 3: Datenbankmodelle erstellen
**Objective:** SQLite-Modelle für Quellen und Import-Jobs anlegen.
**Files:**
- Create: `backend/app/db.py`
- Create: `backend/app/models.py`
- Create: `backend/tests/test_models.py`
**Verification:**
```bash
pytest backend/tests/test_models.py -q
```
---
### Task 4: Provider-Interface definieren
**Objective:** Gemeinsame Provider-Abstraktion für YouTube, Mediathek und zukünftige Quellen schaffen.
**Files:**
- Create: `backend/app/providers/base.py`
- Create: `backend/tests/test_provider_detection.py`
**Implementation Notes:**
- `MediaMetadata` enthält Titel, Beschreibung, Thumbnail, Dauer, Provider, external_id.
- `DownloadOptions` enthält Qualität, Zielbibliothek und Audio-only-Flag.
- `DownloadResult` enthält Ausgabedateien und Metadatenpfade.
---
### Task 5: YouTube Provider Probe
**Objective:** YouTube-URLs erkennen und Metadaten über `yt-dlp --dump-json` abrufen.
**Files:**
- Create: `backend/app/providers/youtube.py`
- Create: `backend/tests/test_youtube_provider.py`
**Security Constraints:**
- Kein Cookie-Import.
- Keine Login-/DRM-/Paywall-Umgehung.
- Subprocess-Aufruf mit fester Argumentliste, keine Shell.
**Verification:**
```bash
python -m app.providers.youtube 'https://www.youtube.com/watch?v=PUBLIC_TEST_ID'
```
Expected: JSON-Metadaten oder sauberer Fehler.
---
### Task 6: Mediathek-/Direktlink Provider Probe
**Objective:** Legale Mediathek- und Direktlinks ohne feste Domain-Allowlist prüfen und Metadaten grob erfassen.
**Files:**
- Create: `backend/app/providers/mediathek.py`
- Create: `backend/tests/test_mediathek_provider.py`
**Rules:**
- Keine feste Domain-Allowlist, weil diese zu aufwändig zu pflegen wäre.
- Stattdessen Sicherheitsprüfung pro URL:
- nur `http`/`https`
- keine privaten/loopback/link-local IP-Ziele nach DNS-Auflösung
- Redirect-Kette prüfen
- Content-Type/Dateiendung plausibilisieren
- Maximalgröße/Timeout/Rate-Limits erzwingen
- keine Cookies, keine Login-Flows, keine Umgehungsflags
- Direkte Medien-URLs müssen erlaubte Content-Types liefern.
- yt-dlp darf als Extractor genutzt werden, wenn Quelle öffentlich und ohne Umgehung erreichbar ist.
---
### Task 7: Probe API erstellen
**Objective:** UI kann eine URL einreichen und Metadaten anzeigen.
**Files:**
- Modify: `backend/app/main.py`
- Create: `backend/app/schemas.py`
- Create: `backend/tests/test_probe_api.py`
**Endpoint:**
```http
POST /api/probe
Content-Type: application/json
{"url":"https://www.youtube.com/watch?v=..."}
```
**Response:**
```json
{
"provider": "youtube",
"title": "...",
"thumbnail": "...",
"duration_seconds": 123,
"allowed": true
}
```
---
### Task 8: Minimal-WebUI bauen
**Objective:** Eine einfache interne WebUI zum Einfügen und Prüfen von Links erstellen.
**Files:**
- Create: `backend/app/web/index.html`
- Create: `backend/app/web/style.css`
- Create: `backend/app/web/app.js`
- Modify: `backend/app/main.py`
**UI:**
- Titel: „Kino-Projekt“
- URL-Eingabe
- Button: „Quelle prüfen“
- Metadatenkarte
- Button: „Import starten“ zunächst disabled, bis Task 10.
---
### Task 9: Download Wrapper bauen
**Objective:** Sicheren Download-Service mit yt-dlp und festen Optionen implementieren.
**Files:**
- Create: `backend/app/services/downloader.py`
- Create: `backend/tests/test_downloader_args.py`
**YouTube Default Command:**
```bash
yt-dlp \
--no-playlist \
--write-thumbnail \
--write-info-json \
--merge-output-format mkv \
-f "bv*[height<=1080]+ba/b[height<=1080]/b" \
-o "<safe-target-template>" \
"<url>"
```
**Important:** In Python `subprocess.run([...], shell=False)` nutzen.
---
### Task 10: Import-Job API
**Objective:** Import-Jobs anlegen und ausführen.
**Files:**
- Modify: `backend/app/main.py`
- Create: `backend/app/services/jobs.py`
- Create: `backend/tests/test_jobs.py`
**Endpoints:**
```http
POST /api/imports
GET /api/imports/{job_id}
GET /api/imports
```
MVP darf Jobs synchron oder mit einfachem BackgroundTask ausführen. Später kann Redis/RQ ergänzt werden.
---
### Task 11: Jellyfin Refresh Service
**Objective:** Nach erfolgreichem Import Jellyfin-Library aktualisieren.
**Files:**
- Create: `backend/app/services/jellyfin.py`
- Create: `backend/tests/test_jellyfin.py`
**Endpoint intern:**
```http
POST {JELLYFIN_URL}/Library/Refresh
X-Emby-Token: {JELLYFIN_API_KEY}
```
**Verification:**
- Mit Test-API-Key gegen Jellyfin prüfen.
- Danach in Jellyfin sichtbar machen.
---
### Task 12: Docker Compose
**Objective:** Dienst als Container betreiben.
**Files:**
- Create: `Dockerfile`
- Create: `docker-compose.yml`
- Create: `.env.example`
**Services:**
```yaml
services:
kino-projekt:
build: .
ports:
- "127.0.0.1:8099:8080"
volumes:
- ./data:/data
- /jellyfin:/jellyfin
env_file:
- .env
```
### Task 13: App-Login einbauen
**Objective:** Zusätzlich zu Mesh-only einen einfachen App-Login mit sicherem Passwort-Hash und Session-Cookie einbauen.
**Files:**
- Create: `backend/app/services/auth.py`
- Modify: `backend/app/models.py`
- Modify: `backend/app/main.py`
- Modify: `backend/app/web/app.js`
- Create: `backend/tests/test_auth.py`
**Rules:**
- Passwort nie im Klartext speichern.
- Hashing mit `argon2-cffi` oder `passlib[bcrypt]`.
- Session-Cookie `HttpOnly`, `Secure`, `SameSite=Lax`.
- Initialer Admin-User wird über `.env`/Setup-Befehl erstellt, nicht im Repo.
**Verification:**
```bash
pytest backend/tests/test_auth.py -q
curl -kI https://kino.dasposchi.de/ # ohne Session: Redirect/Login oder 401
```
---
### Task 14: Mesh-only Reverse Proxy
**Objective:** WebUI nur im Headscale-Netz verfügbar machen.
**NPM Config:**
- Domain: `kino.dasposchi.de`
- Forward: Zielhost Port `8099`
- Access List: `headscale-mesh-only`
- SSL: Lets Encrypt
**Verification:**
```bash
curl -4 -kI https://kino.dasposchi.de/ # public: 403
curl --interface tailscale0 -kI https://kino.dasposchi.de/ # mesh: 200
```
---
### Task 15: Dokumentation
**Objective:** Nutzung und Grenzen dokumentieren.
**Files:**
- Create: `README.md`
- Create: `docs/provider-policy.md`
- Create: `docs/deployment.md`
**README enthält:**
- Projektname bleibt Kino-Projekt.
- Unterstützte Quellen: YouTube, legale Mediatheken, Direktlinks, Internet Archive später.
- Keine feste Mediathek-Domain-Allowlist; stattdessen URL-/Redirect-/IP-/Content-Type-/Größen-/Timeout-Sicherheitschecks.
- Nicht unterstützte Quellen: Piracy-/Hoster-Scraper, DRM, Paywalls, Login-geschützte Inhalte.
- Jellyfin-Konfiguration mit Medienwurzel `/jellyfin`.
- App-Login plus Mesh-only Deployment.
---
## Getroffene Entscheidungen
1. **Host:** Neue kleine Debian-VM oder Debian-LXC/CT auf Proxmox. Empfehlung: LXC/CT, wenn Jellyfin-Medienpfade sauber per Mount/Berechtigung eingebunden werden können; sonst VM.
2. **Zielbibliothek/Zielordner:** Inhalte werden unter `/jellyfin` in passende Jellyfin-Ordner geladen. Die WebUI bietet dafür ein Zielprofil an, z. B. Film, Serie, YouTube/Mediathek. „Passender Ordner“ wird technisch so konkretisiert:
- Filme: `/jellyfin/Filme/<Titel> (<Jahr>)/...`
- Serien: `/jellyfin/Serien/<Serientitel>/Staffel<Nummer>/...` (bestehende Struktur nutzt z. B. `Staffel1`, `Staffel2`, nicht `Season 01`)
- YouTube/Mediathek-Clips ohne Serienstruktur: `/jellyfin/YouTube-Mediathek/<Kanal oder Sender>/<Titel> [<id>].mkv`
- `/jellyfin/YouTube-Mediathek` existiert aktuell noch nicht und soll beim Deployment mit passenden Rechten angelegt werden.
3. **Medienwurzel:** `/jellyfin`.
4. **Login:** Zusätzlich zu Mesh-only wird ein Login eingebaut. MVP: lokaler App-Login mit Passwort-Hash und Session-Cookie; optional später OIDC/Authelia.
5. **Mediathek-Allowlist:** Keine feste Allowlist. Stattdessen URL-Sicherheitsprüfungen, Content-Type-Prüfung, Redirect-Prüfung, Size-/Timeout-Limits und Verbot privater Ziel-IP-Ranges.
Empfohlener MVP:
- Host: neue kleine Debian-VM oder Debian-LXC/CT auf Proxmox.
- Domain: `kino.dasposchi.de`.
- Medienwurzel: `/jellyfin`.
- Bibliothek/Zielprofil: automatisch auswählbare Zielprofile für Film, Serie und YouTube/Mediathek.
- Provider zuerst: YouTube + generischer Mediathek-/Direktlink-Provider ohne Domain-Allowlist, aber mit strikten Sicherheitschecks.
- Zugriff: Headscale-Mesh-only plus App-Login.
---
## Konkretisierung: „passender Jellyfin-Ordner“
Die WebUI soll nicht nach einer abstrakten Jellyfin-Bibliothek fragen, sondern nach einem **Zielprofil**. Das Zielprofil bestimmt den Ordner unter `/jellyfin` und die Dateibenennung.
MVP-Zielprofile:
1. **Film**
- Ziel: `/jellyfin/Filme/<Titel> (<Jahr>)/<Titel> (<Jahr>).mkv`
- Nutzer kann Jahr optional manuell korrigieren.
2. **Serie/Episode**
- Ziel: `/jellyfin/Serien/<Serientitel>/Staffel<SS>/<Serientitel> - S<SS>E<EE> - <Episodentitel>.mkv`
- Bestehende Ordner nutzen `Staffel1`, `Staffel2`, `Staffel3` ohne Leerzeichen/führende Null.
- Staffel/Episode können manuell gesetzt werden, falls Metadaten fehlen.
3. **YouTube/Mediathek-Clip**
- Ziel: `/jellyfin/YouTube-Mediathek/<Kanal oder Sender>/<Titel> [<id>].mkv`
- Default für einzelne YouTube-/Mediathek-Videos.
Die vorhandene Jellyfin-Ordnerstruktur muss vor der Umsetzung einmal geprüft werden; falls die realen Ordnernamen anders sind, werden die Zielprofile entsprechend angepasst.
---
## Verifizierte Jellyfin-Struktur
Geprüft auf Proxmox CT `100` (`jellyfin`, IP `192.168.178.222`). Jellyfin ist aktiv und lauscht auf `8096`.
Jellyfin-Bibliotheken laut Konfiguration:
- Filme: `/jellyfin/Filme`
- Serien: `/jellyfin/Serien`
Rechte/Owner:
- `/jellyfin`: `jellyfinuser:media`, Modus `775`
- `/jellyfin/Filme`: `jellyfinuser:media`, setgid-Gruppe `media`
- `/jellyfin/Serien`: `jellyfinuser:media`, setgid-Gruppe `media`
- Jellyfin-Prozessuser `jellyfin` ist Mitglied der Gruppe `media`
Beispiele aus der vorhandenen Struktur:
```text
/jellyfin/Filme/Der Super Mario Bros. Film (2023)/Der Super Mario Bros. Film (2023).mp4
/jellyfin/Serien/The White Lotus/Staffel1/The-White-Lotus-S01E02-German-720p-WEB-h264-WvF.mp4
```
Für das Kino-Projekt bedeutet das:
- Die Annahme `/jellyfin` als Medienwurzel ist korrekt.
- Filme und Serien sollen direkt in die vorhandenen Ordner `/jellyfin/Filme` und `/jellyfin/Serien` geschrieben werden.
- Serien-Staffelordner sollen zur vorhandenen Struktur passend `Staffel1`, `Staffel2`, ... heißen.
- Für YouTube-/Mediathek-Einzelclips soll neu `/jellyfin/YouTube-Mediathek` angelegt werden, idealerweise ebenfalls `jellyfinuser:media` und setgid.
---
## Akzeptanzkriterien MVP
- `https://kino.dasposchi.de` ist öffentlich blockiert und im Mesh erreichbar.
- Zusätzlich ist ein App-Login aktiv.
- YouTube-URL kann geprüft werden.
- Mediathek-/Direktlink kann ohne feste Domain-Allowlist geprüft werden, sofern URL-, Redirect-, IP-, Content-Type-, Größen- und Timeout-Sicherheitschecks bestehen.
- Import-Job lädt Datei unter `/jellyfin` in das gewählte Zielprofil bzw. den passenden Jellyfin-Ordner.
- Jellyfin Refresh wird ausgelöst.
- Status und Fehler sind in der UI sichtbar.
- Passwort-Hashes/Secrets liegen nicht in Nextcloud oder im Repository.
- Keine nicht erlaubten Quellen/Umgehungsmechanismen sind implementiert.

7
Projektbeschreibung.md Normal file
View File

@@ -0,0 +1,7 @@
Folgendes soll umgesetzt werden;:
Eine WebUI mit der man in der Lage ist, Serien und Filme zu suchen.
Durchsucht werden dabei gängige Hosting-Seiten, wie kinox.to, kinoger.com und co - orientieren an das kodi-plugin xstream.
Zusätzlich soll ein Download stattfinden und die Datei auf den jellyfin-server übertragen und die bibliothek dort refresht werden.
Die Repository zu xstream habe ich zur analyse ebenfalls hochgeladen.

51
README.md Normal file
View File

@@ -0,0 +1,51 @@
# Kino-Projekt
Interne WebUI zum Prüfen und Importieren von Inhalten aus legalen Quellen in Jellyfin.
## Sicherheitsgrenzen
Dieses Projekt implementiert bewusst **keine** Scraper für Piracy-Seiten, keine DRM-/Paywall-/Login-Umgehung und keinen Cookie-Import. Unterstützt werden YouTube-Metadaten via `yt-dlp`, öffentliche Internet-Archive-Items sowie konservativ geprüfte öffentliche Medien-/Mediathek-URLs.
## Entwicklung
```bash
cd backend
python -m venv .venv
. .venv/bin/activate
pip install -e '.[test]'
pytest -q
uvicorn app.main:app --host 0.0.0.0 --port 8080
```
Healthcheck: `curl http://localhost:8080/health`
## API-MVP
- `POST /api/probe` prüft eine URL und liefert Metadaten, ohne einen Download zu starten.
- `POST /api/imports` legt Source + Import-Job in SQLite an und startet den Import im Hintergrund.
- `GET /api/imports` und `GET /api/imports/{id}` zeigen Status/Fortschritt/Fehler an.
- `GET /api/sources` listet bekannte Quellen.
## Zielprofile
Die WebUI bietet drei Jellyfin-Zielprofile an:
- `film``/jellyfin/Filme/<Titel> (<Jahr>)/`
- `serie``/jellyfin/Serien/<Serientitel>/Staffel<N>/<SxxExx - Episodentitel>/`
- `clip``/jellyfin/YouTube-Mediathek/<Kanal oder Quelle>/<Titel> [<id>]/`
Optionale Titel-/Jahr-/Serienfelder dienen nur zur Zielpfad-Korrektur; Pfade werden serverseitig normalisiert und bleiben unter `KINOPROJEKT_MEDIA_ROOT`.
## App-Login
Wenn `KINOPROJEKT_ADMIN_PASSWORD_HASH` und `KINOPROJEKT_SESSION_SECRET` gesetzt sind, schützt ein lokaler App-Login die API. Das Passwort wird nur als Argon2-Hash gespeichert; die Session liegt in einem `HttpOnly`, `Secure`, `SameSite=Lax` Cookie.
Hash/Secret für die geschützte Zielhost-`.env` erzeugen:
```bash
cd backend
python -c "from app.services.auth import hash_password; import getpass; print(hash_password(getpass.getpass()))"
python -c "import secrets; print(secrets.token_urlsafe(48))"
```
Jellyfin wird nur aktualisiert, wenn `JELLYFIN_API_KEY` in einer geschützten Zielhost-Umgebung gesetzt ist; Secrets gehören nicht in Nextcloud.

0
backend/app/__init__.py Normal file
View File

53
backend/app/config.py Normal file
View File

@@ -0,0 +1,53 @@
from functools import lru_cache
from pathlib import Path
from urllib.parse import unquote, urlparse
from pydantic import Field, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
def sqlite_url_to_path(value: str | None) -> Path | None:
if not value:
return None
parsed = urlparse(value)
if parsed.scheme != "sqlite":
return None
if parsed.netloc and parsed.netloc != "localhost":
return None
return Path(unquote(parsed.path))
class Settings(BaseSettings):
db_path: Path = Field(default=Path("/data/kino.sqlite3"), alias="KINOPROJEKT_DB")
legacy_db_url: str | None = Field(default=None, alias="KINOPROJEKT_DB_URL")
media_root: Path = Field(default=Path("/jellyfin"), alias="KINOPROJEKT_MEDIA_ROOT")
tmp_dir: Path = Field(default=Path("/data/tmp"), alias="KINOPROJEKT_TMP")
jellyfin_url: str = Field(default="http://jellyfin:8096", alias="JELLYFIN_URL")
jellyfin_api_key: str | None = Field(default=None, alias="JELLYFIN_API_KEY")
default_max_height: int = Field(default=1080, alias="DEFAULT_MAX_HEIGHT")
max_download_bytes: int = Field(default=15_000_000_000, alias="MAX_DOWNLOAD_BYTES")
request_timeout: float = Field(default=15.0, alias="KINOPROJEKT_REQUEST_TIMEOUT")
admin_username: str = Field(default="admin", alias="KINOPROJEKT_ADMIN_USER")
admin_password_hash: str | None = Field(default=None, alias="KINOPROJEKT_ADMIN_PASSWORD_HASH")
session_secret: str | None = Field(default=None, alias="KINOPROJEKT_SESSION_SECRET")
rsync_target: str | None = Field(default=None, alias="KINOPROJEKT_RSYNC_TARGET")
rsync_ssh_key: Path | None = Field(default=None, alias="KINOPROJEKT_RSYNC_SSH_KEY")
rsync_remote_root: Path = Field(default=Path("/jellyfin"), alias="KINOPROJEKT_RSYNC_REMOTE_ROOT")
@model_validator(mode="after")
def apply_legacy_db_url(self) -> "Settings":
# The first target-host deployment used KINOPROJEKT_DB_URL=sqlite:////...
# Keep it working unless the newer KINOPROJEKT_DB path is explicitly set.
fields_set = getattr(self, "__pydantic_fields_set__", set())
if "db_path" not in fields_set:
legacy_path = sqlite_url_to_path(self.legacy_db_url)
if legacy_path is not None:
self.db_path = legacy_path
return self
model_config = SettingsConfigDict(env_file=".env", extra="ignore")
@lru_cache
def get_settings() -> Settings:
return Settings()

22
backend/app/db.py Normal file
View File

@@ -0,0 +1,22 @@
from pathlib import Path
from sqlmodel import SQLModel, Session, create_engine
from app.config import get_settings
def get_engine(db_path: Path | None = None):
settings = get_settings()
path = db_path or settings.db_path
path.parent.mkdir(parents=True, exist_ok=True)
return create_engine(f"sqlite:///{path}", connect_args={"check_same_thread": False})
def init_db(engine=None) -> None:
SQLModel.metadata.create_all(engine or get_engine())
def get_session():
engine = get_engine()
with Session(engine) as session:
yield session

304
backend/app/main.py Normal file
View File

@@ -0,0 +1,304 @@
from contextlib import asynccontextmanager
from fastapi import BackgroundTasks, Depends, FastAPI, HTTPException, Request
from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from sqlmodel import Session, select
from app.config import get_settings
from app.db import get_engine, get_session, init_db
from app.models import ImportJob, Source
from app.providers.base import ProviderError
from app.providers.internet_archive import InternetArchiveProvider
from app.providers.mediathek import MediathekProvider
from app.providers.youtube import YouTubeProvider
from app.schemas import (
AnalyzeRequest,
AnalyzeResponse,
BrowserCaptureRequest,
ImportJobResponse,
ImportRequest,
InteractiveAnalyzeClickRequest,
InteractiveAnalyzeKeyRequest,
InteractiveAnalyzeResponse,
InteractiveAnalyzeStartRequest,
MediaCandidateResponse,
ProbeRequest,
ProbeResponse,
SourceResponse,
)
from app.services.downloader import metadata_json, run_import_job
from app.services.interactive_browser import interactive_browser_manager
from app.services.page_analyzer import analyze_browser_capture, analyze_page
from app.services.url_safety import UnsafeUrlError
from app.services.auth import LoginPayload, auth_enabled, clear_session_cookie, login_response, require_user
providers = [YouTubeProvider(), InternetArchiveProvider(), MediathekProvider()]
engine = get_engine()
@asynccontextmanager
async def lifespan(app: FastAPI):
app.state.settings = get_settings()
init_db(engine)
yield
app = FastAPI(title="Kino-Projekt", version="0.2.0", lifespan=lifespan)
app.state.settings = get_settings()
def pick_provider(url: str):
return next((p for p in providers if p.can_handle(url)), None)
IMPORTABLE_CANDIDATE_KINDS = {"direct_video", "hls_manifest", "dash_manifest"}
def candidate_import_score(candidate) -> tuple[int, int]:
kind_priority = {"direct_video": 3, "hls_manifest": 2, "dash_manifest": 2}.get(candidate.kind, 0)
return kind_priority, int(candidate.content_length or 0)
async def resolve_importable_url(url: str):
"""Return a provider/metadata for an importable media URL.
Users often paste the player/page URL first. If that page is reachable as
text/html, the generic provider cannot import it directly; analyze it and
switch to the best discovered media candidate before creating the job.
"""
provider = pick_provider(url)
if provider is None:
raise HTTPException(status_code=400, detail="No legal provider accepted this URL")
try:
metadata = await provider.probe(url)
return url, provider, metadata, None
except ProviderError as exc:
original_error = exc
if "text/html" not in str(exc).lower():
raise
candidates = await analyze_page(url)
allowed = [
candidate
for candidate in candidates
if candidate.allowed and candidate.kind in IMPORTABLE_CANDIDATE_KINDS
]
if not allowed:
raise HTTPException(
status_code=400,
detail=(
"HTML page is reachable, but no importable media URL was found automatically. "
"Use 'Seite analysieren' or the Browser-Capture bookmarklet after playing the video, "
"then import one of the discovered media candidates."
),
) from original_error
candidate = max(allowed, key=candidate_import_score)
provider = pick_provider(candidate.url)
if provider is None:
raise HTTPException(status_code=400, detail="No legal provider accepted the discovered media URL")
metadata = await provider.probe(candidate.url)
return candidate.url, provider, metadata, candidate
@app.get("/health")
def health() -> dict[str, str]:
return {"status": "ok"}
@app.get("/api/auth/status")
def auth_status(request: Request) -> dict[str, bool | str | None]:
settings = request.app.state.settings
user = None
if auth_enabled(settings):
try:
current = require_user(request)
user = current.username if current else None
except HTTPException:
user = None
return {"enabled": auth_enabled(settings), "authenticated": user is not None or not auth_enabled(settings), "username": user}
@app.post("/api/auth/login", status_code=204)
def login(payload: LoginPayload, request: Request):
return login_response(payload, request, request.app.state.settings)
@app.post("/api/auth/logout", status_code=204)
def logout(request: Request):
return clear_session_cookie(request)
@app.post("/api/probe", response_model=ProbeResponse)
async def probe(payload: ProbeRequest, _: object = Depends(require_user)) -> ProbeResponse:
url = str(payload.url)
provider = pick_provider(url)
if provider is None:
raise HTTPException(status_code=400, detail="No legal provider accepted this URL")
try:
resolved_url, provider, metadata, candidate = await resolve_importable_url(url)
except ProviderError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
response = metadata.model_dump()
if candidate is not None:
response["title"] = candidate.title or metadata.title
response["external_id"] = resolved_url
response["description"] = "Automatisch aus der HTML-Seite erkannte Medienquelle"
return ProbeResponse(**response, allowed=True)
@app.post("/api/analyze", response_model=AnalyzeResponse)
async def analyze(payload: AnalyzeRequest, _: object = Depends(require_user)) -> AnalyzeResponse:
try:
candidates = await analyze_page(str(payload.url))
except UnsafeUrlError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
except Exception as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
return AnalyzeResponse(
page_url=str(payload.url),
candidates=[MediaCandidateResponse(**candidate.__dict__) for candidate in candidates],
)
@app.post("/api/browser-capture", response_model=AnalyzeResponse)
async def browser_capture(payload: BrowserCaptureRequest, _: object = Depends(require_user)) -> AnalyzeResponse:
try:
candidates = await analyze_browser_capture(
[item.model_dump() for item in payload.candidates],
str(payload.page_url) if payload.page_url else None,
)
except Exception as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
return AnalyzeResponse(
page_url=str(payload.page_url) if payload.page_url else "browser-capture",
candidates=[MediaCandidateResponse(**candidate.__dict__) for candidate in candidates],
)
async def interactive_response(session) -> InteractiveAnalyzeResponse:
candidates = await interactive_browser_manager.candidates(session)
return InteractiveAnalyzeResponse(
session_id=session.id,
page_url=session.page.url or session.page_url,
screenshot=await interactive_browser_manager.screenshot_data_url(session),
viewport={"width": 1280, "height": 720},
candidates=[MediaCandidateResponse(**candidate.__dict__) for candidate in candidates],
ublock_origin=bool(session.ublock_path),
)
@app.post("/api/interactive-analyze", response_model=InteractiveAnalyzeResponse)
async def start_interactive_analyze(payload: InteractiveAnalyzeStartRequest, _: object = Depends(require_user)) -> InteractiveAnalyzeResponse:
try:
session = await interactive_browser_manager.start(str(payload.url))
return await interactive_response(session)
except UnsafeUrlError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
except Exception as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
@app.post("/api/interactive-analyze/{session_id}/click", response_model=InteractiveAnalyzeResponse)
async def click_interactive_analyze(session_id: str, payload: InteractiveAnalyzeClickRequest, _: object = Depends(require_user)) -> InteractiveAnalyzeResponse:
try:
session = await interactive_browser_manager.click(session_id, payload.x, payload.y)
return await interactive_response(session)
except KeyError as exc:
raise HTTPException(status_code=404, detail="Interactive analysis session not found") from exc
except Exception as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
@app.post("/api/interactive-analyze/{session_id}/key", response_model=InteractiveAnalyzeResponse)
async def key_interactive_analyze(session_id: str, payload: InteractiveAnalyzeKeyRequest, _: object = Depends(require_user)) -> InteractiveAnalyzeResponse:
try:
session = await interactive_browser_manager.press(session_id, payload.key)
return await interactive_response(session)
except KeyError as exc:
raise HTTPException(status_code=404, detail="Interactive analysis session not found") from exc
except Exception as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
@app.delete("/api/interactive-analyze/{session_id}", status_code=204)
async def stop_interactive_analyze(session_id: str, _: object = Depends(require_user)) -> None:
await interactive_browser_manager.stop(session_id)
@app.post("/api/imports", response_model=ImportJobResponse, status_code=202)
async def create_import(
request: ImportRequest,
background_tasks: BackgroundTasks,
_: object = Depends(require_user),
session: Session = Depends(get_session),
) -> ImportJob:
url = str(request.url)
provider = pick_provider(url)
if provider is None:
raise HTTPException(status_code=400, detail="No legal provider accepted this URL")
try:
resolved_url, provider, metadata, candidate = await resolve_importable_url(url)
except ProviderError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
source = Source(
kind=provider.name,
provider=provider.name,
url=resolved_url,
title=request.title or (candidate.title if candidate is not None else None) or metadata.title,
external_id=metadata.external_id,
metadata_json=metadata_json(
metadata,
import_request={
"source_page_url": url if resolved_url != url else None,
"discovered_media_url": resolved_url if resolved_url != url else None,
"discovered_candidate_kind": candidate.kind if candidate is not None else None,
"target_profile": request.target_profile,
"title": request.title,
"year": request.year,
"series_title": request.series_title,
"season": request.season,
"episode": request.episode,
"episode_title": request.episode_title,
},
),
)
session.add(source)
session.commit()
session.refresh(source)
job = ImportJob(source_id=source.id, target_library=request.target_profile, status="queued")
session.add(job)
session.commit()
session.refresh(job)
background_tasks.add_task(run_import_job, job.id, engine, providers, get_settings())
return job
@app.get("/api/imports", response_model=list[ImportJobResponse])
def list_imports(_: object = Depends(require_user), session: Session = Depends(get_session)) -> list[ImportJob]:
return list(session.exec(select(ImportJob).order_by(ImportJob.created_at.desc())).all())
@app.get("/api/imports/{job_id}", response_model=ImportJobResponse)
def get_import(job_id: int, _: object = Depends(require_user), session: Session = Depends(get_session)) -> ImportJob:
job = session.get(ImportJob, job_id)
if job is None:
raise HTTPException(status_code=404, detail="Import job not found")
return job
@app.get("/api/sources", response_model=list[SourceResponse])
def list_sources(_: object = Depends(require_user), session: Session = Depends(get_session)) -> list[Source]:
return list(session.exec(select(Source).order_by(Source.created_at.desc())).all())
app.mount("/static", StaticFiles(directory="app/web"), name="static")
@app.get("/", include_in_schema=False)
def index() -> FileResponse:
return FileResponse("app/web/index.html", headers={"Cache-Control": "no-store"})

30
backend/app/models.py Normal file
View File

@@ -0,0 +1,30 @@
from datetime import datetime, timezone
from sqlmodel import Field, SQLModel
def utcnow() -> datetime:
return datetime.now(timezone.utc)
class Source(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
kind: str
url: str
title: str | None = None
provider: str | None = None
external_id: str | None = None
metadata_json: str = "{}"
created_at: datetime = Field(default_factory=utcnow)
class ImportJob(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
source_id: int = Field(foreign_key="source.id")
status: str = "queued"
target_library: str
target_path: str | None = None
progress: float = 0.0
error: str | None = None
created_at: datetime = Field(default_factory=utcnow)
updated_at: datetime = Field(default_factory=utcnow)

View File

View File

@@ -0,0 +1,46 @@
from collections.abc import Callable
from pathlib import Path
from typing import Protocol
from pydantic import BaseModel
class ProviderError(RuntimeError):
pass
class MediaMetadata(BaseModel):
provider: str
title: str | None = None
description: str | None = None
thumbnail: str | None = None
duration_seconds: int | None = None
external_id: str | None = None
class DownloadOptions(BaseModel):
max_height: int = 1080
target_library: str = "movies"
audio_only: bool = False
max_bytes: int = 15_000_000_000
class DownloadResult(BaseModel):
output_files: list[Path]
metadata_files: list[Path] = []
class Provider(Protocol):
name: str
def can_handle(self, url: str) -> bool: ...
async def probe(self, url: str) -> MediaMetadata: ...
async def download(
self,
url: str,
target_dir: Path,
options: DownloadOptions,
progress_callback: Callable[[float], None] | None = None,
) -> DownloadResult: ...

View File

@@ -0,0 +1,122 @@
from collections.abc import Callable
from pathlib import Path
from urllib.parse import quote, urlparse
import httpx
from app.providers.base import DownloadOptions, DownloadResult, MediaMetadata, ProviderError
from app.providers.mediathek import MEDIA_EXTENSIONS, MediathekProvider
ARCHIVE_HOSTS = {"archive.org", "www.archive.org"}
METADATA_BASE = "https://archive.org/metadata"
DOWNLOAD_BASE = "https://archive.org/download"
class InternetArchiveProvider:
name = "internet_archive"
def can_handle(self, url: str) -> bool:
parsed = urlparse(url)
return parsed.scheme in {"http", "https"} and (parsed.hostname or "").lower() in ARCHIVE_HOSTS and self._identifier_from_url(url) is not None
async def probe(self, url: str) -> MediaMetadata:
identifier = self._identifier_from_url(url)
if not identifier:
raise ProviderError("Internet Archive URL must contain an item identifier")
metadata = await self._metadata(identifier)
meta = metadata.get("metadata") or {}
files = metadata.get("files") or []
selected = self._select_media_file(files)
if selected is None:
raise ProviderError("Internet Archive item has no accepted public media file")
title = self._string(meta.get("title")) or identifier
description = self._string(meta.get("description"))
return MediaMetadata(
provider=self.name,
title=title,
description=description,
external_id=identifier,
)
async def download(
self,
url: str,
target_dir: Path,
options: DownloadOptions,
progress_callback: Callable[[float], None] | None = None,
) -> DownloadResult:
identifier = self._identifier_from_url(url)
if not identifier:
raise ProviderError("Internet Archive URL must contain an item identifier")
metadata = await self._metadata(identifier)
selected = self._select_media_file(metadata.get("files") or [])
if selected is None:
raise ProviderError("Internet Archive item has no accepted public media file")
file_name = selected["name"]
safe_url = f"{DOWNLOAD_BASE}/{quote(identifier, safe='')}/{quote(file_name, safe='/')}"
return await MediathekProvider().download(safe_url, target_dir, options, progress_callback=progress_callback)
async def _metadata(self, identifier: str) -> dict:
url = f"{METADATA_BASE}/{quote(identifier, safe='')}"
async with httpx.AsyncClient(timeout=httpx.Timeout(15, read=20), follow_redirects=True) as client:
response = await client.get(url)
if response.status_code == 404:
raise ProviderError(f"Internet Archive item not found: {identifier}")
response.raise_for_status()
data = response.json()
if not isinstance(data, dict):
raise ProviderError("Internet Archive metadata response was not an object")
return data
def _identifier_from_url(self, url: str) -> str | None:
parsed = urlparse(url)
parts = [part for part in parsed.path.split("/") if part]
if not parts:
return None
if parts[0] in {"details", "download", "metadata"} and len(parts) >= 2:
return parts[1]
return parts[0]
def _select_media_file(self, files: list[dict]) -> dict | None:
candidates: list[tuple[int, dict]] = []
for item in files:
if not isinstance(item, dict):
continue
name = item.get("name")
if not isinstance(name, str) or not name or name.endswith("/") or ".." in Path(name).parts:
continue
lower = name.lower()
ext = Path(lower).suffix
fmt = str(item.get("format") or "").lower()
source = str(item.get("source") or "").lower()
if ext not in MEDIA_EXTENSIONS and "mpeg4" not in fmt and "h.264" not in fmt:
continue
if source in {"metadata", "derivative"} and ext not in {".mp4", ".m4v", ".mp3", ".m4a", ".ogg", ".webm"}:
continue
size = self._int_or_zero(item.get("size"))
# Prefer original/public mp4-ish media, then larger files as likely higher quality.
score = 0
if source == "original":
score += 1000
if ext in {".mp4", ".m4v", ".webm"}:
score += 500
if size:
score += min(size // 1_000_000, 400)
candidates.append((score, item))
if not candidates:
return None
candidates.sort(key=lambda pair: pair[0], reverse=True)
return candidates[0][1]
def _int_or_zero(self, value: object) -> int:
try:
return int(value) # type: ignore[arg-type]
except (TypeError, ValueError):
return 0
def _string(self, value: object) -> str | None:
if isinstance(value, str):
return value.strip() or None
if isinstance(value, list) and value and isinstance(value[0], str):
return value[0].strip() or None
return None

View File

@@ -0,0 +1,193 @@
import asyncio
import ipaddress
import shutil
import socket
import sys
from collections.abc import Callable
from pathlib import Path, PurePosixPath
from urllib.parse import urlparse
import httpx
from app.providers.base import DownloadOptions, DownloadResult, MediaMetadata, ProviderError
from app.providers.youtube import parse_yt_dlp_progress
MEDIA_CONTENT_TYPES = (
"video/",
"audio/",
"application/vnd.apple.mpegurl",
"application/x-mpegurl",
"application/dash+xml",
)
MEDIA_EXTENSIONS = {
".mp4",
".mkv",
".webm",
".mov",
".m4v",
".mp3",
".m4a",
".aac",
".ogg",
".flac",
".m3u8",
".mpd",
}
MANIFEST_CONTENT_TYPES = {"application/vnd.apple.mpegurl", "application/x-mpegurl", "application/dash+xml"}
MANIFEST_EXTENSIONS = {".m3u8", ".mpd"}
def is_stream_manifest(url: str, content_type: str | None = None) -> bool:
clean_content_type = (content_type or "").split(";", 1)[0].lower()
ext = PurePosixPath(urlparse(url).path).suffix.lower()
return clean_content_type in MANIFEST_CONTENT_TYPES or ext in MANIFEST_EXTENSIONS
def ytdlp_command() -> list[str]:
binary = shutil.which("yt-dlp")
if binary:
return [binary]
return [sys.executable, "-m", "yt_dlp"]
class MediathekProvider:
name = "mediathek"
def can_handle(self, url: str) -> bool:
parsed = urlparse(url)
host = parsed.hostname or ""
excluded_hosts = {"youtube.com", "www.youtube.com", "m.youtube.com", "youtu.be", "archive.org", "www.archive.org"}
return parsed.scheme in {"http", "https"} and host not in excluded_hosts
async def probe(self, url: str) -> MediaMetadata:
parsed = urlparse(url)
if parsed.scheme not in {"http", "https"} or not parsed.hostname:
raise ProviderError("Only http/https URLs are allowed")
self._reject_private_targets(parsed.hostname)
async with httpx.AsyncClient(follow_redirects=True, timeout=httpx.Timeout(10, read=10), max_redirects=5) as client:
response = await client.head(url)
if response.status_code in {405, 403}:
response = await client.get(url, headers={"Range": "bytes=0-0"})
for hop in [*response.history, response]:
host = hop.url.host
if host:
self._reject_private_targets(host)
content_type = response.headers.get("content-type", "").split(";", 1)[0].lower()
ext = PurePosixPath(urlparse(str(response.url)).path).suffix.lower()
if not (content_type.startswith(MEDIA_CONTENT_TYPES) or ext in MEDIA_EXTENSIONS):
raise ProviderError(f"URL is reachable, but content type/extension is not accepted: {content_type or ext or 'unknown'}")
title = PurePosixPath(urlparse(str(response.url)).path).name or str(response.url.host)
return MediaMetadata(provider=self.name, title=title, external_id=str(response.url))
async def download(
self,
url: str,
target_dir: Path,
options: DownloadOptions,
progress_callback: Callable[[float], None] | None = None,
) -> DownloadResult:
metadata = await self.probe(url)
target_dir.mkdir(parents=True, exist_ok=True)
source_url = metadata.external_id or url
if is_stream_manifest(source_url):
return await self._download_with_ytdlp(source_url, target_dir, options, progress_callback)
name = PurePosixPath(urlparse(source_url).path).name or "download.bin"
output_path = target_dir / name
written = 0
async with httpx.AsyncClient(follow_redirects=True, timeout=httpx.Timeout(20, read=60), max_redirects=5) as client:
async with client.stream("GET", source_url) as response:
response.raise_for_status()
for hop in [*response.history, response]:
host = hop.url.host
if host:
self._reject_private_targets(host)
total = int(response.headers.get("content-length") or 0)
with output_path.open("wb") as fh:
async for chunk in response.aiter_bytes():
written += len(chunk)
if written > options.max_bytes:
raise ProviderError("Download exceeds configured maximum size")
fh.write(chunk)
if total > 0 and progress_callback is not None:
progress_callback(min(written / total, 1.0))
if progress_callback is not None:
progress_callback(1.0)
return DownloadResult(output_files=[output_path])
async def _download_with_ytdlp(
self,
url: str,
target_dir: Path,
options: DownloadOptions,
progress_callback: Callable[[float], None] | None = None,
) -> DownloadResult:
"""Download HLS/DASH manifests as real media, not the tiny manifest file.
A .m3u8/.mpd is only a playlist/manifest. Saving it directly produces a
few-KB text file in Jellyfin instead of the actual media stream. Let
yt-dlp drive ffmpeg so segments are downloaded and remuxed into MP4.
"""
command = ytdlp_command()
output_template = str(target_dir / "%(title,playlist_index,id).120s.%(ext)s")
height_expr = f"bestvideo[height<={options.max_height}]+bestaudio/best[height<={options.max_height}]/best"
if options.audio_only:
height_expr = "bestaudio/best"
proc = await asyncio.create_subprocess_exec(
*command,
"--newline",
"--no-playlist",
"--no-warnings",
"--restrict-filenames",
"--format",
height_expr,
"--remux-video",
"mp4",
"--max-filesize",
str(options.max_bytes),
"--output",
output_template,
url,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
captured: list[str] = []
async def consume(stream: asyncio.StreamReader | None) -> None:
if stream is None:
return
while True:
line = await stream.readline()
if not line:
break
text = line.decode(errors="replace").strip()
if not text:
continue
captured.append(text)
if len(captured) > 80:
del captured[:40]
progress = parse_yt_dlp_progress(text)
if progress is not None and progress_callback is not None:
progress_callback(progress)
await asyncio.wait_for(asyncio.gather(consume(proc.stdout), consume(proc.stderr), proc.wait()), timeout=60 * 60)
if proc.returncode != 0:
detail = "\n".join(captured).strip() or "yt-dlp manifest download failed"
raise ProviderError(detail[:1000])
files = [p for p in target_dir.iterdir() if p.is_file()]
media_files = [p for p in files if p.suffix.lower() not in {".json", ".jpg", ".jpeg", ".png", ".webp", ".m3u8", ".mpd"}]
if not media_files:
raise ProviderError("yt-dlp finished without producing a media file")
if progress_callback is not None:
progress_callback(1.0)
return DownloadResult(output_files=media_files)
def _reject_private_targets(self, hostname: str) -> None:
try:
infos = socket.getaddrinfo(hostname, None, proto=socket.IPPROTO_TCP)
except socket.gaierror as exc:
raise ProviderError(f"DNS lookup failed for {hostname}") from exc
for info in infos:
ip = ipaddress.ip_address(info[4][0])
if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_multicast or ip.is_reserved or ip.is_unspecified:
raise ProviderError(f"Refusing private or non-public target address for {hostname}")

View File

@@ -0,0 +1,136 @@
import asyncio
import json
import re
import shutil
import sys
from collections.abc import Callable
from pathlib import Path
from urllib.parse import parse_qs, urlparse
from app.providers.base import DownloadOptions, DownloadResult, MediaMetadata, ProviderError
class YouTubeProvider:
name = "youtube"
allowed_hosts = {"youtube.com", "www.youtube.com", "m.youtube.com", "youtu.be"}
def can_handle(self, url: str) -> bool:
host = urlparse(url).hostname or ""
return host.lower() in self.allowed_hosts
async def probe(self, url: str) -> MediaMetadata:
if not self.can_handle(url):
raise ProviderError("Not a YouTube URL")
if shutil.which("yt-dlp") is None:
return MediaMetadata(provider=self.name, external_id=self._external_id(url), title="YouTube URL (yt-dlp not installed)")
proc = await asyncio.create_subprocess_exec(
"yt-dlp",
"--dump-single-json",
"--no-playlist",
"--no-warnings",
url,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=45)
if proc.returncode != 0:
raise ProviderError((stderr.decode(errors="replace") or "yt-dlp probe failed").strip()[:500])
data = json.loads(stdout.decode())
return MediaMetadata(
provider=self.name,
title=data.get("title"),
description=data.get("description"),
thumbnail=data.get("thumbnail"),
duration_seconds=data.get("duration"),
external_id=data.get("id") or self._external_id(url),
)
async def download(
self,
url: str,
target_dir: Path,
options: DownloadOptions,
progress_callback: Callable[[float], None] | None = None,
) -> DownloadResult:
if shutil.which("yt-dlp") is None:
raise ProviderError("yt-dlp is not installed")
target_dir.mkdir(parents=True, exist_ok=True)
height_expr = f"bestvideo[height<={options.max_height}]+bestaudio/best[height<={options.max_height}]/best"
if options.audio_only:
height_expr = "bestaudio/best"
output_template = str(target_dir / "%(title).120s [%(id)s].%(ext)s")
proc = await asyncio.create_subprocess_exec(
"yt-dlp",
"--newline",
"--no-playlist",
"--no-warnings",
"--restrict-filenames",
"--write-info-json",
"--write-thumbnail",
"--format",
height_expr,
"--output",
output_template,
url,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
captured: list[str] = []
async def consume(stream: asyncio.StreamReader | None) -> None:
if stream is None:
return
while True:
line = await stream.readline()
if not line:
break
text = line.decode(errors="replace").strip()
if not text:
continue
captured.append(text)
if len(captured) > 80:
del captured[:40]
progress = parse_yt_dlp_progress(text)
if progress is not None and progress_callback is not None:
progress_callback(progress)
await asyncio.wait_for(asyncio.gather(consume(proc.stdout), consume(proc.stderr), proc.wait()), timeout=60 * 60)
if proc.returncode != 0:
detail = "\n".join(captured).strip() or "yt-dlp download failed"
raise ProviderError(detail[:1000])
files = [p for p in target_dir.iterdir() if p.is_file()]
media_files = [p for p in files if p.suffix.lower() not in {".json", ".jpg", ".jpeg", ".png", ".webp"}]
metadata_files = [p for p in files if p not in media_files]
if not media_files:
raise ProviderError("yt-dlp finished without producing a media file")
if progress_callback is not None:
progress_callback(1.0)
return DownloadResult(output_files=media_files, metadata_files=metadata_files)
def _external_id(self, url: str) -> str | None:
parsed = urlparse(url)
if parsed.hostname == "youtu.be":
return parsed.path.strip("/") or None
return parse_qs(parsed.query).get("v", [None])[0]
def parse_yt_dlp_progress(line: str) -> float | None:
"""Return yt-dlp percent progress as 0.0..1.0, or None for non-progress lines."""
if "[download]" not in line:
return None
match = re.search(r"(\d+(?:\.\d+)?)%", line)
if not match:
return None
return max(0.0, min(float(match.group(1)) / 100.0, 1.0))
async def _main(url: str) -> None:
meta = await YouTubeProvider().probe(url)
print(meta.model_dump_json(indent=2))
if __name__ == "__main__":
if len(sys.argv) != 2:
raise SystemExit("usage: python -m app.providers.youtube <url>")
asyncio.run(_main(sys.argv[1]))

115
backend/app/schemas.py Normal file
View File

@@ -0,0 +1,115 @@
from datetime import datetime
from typing import Any
from pydantic import AnyHttpUrl, BaseModel, Field
class ProbeRequest(BaseModel):
url: AnyHttpUrl
class AnalyzeRequest(BaseModel):
url: AnyHttpUrl
class BrowserCapturedCandidate(BaseModel):
url: str
title: str | None = None
name: str | None = None
content_type: str | None = None
type: str | None = None
quality: str | None = None
class BrowserCaptureRequest(BaseModel):
page_url: AnyHttpUrl | None = None
candidates: list[BrowserCapturedCandidate] = Field(default_factory=list, max_length=80)
class MediaCandidateResponse(BaseModel):
url: str
kind: str
title: str | None = None
source: str
content_type: str | None = None
content_length: int | None = None
file_size: str | None = None
quality: str | None = None
allowed: bool = True
reason: str | None = None
metadata: dict[str, Any] = Field(default_factory=dict)
class AnalyzeResponse(BaseModel):
page_url: str
candidates: list[MediaCandidateResponse]
class InteractiveAnalyzeStartRequest(BaseModel):
url: AnyHttpUrl
class InteractiveAnalyzeClickRequest(BaseModel):
x: float = Field(ge=0)
y: float = Field(ge=0)
class InteractiveAnalyzeKeyRequest(BaseModel):
key: str = Field(min_length=1, max_length=40)
class InteractiveAnalyzeResponse(BaseModel):
session_id: str
page_url: str
screenshot: str
viewport: dict[str, int]
candidates: list[MediaCandidateResponse]
ublock_origin: bool = False
class ProbeResponse(BaseModel):
provider: str
title: str | None = None
description: str | None = None
thumbnail: str | None = None
duration_seconds: int | None = None
external_id: str | None = None
allowed: bool = True
class ImportRequest(BaseModel):
url: AnyHttpUrl
target_profile: str = Field(default="clip", pattern="^(film|serie|clip)$")
title: str | None = Field(default=None, max_length=180)
year: int | None = Field(default=None, ge=1888, le=2200)
series_title: str | None = Field(default=None, max_length=180)
season: int | None = Field(default=None, ge=1, le=200)
episode: int | None = Field(default=None, ge=1, le=1000)
episode_title: str | None = Field(default=None, max_length=180)
target_library: str | None = Field(default=None, max_length=80) # legacy API alias
max_height: int = Field(default=1080, ge=144, le=2160)
audio_only: bool = False
class ImportJobResponse(BaseModel):
id: int
source_id: int
status: str
target_library: str
target_path: str | None = None
progress: float
error: str | None = None
created_at: datetime
updated_at: datetime
class SourceResponse(BaseModel):
id: int
kind: str
url: str
title: str | None = None
provider: str | None = None
external_id: str | None = None
created_at: datetime

View File

View File

@@ -0,0 +1,122 @@
import base64
import hashlib
import hmac
import secrets
import time
from dataclasses import dataclass
from argon2 import PasswordHasher
from argon2.exceptions import InvalidHashError, VerifyMismatchError
from fastapi import HTTPException, Request, Response, status
from pydantic import BaseModel
from app.config import Settings
SESSION_COOKIE = "kino_session"
SESSION_TTL_SECONDS = 60 * 60 * 12
_ph = PasswordHasher()
class LoginPayload(BaseModel):
username: str
password: str
@dataclass(frozen=True)
class AuthenticatedUser:
username: str
def hash_password(password: str) -> str:
return _ph.hash(password)
def verify_password(password: str, password_hash: str) -> bool:
try:
return _ph.verify(password_hash, password)
except (VerifyMismatchError, InvalidHashError):
return False
def auth_enabled(settings: Settings) -> bool:
return bool(settings.admin_password_hash and settings.session_secret)
def _sign(value: str, secret: str) -> str:
return hmac.new(secret.encode("utf-8"), value.encode("utf-8"), hashlib.sha256).hexdigest()
def create_session_cookie(username: str, settings: Settings, now: int | None = None) -> str:
if not settings.session_secret:
raise RuntimeError("session secret is not configured")
issued = int(time.time() if now is None else now)
expires = issued + SESSION_TTL_SECONDS
payload = f"{username}:{expires}"
encoded = base64.urlsafe_b64encode(payload.encode("utf-8")).decode("ascii").rstrip("=")
return f"{encoded}.{_sign(encoded, settings.session_secret)}"
def parse_session_cookie(cookie_value: str | None, settings: Settings, now: int | None = None) -> AuthenticatedUser | None:
if not cookie_value or not settings.session_secret or "." not in cookie_value:
return None
encoded, signature = cookie_value.rsplit(".", 1)
if not hmac.compare_digest(signature, _sign(encoded, settings.session_secret)):
return None
try:
padding = "=" * (-len(encoded) % 4)
payload = base64.urlsafe_b64decode((encoded + padding).encode("ascii")).decode("utf-8")
username, expires_raw = payload.rsplit(":", 1)
expires = int(expires_raw)
except (ValueError, UnicodeDecodeError):
return None
current = int(time.time() if now is None else now)
if expires < current:
return None
if not secrets.compare_digest(username, settings.admin_username):
return None
return AuthenticatedUser(username=username)
def require_user(request: Request) -> AuthenticatedUser | None:
settings = request.app.state.settings
if not auth_enabled(settings):
return None
user = parse_session_cookie(request.cookies.get(SESSION_COOKIE), settings)
if user is None:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Login required")
return user
def should_use_secure_cookie(request: Request) -> bool:
forwarded_proto = request.headers.get("x-forwarded-proto", "").split(",", 1)[0].strip().lower()
return request.url.scheme == "https" or forwarded_proto == "https"
def login_response(payload: LoginPayload, request: Request, settings: Settings) -> Response:
if not auth_enabled(settings):
raise HTTPException(status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail="App login is not configured")
if not secrets.compare_digest(payload.username, settings.admin_username):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid username or password")
if not verify_password(payload.password, settings.admin_password_hash or ""):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid username or password")
response = Response(status_code=status.HTTP_204_NO_CONTENT)
response.set_cookie(
SESSION_COOKIE,
create_session_cookie(payload.username, settings),
httponly=True,
secure=should_use_secure_cookie(request),
samesite="lax",
max_age=SESSION_TTL_SECONDS,
)
return response
def clear_session_cookie(request: Request | None = None) -> Response:
response = Response(status_code=status.HTTP_204_NO_CONTENT)
response.delete_cookie(
SESSION_COOKIE,
httponly=True,
secure=should_use_secure_cookie(request) if request is not None else True,
samesite="lax",
)
return response

View File

@@ -0,0 +1,171 @@
import asyncio
import json
import shlex
import shutil
from pathlib import Path
from sqlmodel import Session
from app.config import Settings
from app.models import ImportJob, Source, utcnow
from app.providers.base import DownloadOptions, Provider, ProviderError
from app.services.jellyfin import refresh_jellyfin
from app.services.paths import jellyfin_profile_target, tmp_target
def copy_tree_contents(src: Path, dest: Path) -> list[Path]:
dest.mkdir(parents=True, exist_ok=True)
copied: list[Path] = []
for path in src.iterdir():
if path.is_symlink():
raise ProviderError(f"Refusing to copy symlink from import output: {path.name}")
target = dest / path.name
if path.is_dir():
for child in path.rglob("*"):
if child.is_symlink():
raise ProviderError(f"Refusing to copy symlink from import output: {child.relative_to(src)}")
if target.exists():
shutil.rmtree(target)
shutil.copytree(path, target)
copied.extend(p for p in target.rglob("*") if p.is_file())
else:
shutil.copy2(path, target)
copied.append(target)
return copied
async def sync_to_jellyfin_vm(local_dir: Path, settings: Settings) -> str | None:
"""Copy the completed local import directory to Jellyfin's media VM via rsync."""
if not settings.rsync_target or not settings.rsync_ssh_key:
return None
local_dir = local_dir.resolve()
media_root = settings.media_root.resolve()
if media_root != local_dir and media_root not in local_dir.parents:
raise ProviderError(f"Refusing to sync path outside media root: {local_dir}")
relative = local_dir.relative_to(media_root)
remote_root = Path(str(settings.rsync_remote_root)).as_posix().rstrip("/") or "/jellyfin"
remote_dir = f"{remote_root}/{relative.as_posix()}"
ssh_cmd = [
"ssh",
"-i",
str(settings.rsync_ssh_key),
"-o",
"BatchMode=yes",
"-o",
"StrictHostKeyChecking=accept-new",
]
mkdir_proc = await asyncio.create_subprocess_exec(
*ssh_cmd,
settings.rsync_target,
f"mkdir -p {shlex.quote(remote_dir)}",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
_, mkdir_stderr = await mkdir_proc.communicate()
if mkdir_proc.returncode != 0:
raise ProviderError(f"Jellyfin transfer mkdir failed: {mkdir_stderr.decode(errors='replace')[:500]}")
rsync_proc = await asyncio.create_subprocess_exec(
"rsync",
"-a",
"--delete",
"-s",
"-e",
" ".join(shlex.quote(part) for part in ssh_cmd),
f"{local_dir}/",
f"{settings.rsync_target}:{remote_dir}/",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
_, rsync_stderr = await rsync_proc.communicate()
if rsync_proc.returncode != 0:
raise ProviderError(f"Jellyfin transfer rsync failed: {rsync_stderr.decode(errors='replace')[:500]}")
return f"{settings.rsync_target}:{remote_dir}"
def update_job(session: Session, job: ImportJob, **fields) -> None:
for key, value in fields.items():
setattr(job, key, value)
job.updated_at = utcnow()
session.add(job)
session.commit()
session.refresh(job)
async def run_import_job(job_id: int, engine, providers: list[Provider], settings: Settings) -> None:
with Session(engine) as session:
job = session.get(ImportJob, job_id)
if job is None:
return
source = session.get(Source, job.source_id)
if source is None:
update_job(session, job, status="failed", error="Source missing")
return
provider = next((p for p in providers if p.name == source.provider or p.can_handle(source.url)), None)
if provider is None:
update_job(session, job, status="failed", error="No provider available for source")
return
try:
last_reported_progress = 0.0
def report_download_progress(provider_progress: float) -> None:
nonlocal last_reported_progress
mapped = 0.15 + (max(0.0, min(provider_progress, 1.0)) * 0.60)
if mapped - last_reported_progress >= 0.01 or mapped >= 0.75:
last_reported_progress = mapped
update_job(session, job, status="downloading", progress=round(mapped, 3))
update_job(session, job, status="downloading", progress=0.15)
title = source.title or source.external_id or "media"
stored_metadata = json.loads(source.metadata_json or "{}")
import_request = stored_metadata.get("import_request", {}) if isinstance(stored_metadata, dict) else {}
provider_metadata = stored_metadata.get("metadata", stored_metadata) if isinstance(stored_metadata, dict) else {}
tmp_dir = tmp_target(settings.tmp_dir, job.id or 0, title)
result = await provider.download(
source.url,
tmp_dir,
DownloadOptions(
max_height=settings.default_max_height,
target_library=job.target_library,
max_bytes=settings.max_download_bytes,
),
progress_callback=report_download_progress,
)
update_job(session, job, status="postprocessing", progress=0.75)
target_dir = jellyfin_profile_target(
settings.media_root,
job.target_library,
import_request.get("title") or title,
year=import_request.get("year"),
series_title=import_request.get("series_title"),
season=import_request.get("season"),
episode=import_request.get("episode"),
episode_title=import_request.get("episode_title"),
channel=provider_metadata.get("uploader") or provider_metadata.get("channel") or source.provider,
external_id=source.external_id,
)
copied = copy_tree_contents(tmp_dir, target_dir)
if not copied and not result.output_files:
raise ProviderError("No files were produced by the import")
remote_target = await sync_to_jellyfin_vm(target_dir, settings)
update_job(session, job, target_path=remote_target or str(target_dir), status="refreshing", progress=0.9)
refreshed = await refresh_jellyfin(settings)
note = None if refreshed else "Import completed; Jellyfin refresh skipped because no API key is configured"
update_job(session, job, status="done", progress=1.0, error=note)
except Exception as exc: # keep background task failures visible in DB/API
update_job(session, job, status="failed", error=str(exc)[:1000])
def schedule_import(job_id: int, engine, providers: list[Provider], settings: Settings) -> None:
asyncio.create_task(run_import_job(job_id, engine, providers, settings))
def metadata_json(metadata, *, import_request: dict | None = None) -> str:
payload = metadata.model_dump()
if import_request is not None:
payload = {"metadata": payload, "import_request": import_request}
return json.dumps(payload, ensure_ascii=False)

View File

@@ -0,0 +1,202 @@
from __future__ import annotations
import asyncio
import base64
import time
import uuid
from dataclasses import dataclass, field
from typing import Any
from urllib.parse import unquote, urlparse
from app.config import get_settings
from app.services.page_analyzer import (
MediaCandidate,
_browser_context,
_is_ad_or_tracker_url,
_looks_like_media_response,
_probe_candidates_with_meta,
extract_media_urls,
)
from app.services.url_safety import UnsafeUrlError, assert_public_host, check_url_for_page
_INTERACTIVE_CAPTURE_LIMIT = 80
_DEFAULT_VIEWPORT = {"width": 1280, "height": 720}
_SESSION_TTL_SECONDS = 15 * 60
@dataclass
class InteractiveBrowserSession:
id: str
page_url: str
context: Any
page: Any
browser: Any | None = None
user_data_dir: Any | None = None
ublock_path: str | None = None
seen: dict[str, tuple[str | None, str | None]] = field(default_factory=dict)
pending_response_tasks: set[asyncio.Task] = field(default_factory=set)
created_at: float = field(default_factory=time.monotonic)
updated_at: float = field(default_factory=time.monotonic)
class InteractiveBrowserManager:
"""Small in-process Playwright session manager for interactive media analysis.
The browser never receives private/internal network requests: every request is
routed through the same public-host guard used by the non-interactive browser
analyzer. The UI gets screenshots and sends clicks/keystrokes back, avoiding
iframe/CORS limitations while still allowing the user to press Play.
"""
def __init__(self) -> None:
self._sessions: dict[str, InteractiveBrowserSession] = {}
self._playwright: Any | None = None
self._lock = asyncio.Lock()
async def _ensure_playwright(self):
if self._playwright is None:
from playwright.async_api import async_playwright
self._playwright = await async_playwright().start()
return self._playwright
async def _close_session(self, session: InteractiveBrowserSession) -> None:
for task in list(session.pending_response_tasks):
if not task.done():
task.cancel()
try:
await session.context.close()
finally:
if session.browser is not None:
await session.browser.close()
if session.user_data_dir is not None:
session.user_data_dir.cleanup()
async def cleanup_expired(self) -> None:
now = time.monotonic()
expired = [sid for sid, sess in self._sessions.items() if now - sess.updated_at > _SESSION_TTL_SECONDS]
for sid in expired:
await self.stop(sid)
async def start(self, url: str) -> InteractiveBrowserSession:
await self.cleanup_expired()
settings = get_settings()
checked_page = await check_url_for_page(url, timeout=settings.request_timeout)
playwright = await self._ensure_playwright()
context, browser, user_data_dir, ublock_path = await _browser_context(playwright, ignore_https_errors=False)
await context.set_extra_http_headers({"DNT": "1"})
session = InteractiveBrowserSession(
id=uuid.uuid4().hex,
page_url=checked_page.final_url,
context=context,
page=await context.new_page(),
browser=browser,
user_data_dir=user_data_dir,
ublock_path=ublock_path,
)
await session.page.set_viewport_size(_DEFAULT_VIEWPORT)
async def guard_route(route):
request_url = route.request.url
parsed = urlparse(request_url)
if parsed.scheme not in {"http", "https"}:
await route.abort()
return
if _is_ad_or_tracker_url(request_url):
await route.abort()
return
try:
await assert_public_host(request_url)
except UnsafeUrlError:
await route.abort()
return
await route.continue_()
async def on_response(response):
response_url = response.url
try:
headers = await response.all_headers()
except Exception:
headers = {}
content_type = headers.get("content-type")
if _looks_like_media_response(response_url, content_type):
title = unquote(urlparse(response_url).path.rsplit("/", 1)[-1]) or response_url
session.seen.setdefault(response_url, (title, content_type))
def schedule_response_probe(response):
task = asyncio.create_task(on_response(response))
session.pending_response_tasks.add(task)
task.add_done_callback(session.pending_response_tasks.discard)
await context.route("**/*", guard_route)
context.on("response", schedule_response_probe)
try:
await session.page.goto(checked_page.final_url, wait_until="domcontentloaded", timeout=int(settings.request_timeout * 1000))
try:
await session.page.wait_for_load_state("networkidle", timeout=3000)
except Exception:
pass
await self._collect_dom_media(session)
except Exception:
await self._close_session(session)
raise
self._sessions[session.id] = session
return session
async def _collect_dom_media(self, session: InteractiveBrowserSession) -> None:
try:
html = await session.page.content()
page_url = session.page.url
except Exception:
return
for dom_url in extract_media_urls(html, page_url):
session.seen.setdefault(dom_url, (None, None))
async def _settle(self, session: InteractiveBrowserSession) -> None:
session.updated_at = time.monotonic()
await asyncio.sleep(0.5)
if session.pending_response_tasks:
await asyncio.gather(*list(session.pending_response_tasks), return_exceptions=True)
await self._collect_dom_media(session)
def get(self, session_id: str) -> InteractiveBrowserSession:
session = self._sessions.get(session_id)
if not session:
raise KeyError(session_id)
session.updated_at = time.monotonic()
return session
async def click(self, session_id: str, x: float, y: float) -> InteractiveBrowserSession:
session = self.get(session_id)
await session.page.mouse.click(x, y)
await self._settle(session)
return session
async def press(self, session_id: str, key: str) -> InteractiveBrowserSession:
session = self.get(session_id)
await session.page.keyboard.press(key)
await self._settle(session)
return session
async def stop(self, session_id: str) -> None:
session = self._sessions.pop(session_id, None)
if session is not None:
await self._close_session(session)
async def screenshot_data_url(self, session: InteractiveBrowserSession) -> str:
png = await session.page.screenshot(type="png", full_page=False)
return "data:image/png;base64," + base64.b64encode(png).decode("ascii")
async def candidates(self, session: InteractiveBrowserSession) -> list[MediaCandidate]:
if session.pending_response_tasks:
await asyncio.gather(*list(session.pending_response_tasks), return_exceptions=True)
await self._collect_dom_media(session)
return await _probe_candidates_with_meta(
dict(list(session.seen.items())[:_INTERACTIVE_CAPTURE_LIMIT]),
source="interactive-browser",
)
interactive_browser_manager = InteractiveBrowserManager()

View File

@@ -0,0 +1,23 @@
import httpx
from app.config import Settings
class JellyfinRefreshSkipped(RuntimeError):
pass
async def refresh_jellyfin(settings: Settings) -> bool:
"""Trigger Jellyfin's library refresh when credentials are configured.
Returns False when no API key is present so local/dev imports can still finish
without writing secrets into the project tree.
"""
if not settings.jellyfin_api_key:
return False
url = settings.jellyfin_url.rstrip("/") + "/Library/Refresh"
headers = {"X-Emby-Token": settings.jellyfin_api_key}
async with httpx.AsyncClient(timeout=20) as client:
response = await client.post(url, headers=headers)
response.raise_for_status()
return True

View File

@@ -0,0 +1,489 @@
from __future__ import annotations
import asyncio
import html as html_lib
import os
import re
import tempfile
from dataclasses import dataclass, field
from html.parser import HTMLParser
from pathlib import Path
from typing import Any
from urllib.parse import unquote, urljoin, urlparse
import httpx
try:
from yt_dlp import YoutubeDL
except ImportError: # optional analyzer backend
YoutubeDL = None # type: ignore[assignment]
from app.config import get_settings
from app.services.url_safety import UnsafeUrlError, assert_public_host, check_url_for_download, check_url_for_page
_MEDIA_EXTENSIONS = (".mp4", ".m4v", ".webm", ".mkv", ".mov", ".m3u8", ".mpd")
_MEDIA_CONTENT_TYPES = {
"video/mp4": "direct_video",
"video/webm": "direct_video",
"video/x-matroska": "direct_video",
"video/mp2t": "hls_segment",
"application/vnd.apple.mpegurl": "hls_manifest",
"application/x-mpegurl": "hls_manifest",
"application/dash+xml": "dash_manifest",
}
_QUOTED_URL_RE = re.compile(r"[\"']([^\"']+\.(?:mp4|m4v|webm|mkv|mov|m3u8|mpd)(?:\?[^\"']*)?)[\"']", re.I)
_ABSOLUTE_MEDIA_URL_RE = re.compile(
r"(?:https?://|https?:\\/\\/|//|\\/\\/)[^\s\"'<>]+?\.(?:mp4|m4v|webm|mkv|mov|m3u8|mpd)(?=$|[?&#\s\"'<>])(?:[?&][^\s\"'<>]*)?",
re.I,
)
_BROWSER_CAPTURE_LIMIT = 80
_YTDLP_ALLOWED_HOST_SUFFIXES = (
"youtube.com",
"youtu.be",
"archive.org",
"ardmediathek.de",
"zdf.de",
"arte.tv",
)
_AD_HOST_FRAGMENTS = (
"doubleclick.net",
"googlesyndication.com",
"googleadservices.com",
"adservice.google.",
"adsystem.com",
"adnxs.com",
"adsafeprotected.com",
"amazon-adsystem.com",
"taboola.com",
"outbrain.com",
"scorecardresearch.com",
)
@dataclass
class MediaCandidate:
url: str
kind: str
title: str | None = None
source: str = "html"
content_type: str | None = None
content_length: int | None = None
file_size: str | None = None
quality: str | None = None
allowed: bool = True
reason: str | None = None
metadata: dict[str, Any] = field(default_factory=dict)
class _MediaHTMLParser(HTMLParser):
def __init__(self, base_url: str) -> None:
super().__init__()
self.base_url = base_url
self.urls: list[str] = []
def handle_starttag(self, tag: str, attrs: list[tuple[str, str | None]]) -> None:
attr_map = {key.lower(): value for key, value in attrs if value}
for attr in ("src", "href", "data-src", "data-url"):
value = attr_map.get(attr)
if value and _looks_like_media_url(value):
self.urls.append(urljoin(self.base_url, value))
def _looks_like_media_url(value: str) -> bool:
parsed_path = urlparse(value).path.lower()
return parsed_path.endswith(_MEDIA_EXTENSIONS)
def _looks_like_media_response(url: str, content_type: str | None = None) -> bool:
clean = (content_type or "").split(";", 1)[0].strip().lower()
return clean in _MEDIA_CONTENT_TYPES or clean.startswith("video/") or _looks_like_media_url(url)
def human_size(size: int | None) -> str | None:
if size is None:
return None
value = float(size)
for unit in ("B", "KB", "MB", "GB"):
if value < 1024 or unit == "GB":
if unit == "B":
return f"{int(value)} {unit}"
return f"{value:.1f} {unit}"
value /= 1024
def _is_ad_or_tracker_url(url: str) -> bool:
host = (urlparse(url).hostname or "").lower()
if any(fragment in host for fragment in _AD_HOST_FRAGMENTS):
return True
path = urlparse(url).path.lower()
return any(marker in path for marker in ("/ads/", "/adserver/", "/prebid", "/vast", "/vpaid"))
def _host_allowed_for_ytdlp(url: str) -> bool:
host = (urlparse(url).hostname or "").lower().rstrip(".")
return any(host == suffix or host.endswith(f".{suffix}") for suffix in _YTDLP_ALLOWED_HOST_SUFFIXES)
async def _fetch_public_page_body(url: str, *, timeout: float, max_redirects: int = 8) -> tuple[str, str]:
"""Fetch a page body while validating every GET redirect hop.
``check_url_for_page`` validates HEAD redirects, but some origins redirect
differently for GET. Keep GET redirects manual so the analyzer cannot be
turned into an internal-network fetch proxy by a method-dependent redirect.
"""
current = url
async with httpx.AsyncClient(follow_redirects=False, timeout=timeout) as client:
for _ in range(max_redirects + 1):
checked = await check_url_for_page(current, timeout=timeout, max_redirects=max_redirects)
response = await client.get(checked.final_url)
if response.is_redirect:
loc = response.headers.get("location")
if not loc:
raise UnsafeUrlError("Redirect ohne Location-Header")
current = str(response.url.join(loc))
continue
response.raise_for_status()
await assert_public_host(str(response.url))
return response.text, str(response.url)
raise UnsafeUrlError("Zu viele Weiterleitungen")
def _ublock_origin_path() -> Path | None:
configured = os.environ.get("KINOPROJEKT_UBLOCK_PATH")
candidates = [Path(configured)] if configured else []
candidates.extend([Path("/opt/kino-projekt/ublock-origin"), Path("/opt/kino-projekt/ublock")])
for candidate in candidates:
if not candidate:
continue
if candidate.exists() and candidate.is_dir() and (candidate / "manifest.json").exists():
return candidate
nested = candidate / "uBlock0.chromium"
if nested.exists() and nested.is_dir() and (nested / "manifest.json").exists():
return nested
return None
def _has_importable_media(candidates: dict[str, tuple[str | None, str | None]]) -> bool:
return any(
classify_candidate(url, content_type).kind in {"direct_video", "hls_manifest", "dash_manifest"}
for url, (_, content_type) in candidates.items()
)
def _normalize_media_url(raw: str, base_url: str) -> str | None:
value = html_lib.unescape(raw.strip())
if not value:
return None
value = value.replace("\\/", "/").replace("\\u002F", "/").replace("\\u002f", "/")
value = value.rstrip(",;)}]")
if value.startswith("//"):
value = f"{urlparse(base_url).scheme or 'https'}:{value}"
return urljoin(base_url, unquote(value))
def extract_media_urls(html: str, base_url: str) -> list[str]:
parser = _MediaHTMLParser(base_url)
parser.feed(html)
urls = list(parser.urls)
for match in _QUOTED_URL_RE.finditer(html):
normalized = _normalize_media_url(match.group(1), base_url)
if normalized:
urls.append(normalized)
for match in _ABSOLUTE_MEDIA_URL_RE.finditer(html):
normalized = _normalize_media_url(match.group(0), base_url)
if normalized:
urls.append(normalized)
# Stable de-duplication while preserving discovery order.
return list(dict.fromkeys(urls))
def classify_candidate(url: str, content_type: str | None = None) -> MediaCandidate:
normalized_content_type = (content_type or "").split(";", 1)[0].strip().lower()
if normalized_content_type in _MEDIA_CONTENT_TYPES:
kind = _MEDIA_CONTENT_TYPES[normalized_content_type]
elif normalized_content_type.startswith("video/"):
kind = "direct_video"
else:
path = urlparse(url).path.lower()
if path.endswith(".m3u8"):
kind = "hls_manifest"
elif path.endswith(".mpd"):
kind = "dash_manifest"
elif path.endswith((".mp4", ".m4v", ".webm", ".mkv", ".mov")):
kind = "direct_video"
else:
kind = "unknown_media"
title = unquote(urlparse(url).path.rsplit("/", 1)[-1]) or url
return MediaCandidate(url=url, kind=kind, title=title, content_type=content_type)
async def _probe_candidate(url: str, source: str, title: str | None = None, quality: str | None = None) -> MediaCandidate:
settings = get_settings()
try:
checked = await check_url_for_download(url, max_bytes=settings.max_download_bytes, timeout=settings.request_timeout)
candidate = classify_candidate(checked.final_url, checked.content_type)
candidate.source = source
candidate.title = title or candidate.title
candidate.quality = quality
candidate.content_length = checked.content_length
candidate.file_size = human_size(checked.content_length)
candidate.metadata = {"redirect_chain": checked.chain}
return candidate
except UnsafeUrlError as exc:
c = classify_candidate(url)
c.source = source
c.title = title or c.title
c.quality = quality
c.allowed = False
c.reason = str(exc)
return c
async def analyze_html_page(url: str) -> list[MediaCandidate]:
settings = get_settings()
# First validate the page URL itself so the analyzer cannot be used as an internal-network proxy.
checked_page = await check_url_for_page(url, timeout=settings.request_timeout)
body, final_url = await _fetch_public_page_body(checked_page.final_url, timeout=settings.request_timeout)
found = extract_media_urls(body, final_url)
return await _probe_candidates(found, source="html")
async def analyze_with_ytdlp(url: str) -> list[MediaCandidate]:
checked_page = await check_url_for_page(url, timeout=get_settings().request_timeout)
if not _host_allowed_for_ytdlp(checked_page.final_url):
return []
if YoutubeDL is None:
return []
def _extract() -> dict[str, Any] | None:
opts = {
"quiet": True,
"skip_download": True,
"noplaylist": True,
"extract_flat": False,
"socket_timeout": get_settings().request_timeout,
}
with YoutubeDL(opts) as ydl:
return ydl.extract_info(checked_page.final_url, download=False)
try:
info = await asyncio.to_thread(_extract)
except Exception:
return []
if not info:
return []
candidates: list[tuple[str, str | None, str | None]] = []
title = info.get("title")
if info.get("url") and _looks_like_media_url(info["url"]):
candidates.append((info["url"], title, None))
for fmt in info.get("formats") or []:
fmt_url = fmt.get("url")
if not fmt_url:
continue
ext = fmt.get("ext") or ""
protocol = fmt.get("protocol") or ""
if _looks_like_media_url(fmt_url) or ext in {"mp4", "webm", "m3u8", "mpd"} or "m3u8" in protocol or "dash" in protocol:
quality = fmt.get("format_note") or fmt.get("format") or fmt.get("height")
candidates.append((fmt_url, title, str(quality) if quality else None))
deduped: dict[str, tuple[str | None, str | None]] = {}
for candidate_url, candidate_title, quality in candidates:
deduped.setdefault(candidate_url, (candidate_title, quality))
return await _probe_candidates_with_meta(deduped, source="yt-dlp")
async def _trigger_video_playback(page) -> None:
"""Try to start embedded players so lazy video requests become visible."""
await page.evaluate(
"""
async () => {
for (const text of ['Akzeptieren', 'Accept', 'I agree', 'OK']) {
const buttons = [...document.querySelectorAll('button, [role="button"], input[type="button"], input[type="submit"]')];
const button = buttons.find(el => (el.innerText || el.value || '').toLowerCase().includes(text.toLowerCase()));
if (button) { try { button.click(); } catch (_) {} }
}
for (const video of document.querySelectorAll('video')) {
try {
video.muted = true;
video.playsInline = true;
video.autoplay = true;
video.play().catch(()=>{});
} catch (_) {}
}
const playSelectors = [
'[aria-label*="play" i]', '[title*="play" i]', '.play', '.play-button', '.vjs-big-play-button',
'button[class*="play" i]', 'button[aria-label*="abspielen" i]'
];
for (const selector of playSelectors) {
for (const el of document.querySelectorAll(selector)) {
try { el.click(); } catch (_) {}
}
}
}
"""
)
async def _browser_context(playwright, *, ignore_https_errors: bool = False):
base_args = [
"--disable-dev-shm-usage",
"--disable-gpu",
"--no-sandbox",
"--disable-setuid-sandbox",
"--disable-background-networking",
]
ublock_path = _ublock_origin_path()
if ublock_path:
user_data_dir = tempfile.TemporaryDirectory(prefix="kino-browser-")
context = await playwright.chromium.launch_persistent_context(
user_data_dir.name,
headless=True,
ignore_https_errors=ignore_https_errors,
java_script_enabled=True,
args=[
*base_args,
f"--disable-extensions-except={ublock_path}",
f"--load-extension={ublock_path}",
],
)
return context, None, user_data_dir, str(ublock_path)
browser = await playwright.chromium.launch(headless=True, args=base_args)
context = await browser.new_context(ignore_https_errors=ignore_https_errors, java_script_enabled=True)
return context, browser, None, None
async def analyze_with_browser(url: str) -> list[MediaCandidate]:
"""Headless-Browser-Analyzer (Option B): observe JS-loaded media requests.
Every browser request is still constrained to public http(s) targets. This keeps the
server-side browser from becoming a LAN/internal-network fetch proxy.
"""
settings = get_settings()
checked_page = await check_url_for_page(url, timeout=settings.request_timeout)
try:
from playwright.async_api import TimeoutError as PlaywrightTimeoutError
from playwright.async_api import async_playwright
except ImportError:
return []
seen: dict[str, tuple[str | None, str | None]] = {}
playback_triggered = False
async with async_playwright() as p:
context, browser, user_data_dir, ublock_path = await _browser_context(p, ignore_https_errors=False)
async def guard_route(route):
request_url = route.request.url
parsed = urlparse(request_url)
if parsed.scheme not in {"http", "https"}:
await route.abort()
return
if _is_ad_or_tracker_url(request_url):
await route.abort()
return
try:
await assert_public_host(request_url)
except UnsafeUrlError:
await route.abort()
return
await route.continue_()
async def on_response(response):
response_url = response.url
headers = await response.all_headers()
content_type = headers.get("content-type")
if _looks_like_media_response(response_url, content_type):
title = unquote(urlparse(response_url).path.rsplit("/", 1)[-1]) or response_url
seen.setdefault(response_url, (title, content_type))
await context.route("**/*", guard_route)
pending_response_tasks: set[asyncio.Task] = set()
def schedule_response_probe(response):
task = asyncio.create_task(on_response(response))
pending_response_tasks.add(task)
task.add_done_callback(pending_response_tasks.discard)
context.on("response", schedule_response_probe)
page = await context.new_page()
try:
await page.goto(checked_page.final_url, wait_until="domcontentloaded", timeout=int(settings.request_timeout * 1000))
try:
await page.wait_for_load_state("networkidle", timeout=5000)
except PlaywrightTimeoutError:
pass
if pending_response_tasks:
await asyncio.gather(*pending_response_tasks, return_exceptions=True)
if not _has_importable_media(seen):
playback_triggered = True
await _trigger_video_playback(page)
try:
await page.wait_for_load_state("networkidle", timeout=5000)
except PlaywrightTimeoutError:
pass
if pending_response_tasks:
await asyncio.gather(*pending_response_tasks, return_exceptions=True)
html = await page.content()
for dom_url in extract_media_urls(html, page.url):
seen.setdefault(dom_url, (None, None))
finally:
await context.close()
if browser is not None:
await browser.close()
if user_data_dir is not None:
user_data_dir.cleanup()
candidates = await _probe_candidates_with_meta(dict(list(seen.items())[:_BROWSER_CAPTURE_LIMIT]), source="browser")
for candidate in candidates:
candidate.metadata["playback_triggered"] = playback_triggered
candidate.metadata["ublock_origin"] = bool(ublock_path)
return candidates
async def analyze_browser_capture(captured: list[dict[str, Any]], page_url: str | None = None) -> list[MediaCandidate]:
"""Validate and normalize media URLs captured by a user's real browser (Option C)."""
base_url = page_url or ""
deduped: dict[str, tuple[str | None, str | None]] = {}
for item in captured[:_BROWSER_CAPTURE_LIMIT]:
raw_url = str(item.get("url") or "").strip()
if not raw_url:
continue
candidate_url = urljoin(base_url, raw_url) if base_url else raw_url
if not _looks_like_media_response(candidate_url, item.get("content_type") or item.get("type")):
continue
title = item.get("title") or item.get("name")
quality = item.get("quality")
deduped.setdefault(candidate_url, (str(title) if title else None, str(quality) if quality else None))
return await _probe_candidates_with_meta(deduped, source="browser-capture")
async def _probe_candidates(urls: list[str], source: str) -> list[MediaCandidate]:
return await _probe_candidates_with_meta({url: (None, None) for url in urls}, source=source)
async def _probe_candidates_with_meta(urls: dict[str, tuple[str | None, str | None]], source: str) -> list[MediaCandidate]:
candidates = [await _probe_candidate(url, source, title, quality) for url, (title, quality) in list(urls.items())[:50]]
if any(candidate.kind in {"hls_manifest", "dash_manifest"} for candidate in candidates):
candidates = [candidate for candidate in candidates if candidate.kind != "hls_segment"]
# Show usable candidates first, then blocked candidates as diagnostics.
candidates.sort(key=lambda c: (not c.allowed, c.kind, c.quality or ""))
return candidates
async def analyze_page(url: str) -> list[MediaCandidate]:
results = await asyncio.gather(
analyze_html_page(url),
analyze_with_ytdlp(url),
analyze_with_browser(url),
return_exceptions=True,
)
html_candidates, ytdlp_candidates, browser_candidates = [item if isinstance(item, list) else [] for item in results]
merged: dict[str, MediaCandidate] = {}
# Prefer browser/ytdlp metadata over static HTML when the same URL appears twice.
for candidate in [*browser_candidates, *ytdlp_candidates, *html_candidates]:
existing = merged.get(candidate.url)
if not existing or (not existing.allowed and candidate.allowed):
merged[candidate.url] = candidate
return list(merged.values())

View File

@@ -0,0 +1,67 @@
import re
from pathlib import Path
_SAFE_CHARS = re.compile(r"[^A-Za-z0-9._ -]+")
def slugify_name(value: str, fallback: str = "media") -> str:
cleaned = _SAFE_CHARS.sub("_", value).strip(" ._- ")
return cleaned[:120] or fallback
def ensure_under(base: Path, candidate: Path) -> Path:
base_resolved = base.expanduser().resolve()
path = (base_resolved / candidate).resolve() if not candidate.is_absolute() else candidate.resolve()
if base_resolved != path and base_resolved not in path.parents:
raise ValueError(f"Path escapes base directory: {candidate}")
return path
def tmp_target(tmp_dir: Path, job_id: int, title: str) -> Path:
return ensure_under(tmp_dir, Path(str(job_id)) / slugify_name(title))
def media_target(media_root: Path, library: str, title: str) -> Path:
return ensure_under(media_root, Path(slugify_name(library, "library")) / slugify_name(title))
def jellyfin_profile_target(
media_root: Path,
target_profile: str,
title: str,
*,
year: int | None = None,
series_title: str | None = None,
season: int | None = None,
episode: int | None = None,
episode_title: str | None = None,
channel: str | None = None,
external_id: str | None = None,
) -> Path:
"""Return the Jellyfin-compatible destination directory for a target profile.
The returned path is always below ``media_root``. The downloader copies the
provider output into this directory; provider-selected filenames are kept for
now, while the directory structure follows the real Jellyfin layout from the
project plan.
"""
profile = target_profile.strip().lower()
safe_title = slugify_name(title)
if profile == "film":
movie_dir = f"{safe_title} ({year})" if year else safe_title
return ensure_under(media_root, Path("Filme") / movie_dir)
if profile == "serie":
show = slugify_name(series_title or title, "Serie")
season_number = season if season and season > 0 else 1
episode_prefix = f"S{season_number:02d}E{episode:02d} - " if episode and episode > 0 else ""
episode_dir = slugify_name(f"{episode_prefix}{episode_title or title}", "Episode")
return ensure_under(media_root, Path("Serien") / show / f"Staffel{season_number}" / episode_dir)
if profile == "clip":
owner = slugify_name(channel or "Unbekannte Quelle", "Quelle")
suffix = f" [{slugify_name(external_id)}]" if external_id else ""
return ensure_under(media_root, Path("YouTube-Mediathek") / owner / f"{safe_title}{suffix}")
raise ValueError(f"Unknown target profile: {target_profile}")

View File

@@ -0,0 +1,170 @@
from __future__ import annotations
import asyncio
import ipaddress
import socket
from dataclasses import dataclass
from typing import Iterable
from urllib.parse import urlparse
import httpx
BLOCKED_NETWORKS = [
ipaddress.ip_network("0.0.0.0/8"),
ipaddress.ip_network("10.0.0.0/8"),
ipaddress.ip_network("100.64.0.0/10"),
ipaddress.ip_network("127.0.0.0/8"),
ipaddress.ip_network("169.254.0.0/16"),
ipaddress.ip_network("172.16.0.0/12"),
ipaddress.ip_network("192.168.0.0/16"),
ipaddress.ip_network("224.0.0.0/4"),
ipaddress.ip_network("240.0.0.0/4"),
ipaddress.ip_network("::1/128"),
ipaddress.ip_network("::/128"),
ipaddress.ip_network("fc00::/7"),
ipaddress.ip_network("fe80::/10"),
ipaddress.ip_network("ff00::/8"),
]
ALLOWED_CONTENT_PREFIXES = ("video/", "audio/")
ALLOWED_CONTENT_TYPES = {
"application/vnd.apple.mpegurl",
"application/x-mpegurl",
"application/dash+xml",
"application/octet-stream", # accepted only with known extension
}
ALLOWED_EXTENSIONS = (
".mp4",
".m4v",
".mkv",
".webm",
".mov",
".mp3",
".m4a",
".aac",
".ogg",
".flac",
".m3u8",
".mpd",
)
class UnsafeUrlError(ValueError):
pass
@dataclass
class CheckedUrl:
final_url: str
chain: list[str]
content_type: str | None
content_length: int | None
def _is_blocked_ip(ip: str) -> bool:
addr = ipaddress.ip_address(ip)
return any(addr in net for net in BLOCKED_NETWORKS) or addr.is_private or addr.is_loopback or addr.is_link_local
def validate_scheme(url: str) -> None:
parsed = urlparse(url)
if parsed.scheme not in {"http", "https"}:
raise UnsafeUrlError("Nur http/https URLs sind erlaubt")
if not parsed.hostname:
raise UnsafeUrlError("URL enthält keinen Hostnamen")
if parsed.username or parsed.password:
raise UnsafeUrlError("Zugangsdaten in URLs sind nicht erlaubt")
async def resolve_host(hostname: str, port: int | None = None) -> list[str]:
def _resolve() -> list[str]:
infos = socket.getaddrinfo(hostname, port or 443, type=socket.SOCK_STREAM)
return sorted({str(info[4][0]) for info in infos})
return await asyncio.to_thread(_resolve)
async def assert_public_host(url: str) -> None:
validate_scheme(url)
parsed = urlparse(url)
assert parsed.hostname is not None
try:
ips = await resolve_host(parsed.hostname, parsed.port)
except socket.gaierror as exc:
raise UnsafeUrlError(f"Hostname kann nicht aufgelöst werden: {parsed.hostname}") from exc
if not ips:
raise UnsafeUrlError("Hostname hat keine IP-Adressen")
blocked = [ip for ip in ips if _is_blocked_ip(ip)]
if blocked:
raise UnsafeUrlError("URL zeigt auf interne/private IP-Adresse")
def _content_type_allowed(content_type: str | None, url: str) -> bool:
if not content_type:
return any(urlparse(url).path.lower().endswith(ext) for ext in ALLOWED_EXTENSIONS)
clean = content_type.split(";", 1)[0].strip().lower()
if clean.startswith(ALLOWED_CONTENT_PREFIXES):
return True
if clean in ALLOWED_CONTENT_TYPES and any(urlparse(url).path.lower().endswith(ext) for ext in ALLOWED_EXTENSIONS):
return True
return False
async def check_url_for_page(url: str, *, timeout: float = 15.0, max_redirects: int = 8) -> CheckedUrl:
"""Validate URL and redirects for page analysis without requiring media Content-Type."""
current = str(url)
chain: list[str] = []
async with httpx.AsyncClient(follow_redirects=False, timeout=timeout) as client:
for _ in range(max_redirects + 1):
await assert_public_host(current)
chain.append(current)
response = await client.head(current)
if response.status_code in {405, 403}:
response = await client.get(current, headers={"Range": "bytes=0-0"})
if response.is_redirect:
loc = response.headers.get("location")
if not loc:
raise UnsafeUrlError("Redirect ohne Location-Header")
current = str(response.url.join(loc))
continue
response.raise_for_status()
await assert_public_host(str(response.url))
content_type = response.headers.get("content-type")
length_header = response.headers.get("content-length")
content_length = int(length_header) if length_header and length_header.isdigit() else None
return CheckedUrl(str(response.url), chain, content_type, content_length)
raise UnsafeUrlError("Zu viele Weiterleitungen")
async def check_url_for_download(url: str, *, max_bytes: int, timeout: float = 15.0, max_redirects: int = 8) -> CheckedUrl:
"""Validate URL, every redirect hop, final target, content type, and size.
This is the no-domain-allowlist SSRF guard for Mediathek/direct links.
"""
current = str(url)
chain: list[str] = []
async with httpx.AsyncClient(follow_redirects=False, timeout=timeout) as client:
for _ in range(max_redirects + 1):
await assert_public_host(current)
chain.append(current)
response = await client.head(current)
if response.status_code in {405, 403}:
response = await client.get(current, headers={"Range": "bytes=0-0"})
if response.is_redirect:
loc = response.headers.get("location")
if not loc:
raise UnsafeUrlError("Redirect ohne Location-Header")
current = str(response.url.join(loc))
continue
response.raise_for_status()
await assert_public_host(str(response.url))
content_type = response.headers.get("content-type")
length_header = response.headers.get("content-length")
content_length = int(length_header) if length_header and length_header.isdigit() else None
if content_length is not None and content_length > max_bytes:
raise UnsafeUrlError("Datei ist größer als das konfigurierte Limit")
if not _content_type_allowed(content_type, str(response.url)):
raise UnsafeUrlError("Ziel liefert keinen plausiblen Medien-Content-Type")
return CheckedUrl(str(response.url), chain, content_type, content_length)
raise UnsafeUrlError("Zu viele Weiterleitungen")

336
backend/app/web/app.js Normal file
View File

@@ -0,0 +1,336 @@
const loginSection = document.querySelector('#login-section');
const loginForm = document.querySelector('#login-form');
const loginMessage = document.querySelector('#login-message');
const logoutButton = document.querySelector('#logout');
const form = document.querySelector('#probe-form');
const result = document.querySelector('#result');
const candidatesSection = document.querySelector('#candidates');
const analyzeButton = document.querySelector('#analyze-page');
const interactiveButton = document.querySelector('#interactive-analyze');
const bookmarklet = document.querySelector('#bookmarklet');
const copyBookmarklet = document.querySelector('#copy-bookmarklet');
const bookmarkletResult = document.querySelector('#bookmarklet-result');
function escapeHtml(value) {
return String(value ?? '').replace(/[&<>'"]/g, (char) => ({'&': '&amp;', '<': '&lt;', '>': '&gt;', "'": '&#39;', '"': '&quot;'}[char]));
}
function setLoginVisible(visible) {
loginSection.classList.toggle('hidden', !visible);
form.classList.toggle('hidden', visible);
logoutButton.classList.toggle('hidden', visible);
}
function renderJob(job) {
const percent = Math.round((Number(job.progress) || 0) * 100);
if (job.status === 'done') {
return `Job #${job.id}: fertig (${percent}%)\nDatei übertragen.`;
}
const labels = {
queued: 'wartet',
downloading: 'lädt herunter',
postprocessing: 'verarbeitet',
refreshing: 'überträgt / aktualisiert Jellyfin',
failed: 'fehlgeschlagen',
};
const lines = [`Job #${job.id}: ${labels[job.status] || job.status} (${percent}%)`];
if (job.status === 'failed' && job.error) lines.push(`Fehler: ${job.error}`);
return lines.join('\n');
}
function encodeCapturePayload(payload) {
return btoa(unescape(encodeURIComponent(JSON.stringify(payload))));
}
function decodeCapturePayload(value) {
return JSON.parse(decodeURIComponent(escape(atob(value))));
}
function buildBookmarklet() {
const target = `${location.origin}/`;
const code = `(()=>{const target='${target}';const mediaExt=/\\.(mp4|m4v|webm|mkv|mov|m3u8|mpd)([?#]|$)/i;const push=(arr,url,title,type)=>{try{if(!url)return;const abs=new URL(url,location.href).href;if(mediaExt.test(abs)||String(type||'').startsWith('video/')||/mpegurl|dash\\+xml/i.test(String(type||'')))arr.push({url:abs,title:title||'',content_type:type||''});}catch(e){}};const c=[];document.querySelectorAll('video,source,a').forEach(el=>push(c,el.currentSrc||el.src||el.href,el.title||el.textContent||document.title,el.type||''));performance.getEntriesByType('resource').forEach(e=>push(c,e.name,document.title,e.initiatorType==='video'?'video/unknown':''));const seen=new Set();const candidates=c.filter(x=>!seen.has(x.url)&&seen.add(x.url)).slice(0,80);if(!candidates.length&&!confirm('Keine Medien-URL im aktuellen Tab gefunden. Trotzdem zur Kino-App wechseln? Tipp: Video erst starten und ein paar Sekunden laufen lassen.'))return;const payload={page_url:location.href,candidates};const encoded=btoa(unescape(encodeURIComponent(JSON.stringify(payload))));const dest=target+'?capture='+encodeURIComponent(encoded);const opened=window.open(dest,'_blank');if(!opened)location.href=dest;})();`;
return `javascript:${code}`;
}
function initBookmarklet() {
const href = buildBookmarklet();
bookmarklet.href = href;
copyBookmarklet.addEventListener('click', async () => {
await navigator.clipboard.writeText(href);
bookmarkletResult.textContent = 'Bookmarklet kopiert. Als neues Lesezeichen speichern und auf der Videoseite ausführen.';
});
}
async function pollJob(jobId) {
const output = document.querySelector('#job-output');
for (;;) {
const response = await fetch(`/api/imports/${jobId}`);
const job = await response.json();
if (response.status === 401) {
await refreshAuthStatus();
throw new Error('Bitte zuerst einloggen.');
}
if (!response.ok) throw new Error(job.detail || 'Job-Status konnte nicht gelesen werden');
output.textContent = renderJob(job);
if (['done', 'failed'].includes(job.status)) return job;
await new Promise((resolve) => setTimeout(resolve, 2000));
}
}
async function refreshAuthStatus() {
const response = await fetch('/api/auth/status');
const status = await response.json();
setLoginVisible(status.enabled && !status.authenticated);
}
loginForm.addEventListener('submit', async (event) => {
event.preventDefault();
loginMessage.textContent = 'Melde an…';
const username = document.querySelector('#username').value;
const password = document.querySelector('#password').value;
const response = await fetch('/api/auth/login', {
method: 'POST',
headers: {'content-type': 'application/json'},
body: JSON.stringify({username, password}),
});
if (!response.ok) {
loginMessage.textContent = 'Login fehlgeschlagen.';
return;
}
loginMessage.textContent = '';
await refreshAuthStatus();
});
logoutButton.addEventListener('click', async () => {
await fetch('/api/auth/logout', {method: 'POST'});
await refreshAuthStatus();
});
function currentImportPayload(url) {
const payload = {
url,
target_profile: document.querySelector('input[name="target-profile"]:checked').value,
};
const textFields = [
['title', 'title'],
['series-title', 'series_title'],
['episode-title', 'episode_title'],
];
for (const [elementId, key] of textFields) {
const value = document.querySelector(`#${elementId}`).value.trim();
if (value) payload[key] = value;
}
const numberFields = [
['year', 'year'],
['season', 'season'],
['episode', 'episode'],
];
for (const [elementId, key] of numberFields) {
const value = document.querySelector(`#${elementId}`).value;
if (value) payload[key] = Number(value);
}
return payload;
}
form.addEventListener('submit', async (event) => {
event.preventDefault();
const url = document.querySelector('#url').value;
result.className = 'card';
result.textContent = 'Prüfe Quelle…';
try {
const response = await fetch('/api/probe', {method: 'POST', headers: {'content-type': 'application/json'}, body: JSON.stringify({url})});
const data = await response.json();
if (response.status === 401) {
await refreshAuthStatus();
throw new Error('Bitte zuerst einloggen.');
}
if (!response.ok) throw new Error(data.detail || 'Probe fehlgeschlagen');
result.innerHTML = `
<h2>${escapeHtml(data.title || 'Unbenannte Quelle')}</h2>
<p>Provider: ${escapeHtml(data.provider)}</p>
<p>Dauer: ${escapeHtml(data.duration_seconds ?? 'unbekannt')}</p>
<button id="start-import" type="button">Import starten</button>
<pre id="job-output"></pre>`;
document.querySelector('#start-import').addEventListener('click', async () => {
const jobResponse = await fetch('/api/imports', {method: 'POST', headers: {'content-type': 'application/json'}, body: JSON.stringify(currentImportPayload(url))});
const job = await jobResponse.json();
if (jobResponse.status === 401) {
await refreshAuthStatus();
throw new Error('Bitte zuerst einloggen.');
}
if (!jobResponse.ok) throw new Error(job.detail || 'Import konnte nicht gestartet werden');
document.querySelector('#job-output').textContent = renderJob(job);
await pollJob(job.id);
});
} catch (err) {
result.innerHTML = `<p class="error">${escapeHtml(err.message)}</p>`;
}
});
analyzeButton.addEventListener('click', async () => {
const url = document.querySelector('#url').value;
result.className = 'card';
candidatesSection.className = 'card hidden';
candidatesSection.innerHTML = '';
result.textContent = 'Analysiere Seite per HTML, yt-dlp und Headless-Browser…';
try {
const response = await fetch('/api/analyze', {method: 'POST', headers: {'content-type': 'application/json'}, body: JSON.stringify({url})});
const data = await response.json();
if (response.status === 401) {
await refreshAuthStatus();
throw new Error('Bitte zuerst einloggen.');
}
if (!response.ok) throw new Error(data.detail || 'Analyse fehlgeschlagen');
result.textContent = `${data.candidates.length} Kandidat(en) gefunden.`;
renderCandidates(data.candidates || []);
} catch (err) {
result.innerHTML = `<p class="error">${escapeHtml(err.message)}</p>`;
}
});
let interactiveSessionId = null;
function renderInteractive(data) {
interactiveSessionId = data.session_id;
const count = (data.candidates || []).length;
result.className = 'card interactive-card';
result.innerHTML = `
<h2>Interaktive Serveranalyse</h2>
<p>Screenshot anklicken, um die serverseitige Browser-Session zu steuern. Starte den Player; Medien-Requests erscheinen live unten.</p>
<p>${count} Kandidat(en) gefunden.${data.ublock_origin ? ' uBlock Origin ist aktiv.' : ''}</p>
<div class="interactive-toolbar">
<button id="interactive-space" type="button">Play/Pause (Leertaste)</button>
<button id="interactive-refresh" type="button">Kandidaten aktualisieren</button>
<button id="interactive-stop" type="button">Session beenden</button>
</div>
<img id="interactive-screenshot" class="interactive-screenshot" src="${escapeHtml(data.screenshot)}" alt="Interaktive Analyse-Vorschau" />
`;
const screenshot = document.querySelector('#interactive-screenshot');
screenshot.addEventListener('click', async (event) => {
const rect = screenshot.getBoundingClientRect();
const x = (event.clientX - rect.left) * (data.viewport.width / rect.width);
const y = (event.clientY - rect.top) * (data.viewport.height / rect.height);
await interactiveAction('click', {x, y});
});
document.querySelector('#interactive-space').addEventListener('click', () => interactiveAction('key', {key: 'Space'}));
document.querySelector('#interactive-refresh').addEventListener('click', () => interactiveAction('key', {key: 'Shift'}));
document.querySelector('#interactive-stop').addEventListener('click', stopInteractive);
renderCandidates(data.candidates || []);
}
async function interactiveAction(action, payload) {
if (!interactiveSessionId) return;
result.classList.add('loading');
try {
const endpoint = action === 'click'
? `/api/interactive-analyze/${interactiveSessionId}/click`
: `/api/interactive-analyze/${interactiveSessionId}/key`;
const response = await fetch(endpoint, {method: 'POST', headers: {'content-type': 'application/json'}, body: JSON.stringify(payload)});
const data = await response.json();
if (!response.ok) throw new Error(data.detail || 'Interaktive Aktion fehlgeschlagen');
renderInteractive(data);
} catch (err) {
result.innerHTML += `<p class="error">${escapeHtml(err.message)}</p>`;
} finally {
result.classList.remove('loading');
}
}
async function stopInteractive() {
if (!interactiveSessionId) return;
await fetch(`/api/interactive-analyze/${interactiveSessionId}`, {method: 'DELETE'}).catch(() => {});
interactiveSessionId = null;
result.className = 'card';
result.textContent = 'Interaktive Analyse beendet.';
}
interactiveButton.addEventListener('click', async () => {
const url = document.querySelector('#url').value;
result.className = 'card';
candidatesSection.className = 'card hidden';
candidatesSection.innerHTML = '';
result.textContent = 'Starte serverseitigen Analyse-Browser…';
try {
const response = await fetch('/api/interactive-analyze', {method: 'POST', headers: {'content-type': 'application/json'}, body: JSON.stringify({url})});
const data = await response.json();
if (response.status === 401) {
await refreshAuthStatus();
throw new Error('Bitte zuerst einloggen.');
}
if (!response.ok) throw new Error(data.detail || 'Interaktive Analyse konnte nicht gestartet werden');
renderInteractive(data);
} catch (err) {
result.innerHTML = `<p class="error">${escapeHtml(err.message)}</p>`;
}
});
function renderCandidates(candidates) {
candidatesSection.className = 'card';
if (!candidates.length) {
candidatesSection.innerHTML = '<p>Keine direkt importierbaren Medienquellen gefunden.</p>';
return;
}
candidatesSection.innerHTML = '<h2>Gefundene Medienquellen</h2>';
candidates.forEach((candidate, idx) => {
const card = document.createElement('div');
card.className = `candidate ${candidate.allowed ? '' : 'blocked'}`;
const size = candidate.file_size || (candidate.content_length ? formatBytes(candidate.content_length) : 'Dateigröße unbekannt');
const details = [candidate.kind, candidate.source, candidate.quality, `Dateigröße: ${size}`].filter(Boolean).join(' · ');
card.innerHTML = `
<div>
<strong>${escapeHtml(candidate.title || `Kandidat ${idx + 1}`)}</strong>
<p>${escapeHtml(details)}</p>
<code>${escapeHtml(candidate.url)}</code>
${candidate.reason ? `<p class="error">${escapeHtml(candidate.reason)}</p>` : ''}
</div>
<button ${candidate.allowed ? '' : 'disabled'} data-url="${escapeHtml(candidate.url)}" data-title="${escapeHtml(candidate.title || '')}">Übernehmen</button>
`;
card.querySelector('button').addEventListener('click', (event) => {
document.querySelector('#url').value = event.currentTarget.dataset.url;
if (event.currentTarget.dataset.title && !document.querySelector('#title').value) {
document.querySelector('#title').value = event.currentTarget.dataset.title;
}
result.className = 'card';
result.textContent = 'Kandidat übernommen. Du kannst jetzt Quelle prüfen oder Import starten.';
});
candidatesSection.appendChild(card);
});
}
function formatBytes(bytes) {
const units = ['B', 'KB', 'MB', 'GB'];
let value = Number(bytes || 0);
for (const unit of units) {
if (value < 1024 || unit === 'GB') return unit === 'B' ? `${value} ${unit}` : `${value.toFixed(1)} ${unit}`;
value /= 1024;
}
}
async function importCapturedFromUrl() {
const params = new URLSearchParams(location.search);
const encoded = params.get('capture') || (location.hash.startsWith('#capture=') ? location.hash.slice(9) : '');
if (!encoded) return;
try {
const payload = decodeCapturePayload(encoded);
result.className = 'card';
result.textContent = 'Browser-Capture wird geprüft…';
const response = await fetch('/api/browser-capture', {method: 'POST', headers: {'content-type': 'application/json'}, body: JSON.stringify(payload)});
const data = await response.json();
if (response.status === 401) {
await refreshAuthStatus();
throw new Error('Bitte zuerst einloggen.');
}
if (!response.ok) throw new Error(data.detail || 'Browser-Capture konnte nicht geprüft werden');
document.querySelector('#url').value = payload.page_url || '';
result.textContent = `${data.candidates.length} Kandidat(en) aus Browser-Capture gefunden.`;
renderCandidates(data.candidates || []);
history.replaceState(null, '', '/');
} catch (err) {
result.className = 'card';
result.innerHTML = `<p class="error">${escapeHtml(err.message)}</p>`;
}
}
initBookmarklet();
refreshAuthStatus().catch(() => setLoginVisible(false));
importCapturedFromUrl();

View File

View File

@@ -0,0 +1,71 @@
<!doctype html>
<html lang="de">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Kino-Projekt</title>
<link rel="stylesheet" href="/static/style.css?v=job-status-short-20260707" />
</head>
<body>
<main>
<h1>Kino-Projekt</h1>
<p>Interner Import für YouTube, legale Mediatheken und direkte erlaubte Medienlinks.</p>
<section id="login-section" class="card hidden">
<h2>Login</h2>
<form id="login-form">
<input id="username" autocomplete="username" placeholder="Benutzername" required />
<input id="password" type="password" autocomplete="current-password" placeholder="Passwort" required />
<button type="submit">Einloggen</button>
</form>
<p id="login-message" class="error"></p>
</section>
<button id="logout" type="button" class="hidden">Abmelden</button>
<form id="probe-form">
<label>
Quelle
<input id="url" type="url" placeholder="https://…" required />
</label>
<fieldset>
<legend>Zielprofil</legend>
<label><input type="radio" name="target-profile" value="clip" checked /> YouTube/Mediathek-Clip</label>
<label><input type="radio" name="target-profile" value="film" /> Film</label>
<label><input type="radio" name="target-profile" value="serie" /> Serie/Episode</label>
</fieldset>
<div class="grid">
<label>Titel-Korrektur <input id="title" placeholder="optional" /></label>
<label>Jahr <input id="year" type="number" min="1888" max="2200" placeholder="optional" /></label>
<label>Serientitel <input id="series-title" placeholder="nur Serie" /></label>
<label>Staffel <input id="season" type="number" min="1" placeholder="1" /></label>
<label>Episode <input id="episode" type="number" min="1" placeholder="optional" /></label>
<label>Episodentitel <input id="episode-title" placeholder="optional" /></label>
</div>
<button type="submit">Quelle prüfen</button>
<button id="analyze-page" type="button">Seite analysieren</button>
<button id="interactive-analyze" type="button">Interaktive Analyse starten</button>
</form>
<section id="analyzer-tools" class="card">
<h2>Browser-Capture</h2>
<p>Wenn die Serveranalyse keine Quelle findet: Für einfache Seiten kannst du weiter das Bookmarklet nutzen. Zuverlässiger ist die Kino-Capture-Browser-Extension, weil sie Medien-Requests des aktiven Tabs beobachtet.</p>
<div class="tool-actions">
<a id="extension-download" href="/static/downloads/kino-capture-extension.zip" download>Kino-Capture Extension herunterladen</a>
<a id="bookmarklet" href="#">Kino-Capture Bookmarklet</a>
<button id="copy-bookmarklet" type="button">Bookmarklet kopieren</button>
</div>
<details>
<summary>Extension installieren und nutzen</summary>
<ol>
<li>ZIP herunterladen und entpacken.</li>
<li>Chrome/Edge: <code>chrome://extensions</code> öffnen, Entwicklermodus aktivieren, „Entpackte Erweiterung laden“ wählen.</li>
<li>Firefox/LibreWolf: <code>about:debugging#/runtime/this-firefox</code> öffnen und temporäres Add-on über <code>manifest.json</code> laden.</li>
<li>Video starten, einige Sekunden laufen lassen, Extension-Icon öffnen und „An Kino-App senden“ klicken.</li>
</ol>
</details>
<p id="bookmarklet-result"></p>
</section>
<section id="result" class="card hidden"></section>
<section id="candidates" class="card hidden"></section>
<button id="import" disabled>Import starten</button>
</main>
<script src="/static/app.js?v=job-status-short-20260707"></script>
</body>
</html>

24
backend/app/web/style.css Normal file
View File

@@ -0,0 +1,24 @@
body { font-family: system-ui, sans-serif; margin: 0; background: #111827; color: #f9fafb; }
main { max-width: 860px; margin: 4rem auto; padding: 2rem; }
form { display: flex; gap: .75rem; flex-wrap: wrap; }
label { display: flex; flex: 1 1 220px; flex-direction: column; gap: .35rem; }
fieldset { flex: 1 1 100%; border: 1px solid #374151; border-radius: .75rem; display: flex; gap: 1rem; flex-wrap: wrap; }
fieldset label { flex: 0 1 auto; flex-direction: row; align-items: center; }
input { flex: 1; padding: .8rem; border-radius: .5rem; border: 1px solid #374151; background: #1f2937; color: #fff; }
input[type="radio"] { flex: 0 0 auto; }
button { padding: .8rem 1rem; border: 0; border-radius: .5rem; background: #38bdf8; color: #082f49; font-weight: 700; }
button:disabled { opacity: .45; cursor: not-allowed; }
.grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); gap: .75rem; width: 100%; }
.card { margin-top: 1rem; padding: 1rem; border: 1px solid #374151; border-radius: .75rem; background: #1f2937; }
.hidden { display: none; }
.error { color: #fca5a5; }
.candidate { margin-top: .75rem; padding-top: .75rem; border-top: 1px solid #374151; display: grid; gap: .75rem; }
.candidate code { display: block; overflow-wrap: anywhere; color: #bae6fd; }
.candidate.blocked { opacity: .7; }
#bookmarklet, #extension-download { color: #7dd3fc; overflow-wrap: anywhere; }
.tool-actions { display: flex; flex-wrap: wrap; align-items: center; gap: .75rem; }
details { margin-top: .75rem; }
details code { color: #bae6fd; }
.interactive-toolbar { display: flex; flex-wrap: wrap; gap: .75rem; margin: .75rem 0; }
.interactive-screenshot { width: 100%; max-height: 70vh; object-fit: contain; border: 1px solid #374151; border-radius: .5rem; background: #000; cursor: crosshair; }
.loading { opacity: .8; }

23
backend/pyproject.toml Normal file
View File

@@ -0,0 +1,23 @@
[project]
name = "kino-projekt-backend"
version = "0.1.0"
description = "Legal-source media import dashboard for Jellyfin"
requires-python = ">=3.11"
dependencies = [
"fastapi>=0.115",
"uvicorn[standard]>=0.30",
"pydantic-settings>=2.4",
"sqlmodel>=0.0.22",
"httpx>=0.27",
"argon2-cffi>=23.1",
"yt-dlp>=2025.1.1",
"playwright>=1.49",
]
[project.optional-dependencies]
test = ["pytest>=8.0", "pytest-asyncio>=0.23"]
[tool.pytest.ini_options]
asyncio_mode = "auto"
pythonpath = ["."]
testpaths = ["tests"]

11
backend/tests/conftest.py Normal file
View File

@@ -0,0 +1,11 @@
import pytest
from app.config import Settings
from app.main import app
@pytest.fixture(autouse=True)
def reset_app_settings():
app.state.settings = Settings(KINOPROJEKT_DB=":memory:")
yield
app.state.settings = Settings(KINOPROJEKT_DB=":memory:")

View File

@@ -0,0 +1,82 @@
from fastapi.testclient import TestClient
from app.main import app
from app.services.page_analyzer import MediaCandidate
def test_analyze_api_returns_candidates(monkeypatch):
async def fake_analyze_page(url):
return [
MediaCandidate(
url="https://cdn.example.org/movie.mp4",
kind="direct_video",
source="html",
content_length=1536,
file_size="1.5 KB",
)
]
monkeypatch.setattr("app.main.analyze_page", fake_analyze_page)
client = TestClient(app)
response = client.post("/api/analyze", json={"url": "https://example.org/watch"})
assert response.status_code == 200
body = response.json()
assert body["page_url"] == "https://example.org/watch"
assert body["candidates"][0]["file_size"] == "1.5 KB"
assert body["candidates"][0]["source"] == "html"
def test_browser_capture_api_returns_candidates(monkeypatch):
async def fake_analyze_browser_capture(captured, page_url=None):
assert page_url == "https://example.org/watch"
assert captured[0]["url"] == "/media/movie.mp4"
return [MediaCandidate(url="https://example.org/media/movie.mp4", kind="direct_video", source="browser-capture")]
monkeypatch.setattr("app.main.analyze_browser_capture", fake_analyze_browser_capture)
client = TestClient(app)
response = client.post(
"/api/browser-capture",
json={"page_url": "https://example.org/watch", "candidates": [{"url": "/media/movie.mp4"}]},
)
assert response.status_code == 200
body = response.json()
assert body["candidates"][0]["source"] == "browser-capture"
def test_interactive_analyze_api_returns_screenshot_and_candidates(monkeypatch):
class FakePage:
url = "https://example.org/watch"
class FakeSession:
id = "session-1"
page_url = "https://example.org/watch"
page = FakePage()
ublock_path = None
async def fake_start(url):
assert url == "https://example.org/watch"
return FakeSession()
async def fake_candidates(session):
assert session.id == "session-1"
return [MediaCandidate(url="https://cdn.example.org/live.m3u8", kind="hls_manifest", source="interactive-browser")]
async def fake_screenshot(session):
return "data:image/png;base64,ZmFrZQ=="
monkeypatch.setattr("app.main.interactive_browser_manager.start", fake_start)
monkeypatch.setattr("app.main.interactive_browser_manager.candidates", fake_candidates)
monkeypatch.setattr("app.main.interactive_browser_manager.screenshot_data_url", fake_screenshot)
client = TestClient(app)
response = client.post("/api/interactive-analyze", json={"url": "https://example.org/watch"})
assert response.status_code == 200
body = response.json()
assert body["session_id"] == "session-1"
assert body["screenshot"].startswith("data:image/png;base64,")
assert body["candidates"][0]["source"] == "interactive-browser"

103
backend/tests/test_auth.py Normal file
View File

@@ -0,0 +1,103 @@
from fastapi.testclient import TestClient
from app.config import Settings
from app.main import app
from app.services.auth import hash_password
def configure_auth(password: str = "correct horse battery staple"):
app.state.settings = Settings(
KINOPROJEKT_ADMIN_USER="admin",
KINOPROJEKT_ADMIN_PASSWORD_HASH=hash_password(password),
KINOPROJEKT_SESSION_SECRET="test-secret-that-is-long-enough",
KINOPROJEKT_DB=":memory:",
)
return password
def disable_auth():
app.state.settings = Settings(KINOPROJEKT_DB=":memory:")
def test_auth_disabled_keeps_mvp_api_accessible():
disable_auth()
client = TestClient(app)
response = client.get("/api/auth/status")
assert response.json() == {"enabled": False, "authenticated": True, "username": None}
def test_configured_auth_blocks_api_without_session():
configure_auth()
client = TestClient(app)
response = client.post("/api/probe", json={"url": "http://localhost/video.mp4"})
assert response.status_code == 401
def test_login_sets_secure_httponly_session_and_allows_api():
password = configure_auth()
client = TestClient(app, base_url="https://testserver")
login = client.post("/api/auth/login", json={"username": "admin", "password": password})
assert login.status_code == 204
set_cookie = login.headers["set-cookie"]
assert "HttpOnly" in set_cookie
assert "Secure" in set_cookie
assert "SameSite=lax" in set_cookie
response = client.post("/api/probe", json={"url": "http://localhost/video.mp4"})
assert response.status_code == 400
assert "private" in response.json()["detail"].lower()
def test_login_omits_secure_cookie_on_plain_http_lan_access():
password = configure_auth()
client = TestClient(app, base_url="http://testserver")
login = client.post("/api/auth/login", json={"username": "admin", "password": password})
assert login.status_code == 204
set_cookie = login.headers["set-cookie"]
assert "HttpOnly" in set_cookie
assert "Secure" not in set_cookie
def test_login_sets_secure_cookie_with_forwarded_https_proto():
password = configure_auth()
client = TestClient(app, base_url="http://testserver")
login = client.post(
"/api/auth/login",
json={"username": "admin", "password": password},
headers={"x-forwarded-proto": "https"},
)
assert login.status_code == 204
assert "Secure" in login.headers["set-cookie"]
def test_logout_omits_secure_cookie_on_plain_http_lan_access():
configure_auth()
client = TestClient(app, base_url="http://testserver")
logout = client.post("/api/auth/logout")
assert logout.status_code == 204
assert "Secure" not in logout.headers["set-cookie"]
def test_logout_sets_secure_cookie_with_forwarded_https_proto():
configure_auth()
client = TestClient(app, base_url="http://testserver")
logout = client.post("/api/auth/logout", headers={"x-forwarded-proto": "https"})
assert logout.status_code == 204
assert "Secure" in logout.headers["set-cookie"]
def test_login_rejects_wrong_password():
configure_auth()
client = TestClient(app, base_url="https://testserver")
response = client.post("/api/auth/login", json={"username": "admin", "password": "wrong"})
assert response.status_code == 401

View File

@@ -0,0 +1,101 @@
from pathlib import Path
import pytest
from app.providers.base import DownloadOptions, MediaMetadata
from app.providers.mediathek import MediathekProvider, is_stream_manifest
from app.providers.youtube import parse_yt_dlp_progress
from app.services.downloader import copy_tree_contents
class FakeStream:
def __init__(self, lines: list[bytes]):
self.lines = lines
async def readline(self) -> bytes:
if self.lines:
return self.lines.pop(0)
return b""
class FakeProcess:
def __init__(self, output_path: Path):
self.returncode = 0
self.stdout = FakeStream([b"[download] 100% of 12.00MiB\n"])
self.stderr = FakeStream([])
self.output_path = output_path
async def wait(self) -> int:
self.output_path.write_bytes(b"real media bytes")
return self.returncode
def test_copy_tree_contents_copies_files(tmp_path: Path):
src = tmp_path / 'src'
src.mkdir()
(src / 'movie.mp4').write_bytes(b'data')
(src / 'meta.info.json').write_text('{}')
dest = tmp_path / 'dest'
copied = copy_tree_contents(src, dest)
assert sorted(p.name for p in copied) == ['meta.info.json', 'movie.mp4']
assert (dest / 'movie.mp4').read_bytes() == b'data'
def test_copy_tree_contents_rejects_top_level_symlink(tmp_path: Path):
src = tmp_path / "src"
src.mkdir()
(tmp_path / "secret.txt").write_text("secret")
(src / "leak.txt").symlink_to(tmp_path / "secret.txt")
with pytest.raises(Exception, match="symlink"):
copy_tree_contents(src, tmp_path / "dest")
def test_copy_tree_contents_rejects_nested_symlink(tmp_path: Path):
src = tmp_path / "src"
nested = src / "nested"
nested.mkdir(parents=True)
(tmp_path / "secret.txt").write_text("secret")
(nested / "leak.txt").symlink_to(tmp_path / "secret.txt")
with pytest.raises(Exception, match="symlink"):
copy_tree_contents(src, tmp_path / "dest")
def test_parse_yt_dlp_progress_lines():
assert parse_yt_dlp_progress('[download] 42.7% of 10.00MiB at 1.00MiB/s ETA 00:05') == pytest.approx(0.427)
assert parse_yt_dlp_progress('[download] 100% of 10.00MiB') == 1.0
assert parse_yt_dlp_progress('[info] unrelated') is None
def test_stream_manifest_detection():
assert is_stream_manifest('https://example.org/master.m3u8')
assert is_stream_manifest('https://example.org/manifest.mpd')
assert is_stream_manifest('https://example.org/video', 'application/vnd.apple.mpegurl')
assert not is_stream_manifest('https://example.org/video.mp4', 'video/mp4')
async def test_m3u8_download_uses_ytdlp_not_manifest_file(monkeypatch, tmp_path: Path):
provider = MediathekProvider()
manifest_url = 'https://cdn.example.org/master.m3u8'
async def fake_probe(url: str) -> MediaMetadata:
return MediaMetadata(provider='mediathek', title='master.m3u8', external_id=manifest_url)
async def fake_create_subprocess_exec(*args, **kwargs):
assert args[0].endswith('yt-dlp')
assert '--proxy' not in args
assert '--remux-video' in args
output_template = Path(args[args.index('--output') + 1])
return FakeProcess(output_template.parent / 'downloaded.mp4')
monkeypatch.setattr(provider, 'probe', fake_probe)
monkeypatch.setattr('app.providers.mediathek.shutil.which', lambda name: '/usr/bin/yt-dlp' if name == 'yt-dlp' else None)
monkeypatch.setattr('app.providers.mediathek.asyncio.create_subprocess_exec', fake_create_subprocess_exec)
result = await provider.download(manifest_url, tmp_path, DownloadOptions())
assert [p.name for p in result.output_files] == ['downloaded.mp4']
assert not (tmp_path / 'master.m3u8').exists()

View File

@@ -0,0 +1,7 @@
from fastapi.testclient import TestClient
from app.main import app
def test_health():
client = TestClient(app)
assert client.get('/health').json() == {'status': 'ok'}

View File

@@ -0,0 +1,45 @@
import pytest
from app.main import resolve_importable_url
from app.providers.base import MediaMetadata, ProviderError
from app.services.page_analyzer import MediaCandidate
class FakeProvider:
name = "mediathek"
def can_handle(self, url: str) -> bool:
return url.startswith("https://")
async def probe(self, url: str) -> MediaMetadata:
if url == "https://example.org/watch":
raise ProviderError("URL is reachable, but content type/extension is not accepted: text/html")
if url == "https://cdn.example.org/master.m3u8":
return MediaMetadata(provider=self.name, title="master.m3u8", external_id=url)
raise ProviderError("unexpected url")
async def test_html_page_import_resolves_to_best_media_candidate(monkeypatch):
async def fake_analyze_page(url: str):
assert url == "https://example.org/watch"
return [
MediaCandidate(
url="https://cdn.example.org/master.m3u8",
kind="hls_manifest",
title="HLS stream",
source="browser",
content_length=4096,
allowed=True,
)
]
fake_provider = FakeProvider()
monkeypatch.setattr("app.main.providers", [fake_provider])
monkeypatch.setattr("app.main.analyze_page", fake_analyze_page)
resolved_url, provider, metadata, candidate = await resolve_importable_url("https://example.org/watch")
assert resolved_url == "https://cdn.example.org/master.m3u8"
assert provider is fake_provider
assert metadata.external_id == resolved_url
assert candidate.kind == "hls_manifest"

View File

@@ -0,0 +1,46 @@
import pytest
from app.providers.base import ProviderError
from app.providers.internet_archive import InternetArchiveProvider
def test_internet_archive_detection():
provider = InternetArchiveProvider()
assert provider.can_handle("https://archive.org/details/BigBuckBunny_328")
assert provider.can_handle("https://archive.org/download/BigBuckBunny_328/video.mp4")
assert not provider.can_handle("https://example.com/details/BigBuckBunny_328")
def test_internet_archive_identifier_parsing():
provider = InternetArchiveProvider()
assert provider._identifier_from_url("https://archive.org/details/BigBuckBunny_328") == "BigBuckBunny_328"
assert provider._identifier_from_url("https://archive.org/download/BigBuckBunny_328/video.mp4") == "BigBuckBunny_328"
assert provider._identifier_from_url("https://archive.org/metadata/BigBuckBunny_328") == "BigBuckBunny_328"
def test_select_media_file_prefers_original_mp4():
provider = InternetArchiveProvider()
selected = provider._select_media_file([
{"name": "thumb.jpg", "format": "JPEG", "source": "metadata", "size": "10"},
{"name": "movie.ogv", "format": "Ogg Video", "source": "derivative", "size": "200000000"},
{"name": "movie.mp4", "format": "MPEG4", "source": "original", "size": "100000000"},
])
assert selected is not None
assert selected["name"] == "movie.mp4"
def test_select_media_file_rejects_path_traversal():
provider = InternetArchiveProvider()
assert provider._select_media_file([{"name": "../evil.mp4", "format": "MPEG4", "source": "original"}]) is None
@pytest.mark.asyncio
async def test_probe_requires_media_file(monkeypatch):
provider = InternetArchiveProvider()
async def fake_metadata(identifier: str) -> dict:
return {"metadata": {"title": "No media"}, "files": [{"name": "notes.txt", "format": "Text"}]}
monkeypatch.setattr(provider, "_metadata", fake_metadata)
with pytest.raises(ProviderError):
await provider.probe("https://archive.org/details/no-media")

View File

@@ -0,0 +1,9 @@
import pytest
from app.providers.base import ProviderError
from app.providers.mediathek import MediathekProvider
def test_private_target_rejected():
with pytest.raises(ProviderError):
MediathekProvider()._reject_private_targets('localhost')

View File

@@ -0,0 +1,19 @@
from sqlmodel import Session, select
from app.db import get_engine, init_db
from app.models import ImportJob, Source
def test_models_roundtrip(tmp_path):
engine = get_engine(tmp_path / 'kino.sqlite3')
init_db(engine)
with Session(engine) as session:
src = Source(kind='youtube', url='https://youtu.be/example', title='Example', provider='youtube')
session.add(src)
session.commit()
session.refresh(src)
job = ImportJob(source_id=src.id, target_library='movies')
session.add(job)
session.commit()
assert session.exec(select(Source)).one().title == 'Example'
assert session.exec(select(ImportJob)).one().status == 'queued'

View File

@@ -0,0 +1,175 @@
import pytest
from app.services import page_analyzer
from app.services.page_analyzer import (
MediaCandidate,
analyze_browser_capture,
analyze_page,
analyze_with_ytdlp,
extract_media_urls,
human_size,
_fetch_public_page_body,
)
def test_human_size_labels_file_sizes():
assert human_size(None) is None
assert human_size(512) == "512 B"
assert human_size(1536) == "1.5 KB"
assert human_size(5 * 1024 * 1024) == "5.0 MB"
def test_extract_media_urls_finds_encoded_and_unquoted_js_urls():
html = """
<video data-src="/media/trailer.mp4?token=abc&amp;v=1"></video>
<script>
const hls = "https:\\/\\/cdn.example.org\\/movie\\/master.m3u8?sig=123";
window.player = {dash: https://cdn.example.org/movie/manifest.mpd?x=1};
</script>
"""
assert extract_media_urls(html, "https://example.org/watch") == [
"https://example.org/media/trailer.mp4?token=abc&v=1",
"https://cdn.example.org/movie/master.m3u8?sig=123",
"https://cdn.example.org/movie/manifest.mpd?x=1",
]
@pytest.mark.asyncio
async def test_browser_capture_filters_and_normalizes_candidates(monkeypatch):
captured_probe_input = {}
async def fake_probe(urls, source):
captured_probe_input["urls"] = urls
captured_probe_input["source"] = source
return [MediaCandidate(url=url, kind="direct_video", source=source) for url in urls]
monkeypatch.setattr(page_analyzer, "_probe_candidates_with_meta", fake_probe)
result = await analyze_browser_capture(
[
{"url": "/media/movie.mp4", "title": "Movie"},
{"url": "https://cdn.example.org/live/master.m3u8", "content_type": "application/vnd.apple.mpegurl"},
{"url": "https://cdn.example.org/poster.jpg", "content_type": "image/jpeg"},
],
page_url="https://example.org/watch/1",
)
assert captured_probe_input["source"] == "browser-capture"
assert list(captured_probe_input["urls"]) == [
"https://example.org/media/movie.mp4",
"https://cdn.example.org/live/master.m3u8",
]
assert [item.source for item in result] == ["browser-capture", "browser-capture"]
@pytest.mark.asyncio
async def test_analyze_page_merges_browser_ytdlp_and_html_candidates(monkeypatch):
async def fake_html(url):
return [MediaCandidate(url="https://cdn.example.org/static.mp4", kind="direct_video", source="html")]
async def fake_ytdlp(url):
return [MediaCandidate(url="https://cdn.example.org/ytdlp.m3u8", kind="hls_manifest", source="yt-dlp")]
async def fake_browser(url):
return [MediaCandidate(url="https://cdn.example.org/browser.mpd", kind="dash_manifest", source="browser")]
monkeypatch.setattr(page_analyzer, "analyze_html_page", fake_html)
monkeypatch.setattr(page_analyzer, "analyze_with_ytdlp", fake_ytdlp)
monkeypatch.setattr(page_analyzer, "analyze_with_browser", fake_browser)
result = await analyze_page("https://example.org/watch")
assert {candidate.url: candidate.source for candidate in result} == {
"https://cdn.example.org/static.mp4": "html",
"https://cdn.example.org/ytdlp.m3u8": "yt-dlp",
"https://cdn.example.org/browser.mpd": "browser",
}
@pytest.mark.asyncio
async def test_ytdlp_is_skipped_for_generic_untrusted_hosts(monkeypatch):
calls = []
class Checked:
final_url = "https://example.org/watch"
async def fake_check(url, **kwargs):
return Checked()
class FakeYDL:
def __init__(self, opts):
calls.append(opts)
def __enter__(self):
return self
def __exit__(self, *args):
return None
def extract_info(self, url, download=False):
raise AssertionError("yt-dlp must not run for generic hosts")
monkeypatch.setattr(page_analyzer, "check_url_for_page", fake_check)
monkeypatch.setattr(page_analyzer, "YoutubeDL", FakeYDL)
assert await analyze_with_ytdlp("https://example.org/watch") == []
assert calls == []
@pytest.mark.asyncio
async def test_fetch_public_page_body_revalidates_get_redirect_hops(monkeypatch):
checked_urls = []
class Checked:
def __init__(self, final_url):
self.final_url = final_url
async def fake_check(url, **kwargs):
checked_urls.append(url)
return Checked(url)
async def fake_assert_public_host(url):
checked_urls.append(f"assert:{url}")
class FakeURL:
def __init__(self, url):
self._url = url
def join(self, loc):
return loc
def __str__(self):
return self._url
class FakeResponse:
def __init__(self, url, redirect_to=None, text=""):
self.url = FakeURL(url)
self._redirect_to = redirect_to
self.headers = {"location": redirect_to} if redirect_to else {}
self.text = text
@property
def is_redirect(self):
return self._redirect_to is not None
def raise_for_status(self):
return None
class FakeClient:
def __init__(self, *args, **kwargs):
self.calls = 0
async def __aenter__(self):
return self
async def __aexit__(self, *args):
return None
async def get(self, url):
self.calls += 1
if self.calls == 1:
return FakeResponse(url, redirect_to="https://cdn.example.org/page")
return FakeResponse(url, text="<video src='/movie.mp4'></video>")
monkeypatch.setattr(page_analyzer, "check_url_for_page", fake_check)
monkeypatch.setattr(page_analyzer, "assert_public_host", fake_assert_public_host)
monkeypatch.setattr(page_analyzer.httpx, "AsyncClient", FakeClient)
body, final_url = await _fetch_public_page_body("https://example.org/watch", timeout=1)
assert body.startswith("<video")
assert final_url == "https://cdn.example.org/page"
assert "https://example.org/watch" in checked_urls
assert "https://cdn.example.org/page" in checked_urls
assert "assert:https://cdn.example.org/page" in checked_urls

View File

@@ -0,0 +1,56 @@
from pathlib import Path
import pytest
from app.services.paths import ensure_under, jellyfin_profile_target, media_target, slugify_name, tmp_target
def test_slugify_name_removes_dangerous_chars():
assert slugify_name('../Bad:Title?') == 'Bad_Title'
def test_ensure_under_accepts_child(tmp_path: Path):
assert ensure_under(tmp_path, Path('a/b')).is_relative_to(tmp_path.resolve())
def test_ensure_under_rejects_escape(tmp_path: Path):
with pytest.raises(ValueError):
ensure_under(tmp_path, Path('../escape'))
def test_targets_stay_in_configured_roots(tmp_path: Path):
assert tmp_target(tmp_path / 'tmp', 7, '../../Film').is_relative_to((tmp_path / 'tmp').resolve())
assert media_target(tmp_path / 'media', 'movies', '../../Film').is_relative_to((tmp_path / 'media').resolve())
def test_jellyfin_film_profile_matches_real_library_layout(tmp_path: Path):
target = jellyfin_profile_target(tmp_path / 'media', 'film', 'Big Buck Bunny', year=2008)
assert target == (tmp_path / 'media' / 'Filme' / 'Big Buck Bunny (2008)').resolve()
def test_jellyfin_series_profile_uses_staffel_folder(tmp_path: Path):
target = jellyfin_profile_target(
tmp_path / 'media',
'serie',
'Pilot/Unsafe',
series_title='The Show',
season=2,
episode=3,
episode_title='Pilot/Unsafe',
)
assert target == (tmp_path / 'media' / 'Serien' / 'The Show' / 'Staffel2' / 'S02E03 - Pilot_Unsafe').resolve()
def test_jellyfin_clip_profile_uses_youtube_mediathek(tmp_path: Path):
target = jellyfin_profile_target(
tmp_path / 'media',
'clip',
'../../Clip',
channel='Archive Org',
external_id='abc/123',
)
assert target == (tmp_path / 'media' / 'YouTube-Mediathek' / 'Archive Org' / 'Clip [abc_123]').resolve()
def test_jellyfin_profile_rejects_unknown_profile(tmp_path: Path):
with pytest.raises(ValueError):
jellyfin_profile_target(tmp_path / 'media', 'other', 'Title')

View File

@@ -0,0 +1,9 @@
from fastapi.testclient import TestClient
from app.main import app
def test_probe_rejects_private_direct_url():
client = TestClient(app)
response = client.post('/api/probe', json={'url': 'http://localhost/video.mp4'})
assert response.status_code == 400

View File

@@ -0,0 +1,24 @@
from app.providers.internet_archive import InternetArchiveProvider
from app.providers.mediathek import MediathekProvider
from app.providers.youtube import YouTubeProvider
def test_youtube_detection():
provider = YouTubeProvider()
assert provider.can_handle('https://www.youtube.com/watch?v=abc')
assert provider.can_handle('https://youtu.be/abc')
assert not provider.can_handle('https://example.com/video.mp4')
def test_mediathek_detection_excludes_youtube_and_internet_archive():
provider = MediathekProvider()
assert provider.can_handle('https://example.com/video.mp4')
assert not provider.can_handle('https://youtube.com/watch?v=abc')
assert not provider.can_handle('https://archive.org/details/BigBuckBunny_328')
def test_internet_archive_detection():
provider = InternetArchiveProvider()
assert provider.can_handle('https://archive.org/details/BigBuckBunny_328')
assert provider.can_handle('https://archive.org/download/BigBuckBunny_328/video.mp4')
assert not provider.can_handle('https://example.com/video.mp4')

View File

@@ -0,0 +1,7 @@
from app.providers.youtube import YouTubeProvider
def test_extract_youtube_id():
provider = YouTubeProvider()
assert provider._external_id('https://youtu.be/abc123') == 'abc123'
assert provider._external_id('https://www.youtube.com/watch?v=xyz') == 'xyz'

View File

@@ -0,0 +1,32 @@
# Kino Capture Browser-Extension
Kleine generische Browser-Extension für das Kino-Projekt. Sie sammelt im aktiven Browser-Tab sichtbare Medien-Requests und DOM-/Performance-Kandidaten und übergibt diese an die Kino-App.
## Installation als entpackte Extension
### Chromium / Chrome / Edge
1. Ordner `kino-capture` entpacken.
2. `chrome://extensions` öffnen.
3. Entwicklermodus aktivieren.
4. „Entpackte Erweiterung laden“ klicken.
5. Den Ordner `kino-capture` auswählen.
### Firefox / LibreWolf temporär
1. `about:debugging#/runtime/this-firefox` öffnen.
2. „Temporäres Add-on laden“ klicken.
3. `manifest.json` im Ordner `kino-capture` auswählen.
## Nutzung
1. Video im Browser starten und einige Sekunden laufen lassen.
2. Extension-Icon „Kino Capture“ öffnen.
3. Prüfen, dass Kandidaten angezeigt werden.
4. Kino-App-URL setzen, z. B. `http://192.168.178.213:8099/`.
5. „An Kino-App senden“ klicken.
6. In der Kino-App Kandidat übernehmen und Import starten.
## Grenzen
Die Extension sammelt nur Medien-URLs, die der Browser normal sichtbar anfragt oder im DOM/Performance-API offenlegt. Sie enthält keine hoster-spezifischen Bypässe, keine CAPTCHA-/DRM-/Token-Umgehung und keine Entschlüsselungslogik.

View File

@@ -0,0 +1,68 @@
const MEDIA_EXT_RE = /\.(mp4|m4v|webm|mkv|mov|m3u8|mpd)([?#]|$)/i;
const ABSOLUTE_MEDIA_RE = /(?:https?:\/\/|https?:\\\/\\\/|\/\/|\\\/\\\/)[^\s"'<>]+?\.(?:mp4|m4v|webm|mkv|mov|m3u8|mpd)(?=$|[?&#\s"'<>])(?:[?&][^\s"'<>]*)?/gi;
function ignoreChromeResult(result) {
if (result && typeof result.catch === 'function') result.catch(() => {});
}
function normalizeMediaUrl(rawUrl, baseUrl = location.href) {
if (!rawUrl || typeof rawUrl !== 'string') return null;
try {
const cleaned = rawUrl
.trim()
.replaceAll('\\/', '/')
.replaceAll('\\u002F', '/')
.replaceAll('\\u002f', '/')
.replace(/[;,)}\]]+$/, '');
return new URL(cleaned, baseUrl).href;
} catch (_) {
return null;
}
}
function pushCandidate(candidates, rawUrl, title = document.title, contentType = '', source = 'dom') {
const url = normalizeMediaUrl(rawUrl);
if (!url) return;
const cleanType = String(contentType || '').split(';', 1)[0].trim();
if (MEDIA_EXT_RE.test(url) || cleanType.startsWith('video/') || /mpegurl|dash\+xml/i.test(cleanType)) {
candidates.push({url, title: title || document.title, content_type: contentType || '', source});
}
}
function collectCandidates() {
const candidates = [];
document.querySelectorAll('video,source,a').forEach((el) => {
pushCandidate(candidates, el.currentSrc || el.src || el.href, el.title || el.textContent || document.title, el.type || '', 'dom');
});
performance.getEntriesByType('resource').forEach((entry) => {
pushCandidate(candidates, entry.name, document.title, entry.initiatorType === 'video' ? 'video/unknown' : '', `performance:${entry.initiatorType || 'resource'}`);
});
const html = document.documentElement?.outerHTML || '';
for (const match of html.matchAll(ABSOLUTE_MEDIA_RE)) {
pushCandidate(candidates, match[0], document.title, '', 'html-js');
}
const seen = new Set();
return candidates.filter((candidate) => {
if (seen.has(candidate.url)) return false;
seen.add(candidate.url);
return true;
}).slice(0, 120);
}
chrome.runtime.onMessage.addListener((message, _sender, sendResponse) => {
if (message?.type === 'collect-media') {
sendResponse({candidates: collectCandidates()});
return true;
}
return false;
});
// Opportunistisch nach dem Laden melden, damit das Popup sofort Daten hat.
setTimeout(() => {
for (const candidate of collectCandidates()) {
ignoreChromeResult(chrome.runtime.sendMessage({type: 'candidate', candidate}));
}
}, 1500);

View File

@@ -0,0 +1,30 @@
{
"manifest_version": 2,
"name": "Kino Capture",
"version": "0.1.1",
"description": "Sammelt sichtbare Medien-Requests im aktiven Tab und übergibt sie an die Kino-Projekt-App.",
"permissions": [
"activeTab",
"storage",
"tabs",
"webNavigation",
"webRequest",
"<all_urls>"
],
"background": {
"scripts": ["service_worker.js"],
"persistent": false
},
"content_scripts": [
{
"matches": ["<all_urls>"],
"js": ["content.js"],
"run_at": "document_idle",
"all_frames": true
}
],
"browser_action": {
"default_title": "Kino Capture",
"default_popup": "popup.html"
}
}

View File

@@ -0,0 +1,35 @@
body {
min-width: 380px;
max-width: 520px;
margin: 0;
background: #111827;
color: #f9fafb;
font: 14px system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
}
main { padding: 14px; }
h1 { margin: 0 0 8px; font-size: 18px; }
label { display: grid; gap: 4px; margin: 10px 0; }
input {
padding: 8px;
border-radius: 8px;
border: 1px solid #374151;
background: #1f2937;
color: #fff;
}
button {
padding: 8px 10px;
border: 0;
border-radius: 8px;
background: #38bdf8;
color: #082f49;
font-weight: 700;
cursor: pointer;
}
button.secondary { background: #374151; color: #f9fafb; }
button:disabled { opacity: .5; cursor: not-allowed; }
.actions { display: flex; flex-wrap: wrap; gap: 8px; }
ol { padding-left: 20px; max-height: 320px; overflow: auto; }
li { margin: 8px 0; border-top: 1px solid #374151; padding-top: 8px; }
code { display: block; color: #bae6fd; overflow-wrap: anywhere; font-size: 12px; }
.meta, .hint { color: #d1d5db; font-size: 12px; }
.error { color: #fca5a5; }

View File

@@ -0,0 +1,27 @@
<!doctype html>
<html lang="de">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Kino Capture</title>
<link rel="stylesheet" href="popup.css" />
</head>
<body>
<main>
<h1>Kino Capture</h1>
<p id="status">Lade Kandidaten…</p>
<label>
Kino-App URL
<input id="app-url" type="url" placeholder="http://192.168.178.213:8099/" />
</label>
<div class="actions">
<button id="refresh" type="button">Neu scannen</button>
<button id="send" type="button">An Kino-App senden</button>
<button id="clear" type="button" class="secondary">Leeren</button>
</div>
<ol id="candidates"></ol>
<p class="hint">Tipp: Video starten und einige Sekunden laufen lassen, dann dieses Popup öffnen.</p>
</main>
<script src="popup.js"></script>
</body>
</html>

View File

@@ -0,0 +1,103 @@
const DEFAULT_APP_URL = 'http://192.168.178.213:8099/';
const statusEl = document.querySelector('#status');
const listEl = document.querySelector('#candidates');
const appUrlEl = document.querySelector('#app-url');
const sendButton = document.querySelector('#send');
const refreshButton = document.querySelector('#refresh');
const clearButton = document.querySelector('#clear');
let activeTab = null;
let currentPayload = {page_url: '', candidates: []};
function encodeCapturePayload(payload) {
return btoa(unescape(encodeURIComponent(JSON.stringify(payload))));
}
function normalizeAppUrl(value) {
try {
const url = new URL(value || DEFAULT_APP_URL);
if (!url.pathname.endsWith('/')) url.pathname = `${url.pathname}/`;
url.search = '';
url.hash = '';
return url.href;
} catch (_) {
return DEFAULT_APP_URL;
}
}
function render(payload) {
currentPayload = payload || {page_url: activeTab?.url || '', candidates: []};
const candidates = currentPayload.candidates || [];
statusEl.textContent = `${candidates.length} Kandidat(en) im aktiven Tab gefunden.`;
statusEl.className = candidates.length ? '' : 'error';
sendButton.disabled = candidates.length === 0;
listEl.innerHTML = '';
for (const candidate of candidates.slice(0, 30)) {
const li = document.createElement('li');
const type = candidate.content_type || candidate.source || 'media';
li.innerHTML = `<div class="meta"></div><code></code>`;
li.querySelector('.meta').textContent = type;
li.querySelector('code').textContent = candidate.url;
listEl.appendChild(li);
}
}
async function getActiveTab() {
const [tab] = await chrome.tabs.query({active: true, currentWindow: true});
return tab;
}
async function refresh() {
activeTab = await getActiveTab();
if (!activeTab?.id) {
statusEl.textContent = 'Kein aktiver Tab gefunden.';
statusEl.className = 'error';
return;
}
const response = await chrome.runtime.sendMessage({type: 'get-candidates', tabId: activeTab.id});
if (!response?.ok) {
statusEl.textContent = response?.error || 'Kandidaten konnten nicht gelesen werden.';
statusEl.className = 'error';
return;
}
render(response);
}
async function saveAppUrl() {
const appUrl = normalizeAppUrl(appUrlEl.value);
appUrlEl.value = appUrl;
await chrome.storage.local.set({kinoAppUrl: appUrl});
return appUrl;
}
async function sendToKinoApp() {
const appUrl = await saveAppUrl();
const encoded = encodeCapturePayload({
page_url: currentPayload.page_url || activeTab?.url || '',
candidates: currentPayload.candidates || [],
});
const destination = `${appUrl}?capture=${encodeURIComponent(encoded)}`;
await chrome.tabs.create({url: destination});
}
async function clearCandidates() {
if (!activeTab?.id) activeTab = await getActiveTab();
if (activeTab?.id) await chrome.runtime.sendMessage({type: 'clear-candidates', tabId: activeTab.id});
render({page_url: activeTab?.url || '', candidates: []});
}
async function init() {
const stored = await chrome.storage.local.get({kinoAppUrl: DEFAULT_APP_URL});
appUrlEl.value = normalizeAppUrl(stored.kinoAppUrl);
await refresh();
}
appUrlEl.addEventListener('change', saveAppUrl);
refreshButton.addEventListener('click', refresh);
sendButton.addEventListener('click', sendToKinoApp);
clearButton.addEventListener('click', clearCandidates);
init().catch((error) => {
statusEl.textContent = error?.message || String(error);
statusEl.className = 'error';
});

View File

@@ -0,0 +1,138 @@
const MEDIA_EXT_RE = /\.(mp4|m4v|webm|mkv|mov|m3u8|mpd)([?#]|$)/i;
const MEDIA_CONTENT_RE = /^(video\/|application\/(vnd\.apple\.mpegurl|x-mpegurl|dash\+xml)|audio\/mpegurl)/i;
const MAX_CANDIDATES_PER_TAB = 120;
const tabCandidates = new Map();
const tabPageUrls = new Map();
const actionApi = chrome.action || chrome.browserAction;
function ignoreChromeResult(result) {
if (result && typeof result.catch === 'function') result.catch(() => {});
}
function setBadgeTextSafe(tabId, text) {
if (actionApi?.setBadgeText) ignoreChromeResult(actionApi.setBadgeText({tabId, text}));
}
function setBadgeBackgroundColorSafe(tabId, color) {
if (actionApi?.setBadgeBackgroundColor) ignoreChromeResult(actionApi.setBadgeBackgroundColor({tabId, color}));
}
function candidateKey(candidate) {
return candidate.url;
}
function normalizeUrl(rawUrl) {
if (!rawUrl || typeof rawUrl !== 'string') return null;
try {
return new URL(rawUrl).href;
} catch (_) {
return null;
}
}
function looksLikeMedia(url, contentType = '') {
return MEDIA_EXT_RE.test(url) || MEDIA_CONTENT_RE.test(String(contentType).split(';', 1)[0].trim());
}
function addCandidate(tabId, candidate) {
if (tabId < 0 || !candidate?.url) return;
const url = normalizeUrl(candidate.url);
if (!url) return;
const contentType = candidate.content_type || candidate.type || '';
if (!looksLikeMedia(url, contentType)) return;
const existing = tabCandidates.get(tabId) || [];
if (existing.some((item) => candidateKey(item) === url)) return;
existing.unshift({
url,
title: candidate.title || candidate.name || '',
content_type: contentType,
quality: candidate.quality || '',
source: candidate.source || 'webRequest',
ts: Date.now(),
});
tabCandidates.set(tabId, existing.slice(0, MAX_CANDIDATES_PER_TAB));
setBadgeTextSafe(tabId, String(Math.min(existing.length, 99)));
setBadgeBackgroundColorSafe(tabId, '#38bdf8');
}
chrome.webRequest.onBeforeRequest.addListener(
(details) => {
if (details.tabId < 0 || details.type === 'image') return;
if (looksLikeMedia(details.url)) {
addCandidate(details.tabId, {url: details.url, source: `request:${details.type}`});
}
},
{urls: ['<all_urls>']}
);
chrome.webRequest.onHeadersReceived.addListener(
(details) => {
if (details.tabId < 0) return;
const header = (details.responseHeaders || []).find((item) => item.name.toLowerCase() === 'content-type');
const contentType = header?.value || '';
if (looksLikeMedia(details.url, contentType)) {
addCandidate(details.tabId, {url: details.url, content_type: contentType, source: 'response'});
}
},
{urls: ['<all_urls>']},
['responseHeaders']
);
chrome.webNavigation?.onCommitted?.addListener?.((details) => {
if (details.frameId === 0) {
tabPageUrls.set(details.tabId, details.url);
tabCandidates.delete(details.tabId);
setBadgeTextSafe(details.tabId, '');
}
});
chrome.tabs.onRemoved.addListener((tabId) => {
tabCandidates.delete(tabId);
tabPageUrls.delete(tabId);
});
async function collectDomCandidates(tabId) {
try {
const responses = await chrome.tabs.sendMessage(tabId, {type: 'collect-media'});
return Array.isArray(responses) ? responses : responses?.candidates || [];
} catch (_) {
return [];
}
}
chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
(async () => {
if (message?.type === 'candidate' && sender.tab?.id != null) {
addCandidate(sender.tab.id, message.candidate);
sendResponse({ok: true});
return;
}
if (message?.type === 'get-candidates') {
const tabId = message.tabId;
const domCandidates = await collectDomCandidates(tabId);
for (const candidate of domCandidates) addCandidate(tabId, {...candidate, source: candidate.source || 'dom'});
const tab = await chrome.tabs.get(tabId).catch(() => null);
sendResponse({
ok: true,
page_url: tab?.url || tabPageUrls.get(tabId) || '',
candidates: tabCandidates.get(tabId) || [],
});
return;
}
if (message?.type === 'clear-candidates') {
const tabId = message.tabId;
tabCandidates.delete(tabId);
setBadgeTextSafe(tabId, '');
sendResponse({ok: true});
return;
}
sendResponse({ok: false, error: 'unknown message'});
})();
return true;
});

11
docker-compose.yml Normal file
View File

@@ -0,0 +1,11 @@
services:
kino-projekt:
build: .
ports:
- "127.0.0.1:8099:8080"
volumes:
- ./data:/data
- /jellyfin:/jellyfin
env_file:
- .env
restart: unless-stopped

19
docs/deployment.md Normal file
View File

@@ -0,0 +1,19 @@
# Deployment
1. `.env` auf dem Zielhost aus `.env.example` erstellen; `JELLYFIN_API_KEY`, `KINOPROJEKT_ADMIN_PASSWORD_HASH` und `KINOPROJEKT_SESSION_SECRET` nur dort oder in Vaultwarden speichern.
2. Medien- und Datenverzeichnisse als Volumes mounten.
3. Dienst nur Mesh-only veröffentlichen und den App-Login aktivieren.
4. Nach Start prüfen:
```bash
curl http://127.0.0.1:8080/health
curl -X POST http://127.0.0.1:8080/api/probe \
-H 'content-type: application/json' \
--data '{"url":"http://localhost/video.mp4"}'
# Erwartet: HTTP 400, weil private/loopback Ziele abgelehnt werden.
```
5. Login prüfen: Ohne Session antworten geschützte API-Routen mit `401`; nach `POST /api/auth/login` setzt die App ein `HttpOnly; Secure; SameSite=Lax` Session-Cookie.
6. Zielprofile prüfen: Ein Import mit `target_profile=film|serie|clip` muss unter `/jellyfin/Filme`, `/jellyfin/Serien/.../Staffel<N>` oder `/jellyfin/YouTube-Mediathek` landen. Der Ordner `/jellyfin/YouTube-Mediathek` sollte auf dem Zielhost `jellyfinuser:media` gehören und setgid wie die vorhandenen Bibliotheken nutzen.
7. Für echte Importe muss `yt-dlp` im Container/Host verfügbar sein; das Dockerfile installiert es.
8. Jellyfin-Refresh wird über `POST <JELLYFIN_URL>/Library/Refresh` mit `X-Emby-Token` ausgelöst, wenn `JELLYFIN_API_KEY` gesetzt ist.

11
docs/provider-policy.md Normal file
View File

@@ -0,0 +1,11 @@
# Provider Policy
Erlaubt sind nur öffentlich und legal erreichbare Quellen ohne Umgehung technischer oder rechtlicher Zugriffsbeschränkungen.
## Implementierte Provider
- **YouTube:** Metadaten/Downloads über `yt-dlp`, ohne Cookies, Login-Flows oder Umgehungsflags; Default-Qualität maximal 1080p.
- **Internet Archive:** akzeptiert `archive.org/details/<identifier>`, `archive.org/download/<identifier>/...` und `archive.org/metadata/<identifier>`; Metadaten kommen aus der öffentlichen Archive-Metadata-API, heruntergeladen wird nur ein plausibler öffentlicher Medien-Track.
- **Mediathek/Direktlink:** direkte URLs werden vor Verwendung gegen SSRF-Risiken geprüft: Schema, DNS, Redirects, private IP-Ranges und Content-Type/Dateiendung.
Nicht erlaubt sind Piracy-Scraper, DRM-/Paywall-/Login-/Geoblocking-Umgehung, Cookie-Importe und freie Server-Schreibpfade.