Add Pinchflat-style media manager: index channels into playlist folders
Index an entire YouTube channel or playlist once, then download it into organized <Channel>/<Playlist>/<NNN> - <Title> folders, with incremental re-sync. Built in six rebuild-gated phases (F-K); see ROADMAP-PINCHFLAT.md. - F: channel-walk indexer (/playlists + /videos) -> persisted Source + MediaItem JSON stores; pure classify/dedup logic in indexerCore.ts - G: Windows-safe folder paths + dir sanitizer (collectionOutputTemplate) - H: Library tab + source tree + "Download pending" into the existing queue - I: state-preserving re-index merge + persist-downloaded-on-complete - J: watched sources, RSS fast-check, Task Scheduler + --sync launch (the OS-level scheduling/RSS need a real-install smoke test) - K: .info.json / thumbnail / .description sidecars for media servers Also gitignore .gitea-token. 106 unit tests pass; typecheck + build clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -7,3 +7,6 @@ dist
|
||||
|
||||
# Local screen-capture frames (not source)
|
||||
.frames/
|
||||
|
||||
# Secrets — never commit
|
||||
.gitea-token
|
||||
|
||||
@@ -0,0 +1,235 @@
|
||||
# AeroFetch Roadmap — media-manager mode (Pinchflat parity)
|
||||
|
||||
The [Seal-parity roadmap](ROADMAP.md) took AeroFetch from "paste a URL → download" to a
|
||||
full-featured yt-dlp frontend (Phases A–E, complete). This document is a **second, parallel
|
||||
track**: turning AeroFetch from a one-shot *downloader* into a *media manager* in the spirit
|
||||
of [Pinchflat](https://github.com/kieraneglin/pinchflat) — download **entire channels**,
|
||||
keep them **organized into `Channel / Playlist / Title` folders**, and (eventually) **keep
|
||||
them in sync** as the channel posts new videos.
|
||||
|
||||
Nothing here removes or replaces existing behaviour. The single-video / hand-picked-playlist
|
||||
flow (probe → format pick → queue → history) stays exactly as-is and remains the default. The
|
||||
media-manager features are **additive**: a new "Library" section feeds the *same* download
|
||||
queue, concurrency cap, history, error log, cookies, and post-processing options already built.
|
||||
|
||||
Sources: [Pinchflat GitHub](https://github.com/kieraneglin/pinchflat) ·
|
||||
[Pinchflat architecture (DeepWiki)](https://deepwiki.com/kieraneglin/pinchflat) ·
|
||||
[Pinchflat FAQ — indexing](https://github.com/kieraneglin/pinchflat/wiki/Frequently-Asked-Questions).
|
||||
|
||||
---
|
||||
|
||||
## Implementation status — Phases F–K shipped (2026-06-23)
|
||||
|
||||
All six phases are implemented and **rebuild-gated** (`npm run typecheck` + `npm run test`
|
||||
+ `npm run build` clean after each). The unit suite grew from 76 → **106 tests** (new pure
|
||||
logic in `indexerCore.ts` + the folder/sidecar args). UI phases were additionally verified
|
||||
in the Vite browser preview.
|
||||
|
||||
| Phase | What shipped | Verification |
|
||||
| --- | --- | --- |
|
||||
| **F** | `classifySource`/`buildMediaItems`/`stableSourceId` (`indexerCore.ts`); channel-walk + persist (`indexer.ts`, `sources.ts`); IPC + preload | typecheck · 88 tests · build |
|
||||
| **G** | `sanitizeDirSegment` + `collectionOutputTemplate` (`buildArgs.ts`); `CollectionContext` threaded through `buildCommand` | typecheck · 97 tests · build |
|
||||
| **H** | Library sidebar tab + `LibraryView.tsx` + `store/sources.ts`; grouped tree, "Download N pending" → existing queue, live status pills | **preview-verified** |
|
||||
| **I** | `mergeItemsPreservingState` (re-index keeps downloaded state, drops vanished, counts new); persist-on-complete via `markDownloaded` | **preview-verified** (state survives a queue clear) |
|
||||
| **J** | Watched sources + RSS fast-check (`sync.ts`), per-source Watch toggle, "Check for new", `autoDownloadNew`; Task Scheduler (`schedule.ts`) + `--sync` launch | UI preview-verified; **OS-level scheduling/RSS fenced — needs a real-install smoke test** (same caveat as the `aerofetch://` wiring in ROADMAP.md) |
|
||||
| **K** | `.info.json` / thumbnail / `.description` sidecars via `DownloadOptions` + `DownloadOptionsForm` "Sidecar files" group | typecheck · 106 tests · build + preview-verified |
|
||||
|
||||
**Deferred sub-items (noted, not yet built):** the power-user raw output-path template
|
||||
setting (G bullet 3 — current layout is the fixed `Channel/Playlist/NNN - Title`); an
|
||||
automatic incremental feeder so the live queue self-refills (H — today "Download pending"
|
||||
enqueues a capped batch of `MAX_ENQUEUE_BATCH`); the retention / quality-upgrade re-download
|
||||
(I stretch); a true headless quit-when-done for the scheduled `--sync` run (J — today it
|
||||
launches the normal window unobtrusively and syncs on startup); and Kodi/Jellyfin `.nfo`
|
||||
generation (K — the three native yt-dlp sidecars ship; `.nfo` needs a custom post-step).
|
||||
|
||||
---
|
||||
|
||||
## The core idea borrowed from Pinchflat
|
||||
|
||||
Pinchflat's insight is to **separate indexing from downloading**, with a persisted index in
|
||||
between:
|
||||
|
||||
1. **Index** a *Source* (a channel or playlist) — enumerate *all* its videos with
|
||||
`yt-dlp --flat-playlist` and persist one lightweight record per video, **regardless of any
|
||||
filters**. Changing what you want to download never forces a re-index.
|
||||
2. **Filter** that persisted list to decide what to actually fetch.
|
||||
3. **Download** each chosen video as its own per-item yt-dlp run, placed into folders by an
|
||||
output-path template, with `--download-archive` + a per-item `downloaded` flag preventing
|
||||
re-downloads.
|
||||
|
||||
This is why "download an entire channel" doesn't have to flood the queue with thousands of
|
||||
live cards (the failure mode of naïve per-video enumeration) and doesn't need a custom
|
||||
playlist-progress parser (the failure mode of native `yt-dlp` playlist expansion in one
|
||||
process): **the channel's full video list lives in a store; the live queue only holds what's
|
||||
actively downloading.** AeroFetch's per-video progress, concurrency, cancel, retry, and
|
||||
history all keep working unchanged.
|
||||
|
||||
### Data model — Pinchflat → AeroFetch
|
||||
|
||||
| Pinchflat (Elixir + SQLite + Oban) | AeroFetch (Electron + JSON store + existing queue) |
|
||||
| --- | --- |
|
||||
| **Source** (watched channel/playlist) | `Source` record in a new `sources.json` (mirrors `history.ts`) |
|
||||
| **Media Item** (one discovered video) | `MediaItem` record (id, title, playlist, index, source id, `downloaded`) |
|
||||
| **Media Profile** (download preset) | Maps onto existing `DownloadOptions` + `CommandTemplate` + a new output-path template |
|
||||
| Oban `media_collection_indexing` queue | An async **index job** in main (the probe, persisted) |
|
||||
| Oban `media_fetching` queue | The existing renderer download queue + concurrency cap |
|
||||
| Oban `fast_indexing` (RSS) | Phase J — YouTube RSS re-sync |
|
||||
| `--download-archive` + item state | The existing `--download-archive` setting + `MediaItem.downloaded` |
|
||||
|
||||
**Storage decision:** stay with plain-JSON stores (the pattern Phase D deliberately kept over
|
||||
better-sqlite3 to avoid a native-module build step). A single channel is up to a few thousand
|
||||
`MediaItem`s — still fine to hold and filter client-side. Revisit better-sqlite3 only if a
|
||||
user indexes many large channels and the JSON files get unwieldy (noted as a Phase H risk).
|
||||
|
||||
---
|
||||
|
||||
## Phase F — Channel & playlist indexing (foundation) ✅ COMPLETE
|
||||
|
||||
Make AeroFetch able to *enumerate and remember* a whole channel without downloading anything
|
||||
yet. This is the prerequisite for every later phase.
|
||||
|
||||
- [ ] **Channel-depth probe.** Extend `src/main/probe.ts` so a channel URL (`/@handle`,
|
||||
`/channel/<id>`, `/c/<name>`, `/user/<name>`) resolves to a new `kind: 'channel'`
|
||||
`ProbeResult`. Today `buildPlaylist` reads `data.entries` one level deep and ignores the
|
||||
nesting a channel returns (the channel's tabs/playlists). Walk channel → playlists →
|
||||
videos: probe `…/playlists` (flat) for the playlist list, plus a synthetic **"Uploads"**
|
||||
playlist from `…/videos` for videos that belong to no playlist. Lazy-probe each
|
||||
playlist's video list on demand rather than all up front.
|
||||
- [ ] **Persisted index.** New `src/main/sources.ts` (plain JSON, mirrors `src/main/history.ts`)
|
||||
storing `Source` + `MediaItem` records. New IPC types in `src/shared/ipc.ts`:
|
||||
```ts
|
||||
interface Source {
|
||||
id: string
|
||||
url: string
|
||||
kind: 'channel' | 'playlist'
|
||||
title: string
|
||||
channel?: string
|
||||
addedAt: number
|
||||
lastIndexedAt?: number
|
||||
}
|
||||
interface MediaItem {
|
||||
id: string // yt-dlp video id — the dedup key
|
||||
sourceId: string
|
||||
title: string
|
||||
playlistTitle?: string // for the folder path; 'Uploads' fallback
|
||||
playlistIndex?: number // 1-based, for NNN numbering
|
||||
durationLabel?: string
|
||||
downloaded: boolean
|
||||
downloadedAt?: number
|
||||
filePath?: string
|
||||
}
|
||||
```
|
||||
- [ ] **Dedup on index.** A video appearing in multiple playlists collapses to one `MediaItem`
|
||||
(keyed by video id); first playlist seen wins the folder assignment (surface this choice
|
||||
in the UI so the user can reassign). The "Uploads" synthetic playlist catches anything in
|
||||
no real playlist.
|
||||
- [ ] **Index job.** Async indexing in main (not blocking the UI), pushing progress over a new
|
||||
IPC channel (`index:progress`) the way `download.ts` pushes download events — "indexed
|
||||
N of M playlists." Reuse `cleanError` and the same `assertHttpUrl` guard.
|
||||
|
||||
## Phase G — Folder organization & output-path templates ✅ COMPLETE
|
||||
|
||||
Turn a `MediaItem` into a real per-item download that lands in `Channel / Playlist / Title`.
|
||||
|
||||
- [ ] **Per-item output subpath.** `StartDownloadOptions.outputDir` is *already* plumbed through
|
||||
`buildCommand` (`src/main/download.ts`). Add an `outputSubdir` (or reuse `outputDir` with a
|
||||
joined subpath) so each `MediaItem` downloads into
|
||||
`<root>/<Channel>/<Playlist>/<NNN> - <Title>.ext`. The `NNN` index comes from
|
||||
`MediaItem.playlistIndex` (computed at index time) — **not** `%(playlist_index)s`, which is
|
||||
empty under the per-item `--no-playlist` path that AeroFetch keeps using.
|
||||
- [ ] **Directory sanitizer.** A small helper in `buildArgs.ts` that strips Windows-illegal
|
||||
directory chars (`< > : " / \ | ? *`, trailing dots/spaces, reserved names like `CON`).
|
||||
Critical because `--restrict-filenames` only sanitizes the *filename* yt-dlp generates,
|
||||
not the directory segments AeroFetch constructs from channel/playlist names.
|
||||
- [ ] **Output-path setting.** A new Settings → Downloads field for the folder layout, defaulting
|
||||
to `Channel / Playlist / Title`, with the raw yt-dlp `-o` template exposed for power users
|
||||
(parallels how Phase C exposed raw extra args). Keeps the existing flat
|
||||
`filenameTemplate` as the default for non-collection downloads.
|
||||
- [ ] **"Media Profile" = named download preset (optional sugar).** Pinchflat bundles
|
||||
quality + subs + output template into a reusable Profile. AeroFetch already persists
|
||||
`DownloadOptions` and named `CommandTemplate`s; a Profile is just a named bundle of
|
||||
`{ DownloadOptions, outputTemplate, extraArgs }` a Source can point at. Worth it once
|
||||
multiple sources want different rules; skip for v1 (use the global defaults).
|
||||
|
||||
## Phase H — Library view (the media-manager UI) ✅ COMPLETE
|
||||
|
||||
The new surface where channels live. Everything below feeds the **existing** download queue.
|
||||
|
||||
- [ ] **"Library" sidebar section** (`src/renderer/src/components/Sidebar.tsx`) alongside
|
||||
Downloads / History / Settings. Lists added Sources.
|
||||
- [ ] **Source detail — playlist/video tree.** A collapsible `Channel → Playlist → Video` tree
|
||||
showing each `MediaItem`'s state (indexed · pending · downloading · downloaded · error),
|
||||
reusing the checkbox-selection pattern already in `DownloadBar.tsx`'s playlist panel.
|
||||
Per-playlist select-all, live counts.
|
||||
- [ ] **"Download all pending" → existing queue, batched.** Enqueue selected `MediaItem`s via
|
||||
the existing `addFromUrl` path, **a batch at a time** (e.g. 50) so the queue/store never
|
||||
holds the entire channel at once — the live queue stays small while the Library view is the
|
||||
source of truth for the full list. Completion flips `MediaItem.downloaded = true` and
|
||||
records to history exactly as today.
|
||||
- [ ] **New `useSources` store** (`src/renderer/src/store/sources.ts`, mirrors
|
||||
`store/downloads.ts` / `store/history.ts`) plus its IPC bridge. Browser-preview seed data
|
||||
so the view is demoable without Electron (same convention as `downloads.ts`'s `PREVIEW`).
|
||||
- [ ] **Risk to watch:** rendering a 2,000-row tree. Virtualize the list (or cap + paginate) and,
|
||||
if JSON-store load times bite, revisit the Phase-D better-sqlite3 decision.
|
||||
|
||||
## Phase I — Incremental sync & dedup ✅ COMPLETE
|
||||
|
||||
Re-running a channel should grab only what's new — the everyday media-manager loop.
|
||||
|
||||
- [ ] **Re-index = diff.** Re-indexing a Source compares freshly enumerated video ids against the
|
||||
persisted `MediaItem`s and marks only the new ones; existing records (and their
|
||||
`downloaded` state) are preserved. "X new since last sync" badge.
|
||||
- [ ] **"Download new only."** One action that enqueues just the un-downloaded `MediaItem`s,
|
||||
backed by the existing **`--download-archive`** setting (`Settings.downloadArchive`,
|
||||
`getDownloadArchivePath()`) as a second, yt-dlp-level dedup guard so even a stale index
|
||||
can't re-download.
|
||||
- [ ] **Retention / quality-upgrade (stretch, from Pinchflat).** Optional re-download to upgrade
|
||||
quality/metadata, and optional pruning of old items. Defer — niche for a desktop app.
|
||||
|
||||
## Phase J — Subscriptions & scheduled auto-download ✅ COMPLETE (OS wiring needs smoke test)
|
||||
|
||||
Pinchflat's headline feature: a Source you *watch*, that downloads new uploads on its own.
|
||||
AeroFetch already has the Windows pieces for this elsewhere in this workspace (Task Scheduler +
|
||||
single-instance Mutex + a headless run mode — see the DashMail/Weather Radar patterns).
|
||||
|
||||
- [ ] **Watched sources.** A `Source.watched` flag + an "auto-download new" toggle per source.
|
||||
- [ ] **Fast indexing via RSS.** YouTube exposes a per-channel Atom feed
|
||||
(`https://www.youtube.com/feeds/videos.xml?channel_id=<id>`) — a cheap check for new
|
||||
uploads without a full yt-dlp scan. **Caveat:** the feed only carries the latest ~15
|
||||
videos, so RSS is for *staying current*, not initial indexing (which stays a full
|
||||
Phase-F scan). Pinchflat draws the same line ("fast indexing is not recommended for
|
||||
playlists / initial scans").
|
||||
- [ ] **Scheduled headless run.** A `--sync` CLI entry that indexes all watched sources and
|
||||
enqueues new items, wired to **Windows Task Scheduler** (reuse the
|
||||
single-instance-lock already added in Phase E so a scheduled run hands off to a running
|
||||
window instead of double-launching). Notify on new downloads via the existing
|
||||
`Notification` path.
|
||||
|
||||
## Phase K — Media-server output (optional stretch) ✅ COMPLETE
|
||||
|
||||
Make AeroFetch's output drop-in for Jellyfin/Plex/Kodi the way Pinchflat does — relevant
|
||||
because organized channel folders are exactly what those servers ingest.
|
||||
|
||||
- [ ] **Sidecar metadata.** `--write-info-json`, `--write-thumbnail`, `--write-description`,
|
||||
and optional Kodi/Jellyfin `.nfo` generation. Most of this is already expressible through
|
||||
Phase C custom-command args; a checkbox group in the output settings makes it first-class.
|
||||
|
||||
---
|
||||
|
||||
## What we reuse vs. build
|
||||
|
||||
**Reuse unchanged:** the download queue + concurrency cap + cancel/retry (`store/downloads.ts`),
|
||||
history + error log, cookies + proxy + rate-limit + aria2c (`AccessOptions`), all
|
||||
post-processing `DownloadOptions`, `--download-archive`, native notifications, the
|
||||
single-instance lock, and the JSON-store pattern.
|
||||
|
||||
**Build new:** channel-depth probing (Phase F), the `sources.json` index + `Source`/`MediaItem`
|
||||
model (F), per-item folder output paths + sanitizer (G), the Library view + `useSources` store
|
||||
(H), incremental diff-sync (I), and RSS + scheduled sync (J).
|
||||
|
||||
## Suggested order
|
||||
|
||||
F → G are the foundation and unlock a usable "download whole channel into folders" the moment
|
||||
they land (drive it from a temporary button before the full Library UI exists). H makes it a
|
||||
real product surface. I and J turn it into a true media manager. K is polish. Phases F–H are
|
||||
the high-value core; I–K are incremental and each shippable on its own.
|
||||
@@ -9,6 +9,11 @@ Sources: [Seal GitHub](https://github.com/JunkFood02/Seal) ·
|
||||
[Seal on F-Droid](https://f-droid.org/packages/com.junkfood.seal/) ·
|
||||
[It's FOSS overview](https://itsfoss.com/news/seal/).
|
||||
|
||||
> **See also:** [ROADMAP-PINCHFLAT.md](ROADMAP-PINCHFLAT.md) — a parallel track that extends
|
||||
> AeroFetch from a one-shot downloader into a [Pinchflat](https://github.com/kieraneglin/pinchflat)-style
|
||||
> media manager (index entire channels → organize into `Channel / Playlist / Title` folders →
|
||||
> keep in sync), reusing the existing queue/history/options rather than replacing them.
|
||||
|
||||
---
|
||||
|
||||
## Already at parity ✅
|
||||
|
||||
@@ -8,10 +8,12 @@
|
||||
* resolving it itself; the caller in download.ts passes `getBinDir()`.
|
||||
*/
|
||||
|
||||
import { join } from 'path'
|
||||
import {
|
||||
BEST_FORMAT_ID,
|
||||
type StartDownloadOptions,
|
||||
type DownloadOptions,
|
||||
type CollectionContext,
|
||||
type CookieBrowser
|
||||
} from '@shared/ipc'
|
||||
|
||||
@@ -138,6 +140,52 @@ export function formatCommandLine(exe: string, args: string[]): string {
|
||||
return [exe, ...args].map(quoteForDisplay).join(' ')
|
||||
}
|
||||
|
||||
// --- Collection (media-manager) folder paths --------------------------------
|
||||
|
||||
/**
|
||||
* Sanitize one path segment (a channel or playlist name) into a Windows-safe
|
||||
* directory name. This matters because, unlike the filename yt-dlp itself writes
|
||||
* (which `--restrict-filenames` can clean), these directory segments are built by
|
||||
* AeroFetch from untrusted channel/playlist titles and joined onto the output
|
||||
* dir — so they must be neutered for both illegal characters AND path traversal.
|
||||
*
|
||||
* - illegal chars (`< > : " / \ | ? *`) and control chars → space
|
||||
* - leading/trailing dots and spaces stripped (illegal / invisible on Windows),
|
||||
* which also turns a bare `..` traversal segment into nothing
|
||||
* - reserved device names (CON, PRN, NUL, COM1…) get an underscore prefix
|
||||
* - length-capped so the full path stays well under MAX_PATH
|
||||
* - empty result falls back to 'Untitled'
|
||||
*/
|
||||
export function sanitizeDirSegment(name: string): string {
|
||||
// C0 control chars (charCode < 0x20) and Windows-illegal chars both become a
|
||||
// space. The control-char filter is done by char code so no literal control
|
||||
// byte ever appears in this source file.
|
||||
let s = Array.from(name ?? '', (ch) => (ch.charCodeAt(0) < 0x20 ? ' ' : ch)).join('')
|
||||
s = s.replace(/[<>:"/\\|?*]/g, ' ')
|
||||
s = s.replace(/\s+/g, ' ').trim().replace(/[. ]+$/, '').replace(/^[. ]+/, '')
|
||||
if (/^(con|prn|aux|nul|com[1-9]|lpt[1-9])$/i.test(s)) s = `_${s}`
|
||||
s = s.slice(0, 80).trim()
|
||||
return s || 'Untitled'
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the `-o` output template for a collection download:
|
||||
* <outDir>/<Channel>/<Playlist>/<NNN> - <baseFilename>
|
||||
* where NNN is the 1-based playlist index zero-padded to three digits and
|
||||
* baseFilename keeps its yt-dlp field tokens (e.g. '%(title)s.%(ext)s') so they
|
||||
* still expand. Channel/playlist are sanitized via sanitizeDirSegment.
|
||||
*/
|
||||
export function collectionOutputTemplate(
|
||||
outDir: string,
|
||||
c: CollectionContext,
|
||||
baseFilename: string
|
||||
): string {
|
||||
const n = Number.isFinite(c.index) && c.index > 0 ? Math.floor(c.index) : 1
|
||||
const nnn = String(n).padStart(3, '0')
|
||||
const segments = [sanitizeDirSegment(c.channel), sanitizeDirSegment(c.playlist)]
|
||||
return join(outDir, ...segments, `${nnn} - ${baseFilename}`)
|
||||
}
|
||||
|
||||
function accessArgs(a: AccessOptions): string[] {
|
||||
const args: string[] = []
|
||||
if (a.proxy.trim()) args.push('--proxy', a.proxy.trim())
|
||||
@@ -179,6 +227,12 @@ function postProcessArgs(opts: StartDownloadOptions, o: DownloadOptions): string
|
||||
if (o.embedChapters) args.push('--embed-chapters')
|
||||
if (o.embedMetadata) args.push('--embed-metadata')
|
||||
|
||||
// Media-server sidecar files (Phase K) — written next to the output file so
|
||||
// Jellyfin/Plex/Kodi can ingest metadata, poster art, and the description.
|
||||
if (o.writeInfoJson) args.push('--write-info-json')
|
||||
if (o.writeThumbnailFile) args.push('--write-thumbnail')
|
||||
if (o.writeDescription) args.push('--write-description')
|
||||
|
||||
return args
|
||||
}
|
||||
|
||||
|
||||
+13
-3
@@ -7,7 +7,12 @@ import { getSettings, getDownloadArchivePath } from './settings'
|
||||
import { getCookiesFilePath } from './cookies'
|
||||
import { listTemplates } from './templates'
|
||||
import { assertHttpUrl } from './url'
|
||||
import { buildArgs, parseExtraArgs, formatCommandLine } from './buildArgs'
|
||||
import {
|
||||
buildArgs,
|
||||
parseExtraArgs,
|
||||
formatCommandLine,
|
||||
collectionOutputTemplate
|
||||
} from './buildArgs'
|
||||
import { cleanError } from './log'
|
||||
import { addErrorLog } from './errorlog'
|
||||
import {
|
||||
@@ -161,7 +166,12 @@ function resolveExtraArgs(opts: StartDownloadOptions, settings: Settings): strin
|
||||
export function buildCommand(opts: StartDownloadOptions): string[] {
|
||||
const settings = getSettings()
|
||||
const outDir = opts.outputDir?.trim() || settings.outputDir || app.getPath('downloads')
|
||||
const template = settings.filenameTemplate?.trim() || '%(title)s.%(ext)s'
|
||||
// A collection (media-manager) download is filed into <channel>/<playlist>/
|
||||
// <NNN> - <title> folders; an ordinary download uses the flat filenameTemplate.
|
||||
const filenameTemplate = settings.filenameTemplate?.trim() || '%(title)s.%(ext)s'
|
||||
const outputTemplate = opts.collection
|
||||
? collectionOutputTemplate(outDir, opts.collection, '%(title)s.%(ext)s')
|
||||
: join(outDir, filenameTemplate)
|
||||
// Per-download override wins; otherwise use the persisted defaults.
|
||||
const options = opts.options ?? settings.downloadOptions
|
||||
// Silently fall back to yt-dlp's own downloader if aria2c.exe wasn't dropped
|
||||
@@ -183,7 +193,7 @@ export function buildCommand(opts: StartDownloadOptions): string[] {
|
||||
downloadArchivePath: settings.downloadArchive ? getDownloadArchivePath() : undefined
|
||||
}
|
||||
const extraArgs = resolveExtraArgs(opts, settings)
|
||||
return buildArgs(opts, join(outDir, template), options, getBinDir(), access, extraArgs)
|
||||
return buildArgs(opts, outputTemplate, options, getBinDir(), access, extraArgs)
|
||||
}
|
||||
|
||||
/** Build the exact command line for the current form state, without running it. */
|
||||
|
||||
+47
-1
@@ -22,6 +22,17 @@ import { openCookieLoginWindow, getCookiesStatus, clearCookies } from './cookies
|
||||
import { listErrorLog, addErrorLog, clearErrorLog } from './errorlog'
|
||||
import { exportBackup, importBackup } from './backup'
|
||||
import { extractIncomingUrl, registerSendToShortcut, focusWindow } from './deeplink'
|
||||
import {
|
||||
listSources,
|
||||
getSource,
|
||||
removeSource,
|
||||
listMediaItems,
|
||||
setMediaItemDownloaded,
|
||||
setSourceWatched
|
||||
} from './sources'
|
||||
import { indexSource } from './indexer'
|
||||
import { syncWatchedSources } from './sync'
|
||||
import { getScheduledSync, setScheduledSync, isSyncLaunch } from './schedule'
|
||||
|
||||
// Only one instance ever runs. A second launch — e.g. the OS invoking us again
|
||||
// for an aerofetch:// link or a "Send to AeroFetch" file — hands its argv to
|
||||
@@ -96,7 +107,10 @@ function createWindow(): void {
|
||||
mainWindow = win
|
||||
|
||||
win.on('ready-to-show', () => {
|
||||
win.show()
|
||||
// A scheduled `--sync` launch starts unobtrusively (shown but not focused) so
|
||||
// the daily background sync doesn't steal focus; a normal launch shows + focuses.
|
||||
if (isSyncLaunch(process.argv)) win.showInactive()
|
||||
else win.show()
|
||||
})
|
||||
|
||||
win.on('closed', () => {
|
||||
@@ -229,6 +243,38 @@ function registerIpcHandlers(): void {
|
||||
}
|
||||
return result
|
||||
})
|
||||
|
||||
// --- Media-manager sources (Pinchflat-style index; see ROADMAP-PINCHFLAT.md) ---
|
||||
ipcMain.handle(IpcChannels.sourcesList, () => listSources())
|
||||
ipcMain.handle(IpcChannels.sourceItems, (_e, sourceId: string) => listMediaItems(sourceId))
|
||||
ipcMain.handle(IpcChannels.sourceRemove, (_e, id: string) => removeSource(id))
|
||||
ipcMain.handle(IpcChannels.sourceItemDownloaded, (_e, id: string, filePath?: string) =>
|
||||
setMediaItemDownloaded(id, filePath)
|
||||
)
|
||||
// Indexing pushes live progress to the requesting renderer over `indexProgress`
|
||||
// and resolves with the final result.
|
||||
ipcMain.handle(IpcChannels.sourceIndex, (e, url: string) =>
|
||||
indexSource(url, (p) => {
|
||||
if (!e.sender.isDestroyed()) e.sender.send(IpcChannels.indexProgress, p)
|
||||
})
|
||||
)
|
||||
ipcMain.handle(IpcChannels.sourceReindex, (e, id: string) => {
|
||||
const src = getSource(id)
|
||||
if (!src) return { ok: false, error: 'Source not found.' }
|
||||
return indexSource(src.url, (p) => {
|
||||
if (!e.sender.isDestroyed()) e.sender.send(IpcChannels.indexProgress, p)
|
||||
})
|
||||
})
|
||||
ipcMain.handle(IpcChannels.sourceSetWatched, (_e, id: string, watched: boolean) =>
|
||||
setSourceWatched(id, watched)
|
||||
)
|
||||
ipcMain.handle(IpcChannels.sourcesSync, (e) =>
|
||||
syncWatchedSources((p) => {
|
||||
if (!e.sender.isDestroyed()) e.sender.send(IpcChannels.indexProgress, p)
|
||||
})
|
||||
)
|
||||
ipcMain.handle(IpcChannels.scheduledSyncGet, () => getScheduledSync())
|
||||
ipcMain.handle(IpcChannels.scheduledSyncSet, (_e, enabled: boolean) => setScheduledSync(enabled))
|
||||
}
|
||||
|
||||
// Push OS theme/contrast changes to every window, and keep the native
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
/**
|
||||
* Source indexing orchestration (Pinchflat-style media manager; see
|
||||
* ROADMAP-PINCHFLAT.md). Walks a channel (its /playlists + /videos tabs) or a
|
||||
* single playlist with `yt-dlp --flat-playlist`, merges the result into a deduped
|
||||
* MediaItem list, and persists it as a Source. The download step (Phase G+) then
|
||||
* pulls from that persisted list rather than the live queue holding it all.
|
||||
*
|
||||
* The pure URL-classification + merge logic lives in indexerCore.ts (unit-tested);
|
||||
* this module is the impure shell that spawns yt-dlp and writes to disk.
|
||||
*/
|
||||
|
||||
import { execFile } from 'child_process'
|
||||
import { existsSync } from 'fs'
|
||||
import { getYtdlpPath } from './binaries'
|
||||
import { cleanError } from './log'
|
||||
import { assertHttpUrl } from './url'
|
||||
import {
|
||||
classifySource,
|
||||
buildMediaItems,
|
||||
buildFeedUrl,
|
||||
entryUrl,
|
||||
stripTabSuffix,
|
||||
stableSourceId,
|
||||
type RawEntry,
|
||||
type NamedPlaylist
|
||||
} from './indexerCore'
|
||||
import { getSource, upsertSource, mergeMediaItems } from './sources'
|
||||
import type { IndexProgress, IndexSourceResult, Source, SourceKind } from '@shared/ipc'
|
||||
|
||||
/** The slice of `yt-dlp -J --flat-playlist` output the indexer reads. */
|
||||
interface FlatInfo {
|
||||
_type?: string
|
||||
id?: string
|
||||
title?: string
|
||||
uploader?: string
|
||||
channel?: string
|
||||
/** the UC… channel id, used to build the RSS feed URL */
|
||||
channel_id?: string
|
||||
entries?: RawEntry[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `yt-dlp -J --flat-playlist` on a URL and parse the JSON. Rejects on a
|
||||
* non-zero exit or unparseable output. maxBuffer is large because a big channel's
|
||||
* flat upload list can be a few MB of JSON; the timeout is generous for the same
|
||||
* reason. `--` terminates option parsing so the URL can't be read as a flag.
|
||||
*/
|
||||
function probeFlat(url: string): Promise<FlatInfo> {
|
||||
return new Promise((resolve, reject) => {
|
||||
execFile(
|
||||
getYtdlpPath(),
|
||||
['-J', '--flat-playlist', '--no-warnings', '--', url],
|
||||
{ windowsHide: true, maxBuffer: 256 * 1024 * 1024, timeout: 180_000 },
|
||||
(err, stdout, stderr) => {
|
||||
if (err) {
|
||||
const msg = (err as { killed?: boolean }).killed
|
||||
? 'Timed out indexing source. Check the link or your connection.'
|
||||
: cleanError(stderr) || err.message
|
||||
reject(new Error(msg))
|
||||
return
|
||||
}
|
||||
try {
|
||||
resolve(JSON.parse(stdout) as FlatInfo)
|
||||
} catch {
|
||||
reject(new Error('Could not parse source info from yt-dlp.'))
|
||||
}
|
||||
}
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
/** Probe a channel tab (e.g. /videos, /playlists); never throws — returns null. */
|
||||
function probeTab(url: string): Promise<FlatInfo | null> {
|
||||
return probeFlat(url).catch(() => null)
|
||||
}
|
||||
|
||||
/**
|
||||
* Index (or re-index) a Source. Resolves to the persisted Source + item count, or
|
||||
* an error. `onProgress` is invoked throughout so the renderer can show a live
|
||||
* status line — it's best-effort and never affects the result.
|
||||
*/
|
||||
export async function indexSource(
|
||||
url: string,
|
||||
onProgress: (p: IndexProgress) => void
|
||||
): Promise<IndexSourceResult> {
|
||||
try {
|
||||
assertHttpUrl(url)
|
||||
} catch (e) {
|
||||
return { ok: false, error: (e as Error).message }
|
||||
}
|
||||
if (!existsSync(getYtdlpPath())) {
|
||||
return { ok: false, error: 'yt-dlp.exe not found. Drop it into resources/bin/.' }
|
||||
}
|
||||
|
||||
const cls = classifySource(url)
|
||||
onProgress({ url, phase: 'start', message: 'Reading source…' })
|
||||
|
||||
try {
|
||||
let kind: SourceKind
|
||||
let title: string
|
||||
let channel: string | undefined
|
||||
let feedId: string | undefined
|
||||
const playlists: NamedPlaylist[] = []
|
||||
let uploads: RawEntry[] = []
|
||||
|
||||
if (cls?.kind === 'channel') {
|
||||
kind = 'channel'
|
||||
|
||||
// 1. Enumerate the channel's playlists, then each playlist's videos.
|
||||
onProgress({ url, phase: 'playlists', message: 'Finding playlists…' })
|
||||
const playlistTab = await probeTab(`${cls.base}/playlists`)
|
||||
const playlistEntries = playlistTab?.entries ?? []
|
||||
const total = playlistEntries.length
|
||||
let done = 0
|
||||
for (const pe of playlistEntries) {
|
||||
done++
|
||||
const purl = entryUrl(pe)
|
||||
if (!purl) continue
|
||||
onProgress({
|
||||
url,
|
||||
phase: 'playlist',
|
||||
message: `Indexing playlist ${done}/${total}: ${pe.title ?? ''}`.trim(),
|
||||
current: done,
|
||||
total
|
||||
})
|
||||
const pdata = await probeTab(purl)
|
||||
if (pdata?.entries?.length) {
|
||||
playlists.push({ title: pe.title || 'Playlist', entries: pdata.entries })
|
||||
}
|
||||
}
|
||||
|
||||
// 2. The full uploads feed — the catch-all for videos in no playlist.
|
||||
onProgress({ url, phase: 'uploads', message: 'Indexing channel uploads…' })
|
||||
const videoTab = await probeTab(`${cls.base}/videos`)
|
||||
uploads = videoTab?.entries ?? []
|
||||
|
||||
title =
|
||||
stripTabSuffix(videoTab?.channel) ||
|
||||
stripTabSuffix(videoTab?.title) ||
|
||||
stripTabSuffix(playlistTab?.title) ||
|
||||
cls.base
|
||||
channel = stripTabSuffix(videoTab?.channel || videoTab?.uploader) || title
|
||||
feedId = videoTab?.channel_id || playlistTab?.channel_id
|
||||
} else {
|
||||
// A single playlist (classified) — or an unclassified URL that might still
|
||||
// resolve to a playlist when probed. A lone video has no entries → error.
|
||||
kind = cls?.kind ?? 'playlist'
|
||||
onProgress({ url, phase: 'uploads', message: 'Indexing playlist…' })
|
||||
const data = await probeFlat(cls?.base ?? url)
|
||||
const entries = data.entries ?? []
|
||||
if (entries.length === 0) {
|
||||
return {
|
||||
ok: false,
|
||||
error: 'That link is a single video, not a channel or playlist. Use the download bar for one video.'
|
||||
}
|
||||
}
|
||||
title = data.title || 'Playlist'
|
||||
channel = data.uploader || data.channel
|
||||
// A playlist feed keys off the playlist id (PL…), carried as `id` here.
|
||||
feedId = data.id
|
||||
// File everything under the playlist's own name (not the 'Uploads' fallback).
|
||||
playlists.push({ title, entries })
|
||||
}
|
||||
|
||||
const sourceId = stableSourceId(cls?.base ?? url.trim())
|
||||
const fresh = buildMediaItems(sourceId, playlists, uploads)
|
||||
if (fresh.length === 0) {
|
||||
return { ok: false, error: 'No downloadable videos found for this source.' }
|
||||
}
|
||||
|
||||
// Incremental merge: preserve the downloaded state of anything already on
|
||||
// disk, and report how many videos are new since the last index.
|
||||
const { items, newCount } = mergeMediaItems(sourceId, fresh)
|
||||
|
||||
const prev = getSource(sourceId)
|
||||
const source: Source = {
|
||||
id: sourceId,
|
||||
url: url.trim(),
|
||||
kind,
|
||||
title,
|
||||
channel,
|
||||
addedAt: prev?.addedAt ?? Date.now(),
|
||||
lastIndexedAt: Date.now(),
|
||||
itemCount: items.length,
|
||||
// Preserve the watched flag across a re-index; refresh the RSS feed URL.
|
||||
watched: prev?.watched,
|
||||
feedUrl: buildFeedUrl(kind, feedId) ?? prev?.feedUrl
|
||||
}
|
||||
upsertSource(source)
|
||||
|
||||
const newNote = newCount === items.length ? '' : ` (${newCount} new)`
|
||||
onProgress({
|
||||
url,
|
||||
phase: 'done',
|
||||
message: `Indexed ${items.length} videos${newNote}.`,
|
||||
total: items.length
|
||||
})
|
||||
return { ok: true, source, itemCount: items.length, newCount }
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
onProgress({ url, phase: 'error', message: msg })
|
||||
return { ok: false, error: msg }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
/**
|
||||
* Pure, dependency-free core for Source indexing (Pinchflat-style media manager;
|
||||
* see ROADMAP-PINCHFLAT.md). Like buildArgs.ts / validation.ts this module imports
|
||||
* nothing from electron or the node runtime, so its URL-classification and
|
||||
* item-merge logic can be unit-tested without spinning up Electron.
|
||||
*
|
||||
* The impure orchestration (spawning yt-dlp, persisting to disk) lives in
|
||||
* indexer.ts, which composes these helpers.
|
||||
*/
|
||||
|
||||
import type { MediaItem, SourceKind } from '@shared/ipc'
|
||||
|
||||
// --- yt-dlp --flat-playlist entry shape (shared with probe.ts) --------------
|
||||
|
||||
/** The slice of a flat-playlist entry we read (a video, or a nested playlist). */
|
||||
export interface RawEntry {
|
||||
id?: string
|
||||
title?: string
|
||||
url?: string
|
||||
webpage_url?: string
|
||||
duration?: number
|
||||
uploader?: string
|
||||
channel?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the canonical URL for a flat-playlist entry. Prefer an explicit http(s)
|
||||
* URL; otherwise build a YouTube watch URL from the id (the common case where flat
|
||||
* entries carry only an id). Returns null when nothing usable is present.
|
||||
*
|
||||
* Note: the returned value is always either an http(s) URL or null, so it can
|
||||
* never begin with '-' and be mis-read as a yt-dlp option (callers also pass `--`).
|
||||
*/
|
||||
export function entryUrl(e: RawEntry): string | null {
|
||||
const cand = e.url || e.webpage_url
|
||||
if (cand && /^https?:\/\//i.test(cand)) return cand
|
||||
if (e.id) return `https://www.youtube.com/watch?v=${e.id}`
|
||||
return null
|
||||
}
|
||||
|
||||
/** Seconds → 'M:SS' or 'H:MM:SS'. Flat entries give duration as a number. */
|
||||
export function fmtDuration(sec?: number): string | undefined {
|
||||
if (sec == null || !Number.isFinite(sec)) return undefined
|
||||
const s = Math.max(0, Math.round(sec))
|
||||
const h = Math.floor(s / 3600)
|
||||
const m = Math.floor((s % 3600) / 60)
|
||||
const r = s % 60
|
||||
const mm = h ? String(m).padStart(2, '0') : String(m)
|
||||
return `${h ? `${h}:` : ''}${mm}:${String(r).padStart(2, '0')}`
|
||||
}
|
||||
|
||||
// --- Source-URL classification ----------------------------------------------
|
||||
|
||||
/** A classified Source URL: the kind, plus a normalised base for tab probing. */
|
||||
export interface SourceClass {
|
||||
kind: SourceKind
|
||||
/**
|
||||
* For 'channel': the channel root (e.g. https://www.youtube.com/@handle) onto
|
||||
* which '/videos' and '/playlists' tabs are appended. For 'playlist': the
|
||||
* canonical playlist URL.
|
||||
*/
|
||||
base: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Classify a pasted URL as a YouTube channel, a playlist, or neither.
|
||||
*
|
||||
* Channel forms: /@handle, /channel/<id>, /c/<name>, /user/<name> — any trailing
|
||||
* tab (/videos, /playlists, /streams, /shorts, /featured) is stripped to the root.
|
||||
* Playlist form: any URL carrying a `list=` query param. Returns null for a lone
|
||||
* video or a non-YouTube URL (Phase F scopes the channel walk to YouTube, whose
|
||||
* /videos + /playlists tab structure this relies on).
|
||||
*/
|
||||
export function classifySource(raw: string): SourceClass | null {
|
||||
let u: URL
|
||||
try {
|
||||
u = new URL((raw ?? '').trim())
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
if (u.protocol !== 'http:' && u.protocol !== 'https:') return null
|
||||
|
||||
const host = u.hostname.replace(/^www\./i, '').toLowerCase()
|
||||
const isYouTube = host === 'youtube.com' || host.endsWith('.youtube.com') || host === 'youtu.be'
|
||||
|
||||
// A playlist is identified purely by its list= param (works on any youtube host).
|
||||
const list = u.searchParams.get('list')
|
||||
if (isYouTube && list) {
|
||||
return { kind: 'playlist', base: `https://www.youtube.com/playlist?list=${encodeURIComponent(list)}` }
|
||||
}
|
||||
|
||||
if (!isYouTube) return null
|
||||
|
||||
// Channel roots. The handle/id segment is captured; later path segments
|
||||
// (the tab) are discarded so we always probe from the channel root.
|
||||
const m = u.pathname.match(/^\/(@[^/]+|channel\/[^/]+|c\/[^/]+|user\/[^/]+)/i)
|
||||
if (m) return { kind: 'channel', base: `https://www.youtube.com/${m[1]}` }
|
||||
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Channel tab titles from yt-dlp often look like "<Name> - Videos" or
|
||||
* "<Name> - Playlists". Strip a trailing known-tab suffix to recover the bare
|
||||
* channel name. Leaves anything else untouched.
|
||||
*/
|
||||
export function stripTabSuffix(title: string | undefined): string | undefined {
|
||||
if (!title) return title
|
||||
return title.replace(/\s[-–—]\s(Videos|Playlists|Shorts|Live|Streams|Home|Featured)$/i, '').trim()
|
||||
}
|
||||
|
||||
/**
|
||||
* Deterministic 32-bit FNV-1a hash → 8 hex chars. Used to derive a stable Source
|
||||
* id from its normalised base URL so re-indexing the same channel updates the
|
||||
* existing record instead of creating a duplicate. Pure (no crypto import).
|
||||
*/
|
||||
export function stableSourceId(base: string): string {
|
||||
let h = 0x811c9dc5
|
||||
for (let i = 0; i < base.length; i++) {
|
||||
h ^= base.charCodeAt(i)
|
||||
h = Math.imul(h, 0x01000193)
|
||||
}
|
||||
return (h >>> 0).toString(16).padStart(8, '0')
|
||||
}
|
||||
|
||||
// --- RSS fast-check (Phase J: watched sources) ------------------------------
|
||||
|
||||
/**
|
||||
* Build a YouTube RSS feed URL for cheap "anything new?" polling of a watched
|
||||
* source. Channels feed by `channel_id` (UC…), playlists by `playlist_id` (PL…).
|
||||
* Returns undefined when the id is missing (the sync then falls back to a full
|
||||
* re-index). The feed only carries the latest ~15 uploads, so it's a freshness
|
||||
* check, not a substitute for the initial full index.
|
||||
*/
|
||||
export function buildFeedUrl(kind: SourceKind, ytId: string | undefined): string | undefined {
|
||||
if (!ytId) return undefined
|
||||
const param = kind === 'channel' ? 'channel_id' : 'playlist_id'
|
||||
return `https://www.youtube.com/feeds/videos.xml?${param}=${encodeURIComponent(ytId)}`
|
||||
}
|
||||
|
||||
/** Extract the video ids from a YouTube RSS/Atom feed body (the <yt:videoId> tags). */
|
||||
export function parseRssVideoIds(xml: string): string[] {
|
||||
const ids: string[] = []
|
||||
const re = /<yt:videoId>\s*([\w-]+)\s*<\/yt:videoId>/g
|
||||
let m: RegExpExecArray | null
|
||||
while ((m = re.exec(xml)) !== null) ids.push(m[1])
|
||||
return ids
|
||||
}
|
||||
|
||||
// --- Entry merge / dedup ----------------------------------------------------
|
||||
|
||||
/** A named playlist and its (flat) video entries, ready to merge. */
|
||||
export interface NamedPlaylist {
|
||||
title: string
|
||||
entries: RawEntry[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge a Source's playlists (and its catch-all uploads) into a deduped list of
|
||||
* MediaItem records. Dedup is by video id: the FIRST playlist a video appears in
|
||||
* wins its folder assignment, so videos grouped into a real playlist land there,
|
||||
* and only videos in no playlist fall through to the synthetic 'Uploads' folder.
|
||||
*
|
||||
* `playlistIndex` is the 1-based position WITHIN the winning playlist (not the
|
||||
* global upload order), so the on-disk "NNN - Title" numbering matches the
|
||||
* playlist the file is filed under.
|
||||
*/
|
||||
export function buildMediaItems(
|
||||
sourceId: string,
|
||||
playlists: NamedPlaylist[],
|
||||
uploads: RawEntry[]
|
||||
): MediaItem[] {
|
||||
const byVideo = new Map<string, MediaItem>()
|
||||
|
||||
const add = (e: RawEntry, playlistTitle: string, index: number): void => {
|
||||
const videoId = e.id
|
||||
if (!videoId) return
|
||||
if (byVideo.has(videoId)) return // first playlist wins
|
||||
const url = entryUrl(e)
|
||||
if (!url) return // not turnable into a downloadable URL
|
||||
byVideo.set(videoId, {
|
||||
id: `${sourceId}:${videoId}`,
|
||||
sourceId,
|
||||
videoId,
|
||||
title: e.title || `Video ${videoId}`,
|
||||
url,
|
||||
playlistTitle,
|
||||
playlistIndex: index,
|
||||
durationLabel: fmtDuration(e.duration),
|
||||
downloaded: false
|
||||
})
|
||||
}
|
||||
|
||||
for (const p of playlists) p.entries.forEach((e, i) => add(e, p.title, i + 1))
|
||||
uploads.forEach((e, i) => add(e, 'Uploads', i + 1))
|
||||
|
||||
return [...byVideo.values()]
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge a freshly-enumerated item list with the previously-persisted one for the
|
||||
* same source (incremental re-index; see ROADMAP-PINCHFLAT.md Phase I). The fresh
|
||||
* list defines the current membership/ordering, but any video already marked
|
||||
* downloaded keeps its `downloaded`/`downloadedAt`/`filePath` so a re-sync never
|
||||
* loses that state or re-downloads it. Videos that vanished from the source (now
|
||||
* private/deleted) are dropped. `newCount` is how many fresh videos weren't in
|
||||
* the previous set — the "X new since last sync" figure.
|
||||
*/
|
||||
export function mergeItemsPreservingState(
|
||||
existing: MediaItem[],
|
||||
fresh: MediaItem[]
|
||||
): { items: MediaItem[]; newCount: number } {
|
||||
const prevByVideo = new Map(existing.map((m) => [m.videoId, m]))
|
||||
let newCount = 0
|
||||
const items = fresh.map((f) => {
|
||||
const prev = prevByVideo.get(f.videoId)
|
||||
if (!prev) {
|
||||
newCount++
|
||||
return f
|
||||
}
|
||||
return prev.downloaded
|
||||
? { ...f, downloaded: true, downloadedAt: prev.downloadedAt, filePath: prev.filePath }
|
||||
: f
|
||||
})
|
||||
return { items, newCount }
|
||||
}
|
||||
+1
-33
@@ -4,6 +4,7 @@ import { getYtdlpPath } from './binaries'
|
||||
import { fmtBytes } from './download'
|
||||
import { cleanError } from './log'
|
||||
import { assertHttpUrl } from './url'
|
||||
import { entryUrl, fmtDuration, type RawEntry } from './indexerCore'
|
||||
import {
|
||||
BEST_FORMAT_ID,
|
||||
type ProbeResult,
|
||||
@@ -26,16 +27,6 @@ interface RawFormat {
|
||||
filesize_approx?: number
|
||||
}
|
||||
|
||||
interface RawEntry {
|
||||
id?: string
|
||||
title?: string
|
||||
url?: string
|
||||
webpage_url?: string
|
||||
duration?: number
|
||||
uploader?: string
|
||||
channel?: string
|
||||
}
|
||||
|
||||
interface RawInfo {
|
||||
_type?: string
|
||||
title?: string
|
||||
@@ -92,29 +83,6 @@ function buildInfo(data: RawInfo): MediaInfo {
|
||||
}
|
||||
}
|
||||
|
||||
/** Seconds → 'M:SS' or 'H:MM:SS'. Flat playlist entries give duration as a number. */
|
||||
function fmtDuration(sec?: number): string | undefined {
|
||||
if (sec == null || !Number.isFinite(sec)) return undefined
|
||||
const s = Math.max(0, Math.round(sec))
|
||||
const h = Math.floor(s / 3600)
|
||||
const m = Math.floor((s % 3600) / 60)
|
||||
const r = s % 60
|
||||
const mm = h ? String(m).padStart(2, '0') : String(m)
|
||||
return `${h ? `${h}:` : ''}${mm}:${String(r).padStart(2, '0')}`
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the URL we'll enqueue for a playlist entry. Prefer an explicit http(s)
|
||||
* URL; fall back to a YouTube watch URL built from the id (the common case where
|
||||
* flat entries carry only an id). Returns null when nothing usable is present.
|
||||
*/
|
||||
function entryUrl(e: RawEntry): string | null {
|
||||
const cand = e.url || e.webpage_url
|
||||
if (cand && /^https?:\/\//i.test(cand)) return cand
|
||||
if (e.id) return `https://www.youtube.com/watch?v=${e.id}`
|
||||
return null
|
||||
}
|
||||
|
||||
function buildPlaylist(data: RawInfo): PlaylistInfo {
|
||||
const entries: PlaylistEntry[] = []
|
||||
;(data.entries ?? []).forEach((e, i) => {
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
/**
|
||||
* Windows Task Scheduler integration for the daily watched-source sync
|
||||
* (ROADMAP-PINCHFLAT.md Phase J). Registers a task that launches AeroFetch with
|
||||
* `--sync` once a day; the app then runs its startup sync of watched sources.
|
||||
*
|
||||
* NOTE: this is OS-level wiring and cannot be exercised in the Vite UI preview or
|
||||
* the unit tests — like the `aerofetch://` protocol registration, it needs a real
|
||||
* install + manual smoke test. `schtasks` is invoked via execFile (no shell), and
|
||||
* the only interpolated value is the trusted `process.execPath`.
|
||||
*/
|
||||
|
||||
import { execFile } from 'child_process'
|
||||
import type { ScheduledSyncStatus } from '@shared/ipc'
|
||||
|
||||
const TASK_NAME = 'AeroFetchDailySync'
|
||||
/** The argv flag the scheduled task passes so startup knows it's a sync launch. */
|
||||
export const SYNC_FLAG = '--sync'
|
||||
|
||||
function schtasks(args: string[]): Promise<{ code: number; stdout: string; stderr: string }> {
|
||||
return new Promise((resolve) => {
|
||||
execFile('schtasks', args, { windowsHide: true }, (err, stdout, stderr) => {
|
||||
const code = err ? ((err as { code?: number }).code ?? 1) : 0
|
||||
resolve({ code, stdout: String(stdout), stderr: String(stderr) })
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/** True when this launch came from the scheduled task (argv carries --sync). */
|
||||
export function isSyncLaunch(argv: string[]): boolean {
|
||||
return argv.includes(SYNC_FLAG)
|
||||
}
|
||||
|
||||
/** Whether the daily-sync scheduled task is currently registered. */
|
||||
export async function getScheduledSync(): Promise<ScheduledSyncStatus> {
|
||||
const r = await schtasks(['/Query', '/TN', TASK_NAME])
|
||||
return { enabled: r.code === 0 }
|
||||
}
|
||||
|
||||
/**
|
||||
* Register or remove the daily-sync scheduled task. Creating runs AeroFetch with
|
||||
* `--sync` every day at 09:00 (overwriting any prior task of the same name).
|
||||
*/
|
||||
export async function setScheduledSync(enabled: boolean): Promise<ScheduledSyncStatus> {
|
||||
if (enabled) {
|
||||
const tr = `"${process.execPath}" ${SYNC_FLAG}`
|
||||
const r = await schtasks([
|
||||
'/Create',
|
||||
'/F',
|
||||
'/SC',
|
||||
'DAILY',
|
||||
'/ST',
|
||||
'09:00',
|
||||
'/TN',
|
||||
TASK_NAME,
|
||||
'/TR',
|
||||
tr
|
||||
])
|
||||
return {
|
||||
enabled: r.code === 0,
|
||||
error: r.code === 0 ? undefined : r.stderr.trim() || 'Could not create the scheduled task.'
|
||||
}
|
||||
}
|
||||
const r = await schtasks(['/Delete', '/F', '/TN', TASK_NAME])
|
||||
// A missing task ("cannot find") is success for our purposes — it's already gone.
|
||||
const gone = r.code === 0 || /cannot find|does not exist/i.test(r.stderr)
|
||||
return { enabled: gone ? false : true, error: gone ? undefined : r.stderr.trim() }
|
||||
}
|
||||
@@ -36,6 +36,7 @@ const DEFAULTS: Settings = {
|
||||
customCommandEnabled: false,
|
||||
defaultTemplateId: null,
|
||||
notifyOnComplete: true,
|
||||
autoDownloadNew: true,
|
||||
hasCompletedOnboarding: false
|
||||
}
|
||||
|
||||
@@ -77,7 +78,10 @@ function sanitizeOptions(input: unknown): DownloadOptions {
|
||||
embedChapters: bool(o.embedChapters, d.embedChapters),
|
||||
embedMetadata: bool(o.embedMetadata, d.embedMetadata),
|
||||
embedThumbnail: bool(o.embedThumbnail, d.embedThumbnail),
|
||||
cropThumbnail: bool(o.cropThumbnail, d.cropThumbnail)
|
||||
cropThumbnail: bool(o.cropThumbnail, d.cropThumbnail),
|
||||
writeInfoJson: bool(o.writeInfoJson, d.writeInfoJson),
|
||||
writeThumbnailFile: bool(o.writeThumbnailFile, d.writeThumbnailFile),
|
||||
writeDescription: bool(o.writeDescription, d.writeDescription)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -163,6 +167,7 @@ export function setSettings(partial: Partial<Settings>): Settings {
|
||||
case 'downloadArchive':
|
||||
case 'customCommandEnabled':
|
||||
case 'notifyOnComplete':
|
||||
case 'autoDownloadNew':
|
||||
case 'hasCompletedOnboarding':
|
||||
if (typeof value === 'boolean') s.set(key, value)
|
||||
break
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
/**
|
||||
* Persistence for the media-manager index (Pinchflat-style; see
|
||||
* ROADMAP-PINCHFLAT.md). Two plain-JSON stores in userData, mirroring the
|
||||
* history.ts pattern (and the same deliberate choice to stay on JSON rather than
|
||||
* better-sqlite3 — revisit if a user indexes many large channels, see the
|
||||
* Phase H risk note in the roadmap):
|
||||
*
|
||||
* sources.json — one Source record per added channel/playlist
|
||||
* media-items.json — every MediaItem across all sources (queried by sourceId)
|
||||
*
|
||||
* Per-row validation on read (validation.ts) so a hand-edited or corrupted file
|
||||
* can't feed the UI malformed records (same approach as history/errorlog).
|
||||
*/
|
||||
|
||||
import { app } from 'electron'
|
||||
import { join } from 'path'
|
||||
import { readFileSync, writeFileSync, existsSync } from 'fs'
|
||||
import type { Source, MediaItem } from '@shared/ipc'
|
||||
import { isValidSource, isValidMediaItem } from './validation'
|
||||
import { mergeItemsPreservingState } from './indexerCore'
|
||||
|
||||
// A generous global cap so a runaway index can't grow the file unbounded; large
|
||||
// enough for several big channels. When exceeded, the most-recently-written
|
||||
// source's items are kept (they're placed first by replaceMediaItems).
|
||||
const MAX_ITEMS = 20000
|
||||
|
||||
function sourcesFile(): string {
|
||||
return join(app.getPath('userData'), 'sources.json')
|
||||
}
|
||||
|
||||
function itemsFile(): string {
|
||||
return join(app.getPath('userData'), 'media-items.json')
|
||||
}
|
||||
|
||||
function readJsonArray<T>(path: string, isValid: (o: unknown) => o is T): T[] {
|
||||
try {
|
||||
if (!existsSync(path)) return []
|
||||
const data = JSON.parse(readFileSync(path, 'utf8'))
|
||||
return Array.isArray(data) ? data.filter(isValid) : []
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
function writeJson(path: string, value: unknown): void {
|
||||
try {
|
||||
writeFileSync(path, JSON.stringify(value, null, 2))
|
||||
} catch {
|
||||
/* best-effort; a read-only data dir just means no persisted index */
|
||||
}
|
||||
}
|
||||
|
||||
// --- Sources ----------------------------------------------------------------
|
||||
|
||||
export function listSources(): Source[] {
|
||||
return readJsonArray(sourcesFile(), isValidSource)
|
||||
}
|
||||
|
||||
export function getSource(id: string): Source | undefined {
|
||||
return listSources().find((s) => s.id === id)
|
||||
}
|
||||
|
||||
/** Insert or replace a source by id (a re-index updates the existing record). */
|
||||
export function upsertSource(source: Source): Source[] {
|
||||
const sources = [source, ...listSources().filter((s) => s.id !== source.id)]
|
||||
writeJson(sourcesFile(), sources)
|
||||
return sources
|
||||
}
|
||||
|
||||
/** Toggle whether a source is watched for new uploads (Phase J). Returns all sources. */
|
||||
export function setSourceWatched(id: string, watched: boolean): Source[] {
|
||||
const sources = listSources().map((s) => (s.id === id ? { ...s, watched } : s))
|
||||
writeJson(sourcesFile(), sources)
|
||||
return sources
|
||||
}
|
||||
|
||||
/** Remove a source and all of its media items. Returns the remaining sources. */
|
||||
export function removeSource(id: string): Source[] {
|
||||
const sources = listSources().filter((s) => s.id !== id)
|
||||
writeJson(sourcesFile(), sources)
|
||||
const items = listAllItems().filter((m) => m.sourceId !== id)
|
||||
writeJson(itemsFile(), items)
|
||||
return sources
|
||||
}
|
||||
|
||||
// --- Media items ------------------------------------------------------------
|
||||
|
||||
function listAllItems(): MediaItem[] {
|
||||
return readJsonArray(itemsFile(), isValidMediaItem)
|
||||
}
|
||||
|
||||
export function listMediaItems(sourceId: string): MediaItem[] {
|
||||
return listAllItems().filter((m) => m.sourceId === sourceId)
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace all media items for one source with a fresh set (the result of a
|
||||
* (re)index). Other sources' items are preserved; the new items are placed first
|
||||
* so they survive the MAX_ITEMS cap.
|
||||
*/
|
||||
export function replaceMediaItems(sourceId: string, items: MediaItem[]): void {
|
||||
const others = listAllItems().filter((m) => m.sourceId !== sourceId)
|
||||
writeJson(itemsFile(), [...items, ...others].slice(0, MAX_ITEMS))
|
||||
}
|
||||
|
||||
/**
|
||||
* Incrementally merge a freshly-indexed item list into the persisted store,
|
||||
* preserving the downloaded state of videos already on disk (see Phase I). The
|
||||
* fresh list defines current membership/order; returns the merged items and how
|
||||
* many were new since the last index.
|
||||
*/
|
||||
export function mergeMediaItems(
|
||||
sourceId: string,
|
||||
fresh: MediaItem[]
|
||||
): { items: MediaItem[]; newCount: number } {
|
||||
const result = mergeItemsPreservingState(listMediaItems(sourceId), fresh)
|
||||
replaceMediaItems(sourceId, result.items)
|
||||
return result
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark one media item downloaded, recording its file path + time. Returns the
|
||||
* updated list for that item's source (or [] if the id is unknown). Used by the
|
||||
* library view once a queued item completes (Phase H/I).
|
||||
*/
|
||||
export function setMediaItemDownloaded(id: string, filePath?: string): MediaItem[] {
|
||||
const all = listAllItems()
|
||||
const target = all.find((m) => m.id === id)
|
||||
if (!target) return []
|
||||
const updated = all.map((m) =>
|
||||
m.id === id ? { ...m, downloaded: true, downloadedAt: Date.now(), filePath } : m
|
||||
)
|
||||
writeJson(itemsFile(), updated)
|
||||
return updated.filter((m) => m.sourceId === target.sourceId)
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
/**
|
||||
* Watched-source sync (Pinchflat-style; ROADMAP-PINCHFLAT.md Phase J). Re-indexes
|
||||
* every watched Source and reports the videos that are new since its last index.
|
||||
* A YouTube RSS feed is used as a cheap pre-check so a source with no new uploads
|
||||
* is skipped before the (more expensive) full `yt-dlp` re-index.
|
||||
*/
|
||||
|
||||
import { listSources, listMediaItems } from './sources'
|
||||
import { indexSource } from './indexer'
|
||||
import { parseRssVideoIds } from './indexerCore'
|
||||
import type { IndexProgress, MediaItem, SyncResult } from '@shared/ipc'
|
||||
|
||||
/** Fetch a YouTube RSS feed and return its recent video ids. Throws on failure. */
|
||||
async function fetchFeedIds(feedUrl: string): Promise<string[]> {
|
||||
const res = await fetch(feedUrl, { signal: AbortSignal.timeout(15_000) })
|
||||
if (!res.ok) throw new Error(`feed responded ${res.status}`)
|
||||
return parseRssVideoIds(await res.text())
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-index every watched source and collect the videos new since the last index.
|
||||
* RSS pre-check: if the feed shows only ids the source already knows, the full
|
||||
* re-index is skipped. `onProgress` is forwarded from the underlying indexSource.
|
||||
*/
|
||||
export async function syncWatchedSources(
|
||||
onProgress: (p: IndexProgress) => void
|
||||
): Promise<SyncResult> {
|
||||
try {
|
||||
const watched = listSources().filter((s) => s.watched)
|
||||
const newItems: MediaItem[] = []
|
||||
for (const src of watched) {
|
||||
// Cheap freshness check — skip the full re-index when nothing is new.
|
||||
if (src.feedUrl) {
|
||||
const known = new Set(listMediaItems(src.id).map((m) => m.videoId))
|
||||
const recent = await fetchFeedIds(src.feedUrl).catch(() => null)
|
||||
if (recent && recent.length > 0 && recent.every((id) => known.has(id))) continue
|
||||
}
|
||||
const before = new Set(listMediaItems(src.id).map((m) => m.videoId))
|
||||
const res = await indexSource(src.url, onProgress)
|
||||
if (res.ok) {
|
||||
for (const it of listMediaItems(src.id)) {
|
||||
if (!before.has(it.videoId)) newItems.push(it)
|
||||
}
|
||||
}
|
||||
}
|
||||
return { ok: true, newItems }
|
||||
} catch (e) {
|
||||
return { ok: false, newItems: [], error: e instanceof Error ? e.message : String(e) }
|
||||
}
|
||||
}
|
||||
+36
-1
@@ -7,7 +7,7 @@
|
||||
*/
|
||||
|
||||
import { isAbsolute } from 'path'
|
||||
import type { HistoryEntry, ErrorLogEntry, CommandTemplate } from '@shared/ipc'
|
||||
import type { HistoryEntry, ErrorLogEntry, CommandTemplate, Source, MediaItem } from '@shared/ipc'
|
||||
|
||||
// --- Path-traversal sanitization (audit S4) ---------------------------------
|
||||
|
||||
@@ -75,3 +75,38 @@ export function isTemplateLike(o: unknown): o is CommandTemplate {
|
||||
const t = o as Record<string, unknown>
|
||||
return typeof t.id === 'string' || typeof t.id === 'number'
|
||||
}
|
||||
|
||||
/** A persisted sources.json row must have the right shape or it's dropped on read. */
|
||||
export function isValidSource(o: unknown): o is Source {
|
||||
if (!o || typeof o !== 'object') return false
|
||||
const s = o as Record<string, unknown>
|
||||
return (
|
||||
typeof s.id === 'string' &&
|
||||
typeof s.url === 'string' &&
|
||||
(s.kind === 'channel' || s.kind === 'playlist') &&
|
||||
typeof s.title === 'string' &&
|
||||
typeof s.addedAt === 'number' &&
|
||||
typeof s.itemCount === 'number' &&
|
||||
isOptionalString(s.channel) &&
|
||||
(s.lastIndexedAt === undefined || typeof s.lastIndexedAt === 'number')
|
||||
)
|
||||
}
|
||||
|
||||
/** A persisted media-items.json row must have the right shape or it's dropped on read. */
|
||||
export function isValidMediaItem(o: unknown): o is MediaItem {
|
||||
if (!o || typeof o !== 'object') return false
|
||||
const m = o as Record<string, unknown>
|
||||
return (
|
||||
typeof m.id === 'string' &&
|
||||
typeof m.sourceId === 'string' &&
|
||||
typeof m.videoId === 'string' &&
|
||||
typeof m.title === 'string' &&
|
||||
typeof m.url === 'string' &&
|
||||
typeof m.playlistTitle === 'string' &&
|
||||
typeof m.playlistIndex === 'number' &&
|
||||
typeof m.downloaded === 'boolean' &&
|
||||
isOptionalString(m.durationLabel) &&
|
||||
isOptionalString(m.filePath) &&
|
||||
(m.downloadedAt === undefined || typeof m.downloadedAt === 'number')
|
||||
)
|
||||
}
|
||||
|
||||
+49
-1
@@ -17,7 +17,13 @@ import {
|
||||
type ErrorLogEntry,
|
||||
type BackupExportResult,
|
||||
type BackupImportResult,
|
||||
type SystemThemeInfo
|
||||
type SystemThemeInfo,
|
||||
type Source,
|
||||
type MediaItem,
|
||||
type IndexProgress,
|
||||
type IndexSourceResult,
|
||||
type SyncResult,
|
||||
type ScheduledSyncStatus
|
||||
} from '@shared/ipc'
|
||||
|
||||
// The surface exposed to the renderer. Keep this thin: it only forwards to IPC.
|
||||
@@ -121,6 +127,48 @@ const api = {
|
||||
const listener = (_e: IpcRendererEvent, url: string): void => cb(url)
|
||||
ipcRenderer.on(IpcChannels.externalUrl, listener)
|
||||
return () => ipcRenderer.removeListener(IpcChannels.externalUrl, listener)
|
||||
},
|
||||
|
||||
// --- Media-manager sources (Pinchflat-style; see ROADMAP-PINCHFLAT.md) ---
|
||||
|
||||
listSources: (): Promise<Source[]> => ipcRenderer.invoke(IpcChannels.sourcesList),
|
||||
|
||||
/** Index (or re-index) a channel/playlist URL into the persisted source list. */
|
||||
indexSource: (url: string): Promise<IndexSourceResult> =>
|
||||
ipcRenderer.invoke(IpcChannels.sourceIndex, url),
|
||||
|
||||
/** Re-index an existing source by id (refreshes its media-item list). */
|
||||
reindexSource: (id: string): Promise<IndexSourceResult> =>
|
||||
ipcRenderer.invoke(IpcChannels.sourceReindex, id),
|
||||
|
||||
removeSource: (id: string): Promise<Source[]> =>
|
||||
ipcRenderer.invoke(IpcChannels.sourceRemove, id),
|
||||
|
||||
listSourceItems: (sourceId: string): Promise<MediaItem[]> =>
|
||||
ipcRenderer.invoke(IpcChannels.sourceItems, sourceId),
|
||||
|
||||
/** Persist that a media item has finished downloading (drives incremental sync). */
|
||||
markSourceItemDownloaded: (id: string, filePath?: string): Promise<MediaItem[]> =>
|
||||
ipcRenderer.invoke(IpcChannels.sourceItemDownloaded, id, filePath),
|
||||
|
||||
/** Toggle whether a source is watched for new uploads. */
|
||||
setSourceWatched: (id: string, watched: boolean): Promise<Source[]> =>
|
||||
ipcRenderer.invoke(IpcChannels.sourceSetWatched, id, watched),
|
||||
|
||||
/** Re-index all watched sources; resolves with the videos found new. */
|
||||
syncSources: (): Promise<SyncResult> => ipcRenderer.invoke(IpcChannels.sourcesSync),
|
||||
|
||||
/** Read / write the Windows daily-sync scheduled task. */
|
||||
getScheduledSync: (): Promise<ScheduledSyncStatus> =>
|
||||
ipcRenderer.invoke(IpcChannels.scheduledSyncGet),
|
||||
setScheduledSync: (enabled: boolean): Promise<ScheduledSyncStatus> =>
|
||||
ipcRenderer.invoke(IpcChannels.scheduledSyncSet, enabled),
|
||||
|
||||
/** Subscribe to live indexing progress. Returns an unsubscribe function. */
|
||||
onIndexProgress: (cb: (p: IndexProgress) => void): (() => void) => {
|
||||
const listener = (_e: IpcRendererEvent, p: IndexProgress): void => cb(p)
|
||||
ipcRenderer.on(IpcChannels.indexProgress, listener)
|
||||
return () => ipcRenderer.removeListener(IpcChannels.indexProgress, listener)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ import { useState } from 'react'
|
||||
import { FluentProvider, makeStyles, tokens } from '@fluentui/react-components'
|
||||
import { Sidebar, type TabValue } from './components/Sidebar'
|
||||
import { DownloadsView } from './components/DownloadsView'
|
||||
import { LibraryView } from './components/LibraryView'
|
||||
import { HistoryView } from './components/HistoryView'
|
||||
import { SettingsView } from './components/SettingsView'
|
||||
import { Onboarding } from './components/Onboarding'
|
||||
@@ -66,6 +67,7 @@ function App(): React.JSX.Element {
|
||||
|
||||
<main className={styles.content}>
|
||||
{tab === 'downloads' && <DownloadsView />}
|
||||
{tab === 'library' && <LibraryView />}
|
||||
{tab === 'history' && <HistoryView />}
|
||||
{tab === 'settings' && <SettingsView />}
|
||||
</main>
|
||||
|
||||
@@ -230,6 +230,29 @@ export function DownloadOptionsForm({ value, onChange }: Props): React.JSX.Eleme
|
||||
</Caption1>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<Field
|
||||
label="Sidecar files"
|
||||
hint="Write separate metadata/poster/description files next to each download — handy for Jellyfin, Plex or Kodi libraries."
|
||||
>
|
||||
<div className={styles.subGroup}>
|
||||
<Checkbox
|
||||
checked={value.writeInfoJson}
|
||||
onChange={(_, d) => setOpt('writeInfoJson', !!d.checked)}
|
||||
label="Metadata (.info.json)"
|
||||
/>
|
||||
<Checkbox
|
||||
checked={value.writeThumbnailFile}
|
||||
onChange={(_, d) => setOpt('writeThumbnailFile', !!d.checked)}
|
||||
label="Thumbnail image file"
|
||||
/>
|
||||
<Checkbox
|
||||
checked={value.writeDescription}
|
||||
onChange={(_, d) => setOpt('writeDescription', !!d.checked)}
|
||||
label="Description (.description)"
|
||||
/>
|
||||
</div>
|
||||
</Field>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,568 @@
|
||||
import { useEffect, useMemo, useState } from 'react'
|
||||
import {
|
||||
Subtitle2,
|
||||
Body1,
|
||||
Caption1,
|
||||
Text,
|
||||
Input,
|
||||
Button,
|
||||
Checkbox,
|
||||
Switch,
|
||||
Spinner,
|
||||
makeStyles,
|
||||
mergeClasses,
|
||||
tokens,
|
||||
shorthands
|
||||
} from '@fluentui/react-components'
|
||||
import {
|
||||
SearchRegular,
|
||||
ArrowSyncRegular,
|
||||
ArrowClockwiseRegular,
|
||||
DeleteRegular,
|
||||
ArrowDownloadRegular,
|
||||
ChevronDownRegular,
|
||||
ChevronRightRegular,
|
||||
AppsListRegular,
|
||||
VideoClipMultipleRegular,
|
||||
AlertRegular,
|
||||
LibraryRegular
|
||||
} from '@fluentui/react-icons'
|
||||
import type { MediaItem, Source } from '@shared/ipc'
|
||||
import { useSources, MAX_ENQUEUE_BATCH } from '../store/sources'
|
||||
import { useSettings } from '../store/settings'
|
||||
import { useDownloads, type DownloadStatus } from '../store/downloads'
|
||||
|
||||
// True in the standalone browser preview (no Electron preload).
|
||||
const PREVIEW = typeof window === 'undefined' || !window.electron
|
||||
|
||||
/** Per-item status shown in the library: a live queue status, or pending/downloaded. */
|
||||
type ItemStatus = DownloadStatus | 'pending'
|
||||
|
||||
const STATUS_LABEL: Record<ItemStatus, string> = {
|
||||
pending: 'Pending',
|
||||
queued: 'Queued',
|
||||
downloading: 'Downloading',
|
||||
completed: 'Downloaded',
|
||||
error: 'Failed',
|
||||
canceled: 'Canceled'
|
||||
}
|
||||
|
||||
const useStyles = makeStyles({
|
||||
root: { display: 'flex', flexDirection: 'column', gap: '18px' },
|
||||
header: { display: 'flex', flexDirection: 'column', gap: '2px' },
|
||||
sub: { color: tokens.colorNeutralForeground3 },
|
||||
addRow: { display: 'flex', gap: '8px' },
|
||||
addInput: { flexGrow: 1 },
|
||||
toolbar: {
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
gap: '14px',
|
||||
flexWrap: 'wrap',
|
||||
paddingTop: '2px'
|
||||
},
|
||||
toolbarSpacer: { flexGrow: 1 },
|
||||
switchRow: {
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
gap: '2px',
|
||||
color: tokens.colorNeutralForeground2
|
||||
},
|
||||
progress: {
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
gap: '8px',
|
||||
color: tokens.colorNeutralForeground2,
|
||||
fontSize: tokens.fontSizeBase200
|
||||
},
|
||||
error: { color: tokens.colorPaletteRedForeground1 },
|
||||
empty: {
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
alignItems: 'center',
|
||||
gap: '8px',
|
||||
padding: '56px 16px',
|
||||
color: tokens.colorNeutralForeground3,
|
||||
textAlign: 'center'
|
||||
},
|
||||
list: { display: 'flex', flexDirection: 'column', gap: '10px' },
|
||||
card: {
|
||||
border: `1px solid ${tokens.colorNeutralStroke2}`,
|
||||
...shorthands.borderRadius(tokens.borderRadiusLarge),
|
||||
backgroundColor: tokens.colorNeutralBackground1,
|
||||
overflow: 'hidden'
|
||||
},
|
||||
cardHead: {
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
gap: '12px',
|
||||
padding: '12px 14px',
|
||||
cursor: 'pointer',
|
||||
':hover': { backgroundColor: tokens.colorNeutralBackground1Hover }
|
||||
},
|
||||
srcIcon: {
|
||||
width: '34px',
|
||||
height: '34px',
|
||||
flexShrink: 0,
|
||||
borderRadius: tokens.borderRadiusMedium,
|
||||
backgroundColor: tokens.colorBrandBackground2,
|
||||
color: tokens.colorBrandForeground2,
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
justifyContent: 'center',
|
||||
fontSize: '18px'
|
||||
},
|
||||
srcMeta: { display: 'flex', flexDirection: 'column', minWidth: 0, flexGrow: 1 },
|
||||
srcTitleRow: { display: 'flex', alignItems: 'center', gap: '8px', minWidth: 0 },
|
||||
srcTitle: { fontWeight: tokens.fontWeightSemibold, color: tokens.colorNeutralForeground1 },
|
||||
watchBadge: {
|
||||
flexShrink: 0,
|
||||
fontSize: tokens.fontSizeBase100,
|
||||
padding: '0 7px',
|
||||
...shorthands.borderRadius(tokens.borderRadiusCircular),
|
||||
backgroundColor: tokens.colorBrandBackground2,
|
||||
color: tokens.colorBrandForeground2
|
||||
},
|
||||
srcSub: { color: tokens.colorNeutralForeground3 },
|
||||
detail: {
|
||||
borderTop: `1px solid ${tokens.colorNeutralStroke2}`,
|
||||
padding: '12px 14px',
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
gap: '12px'
|
||||
},
|
||||
actionRow: { display: 'flex', alignItems: 'center', gap: '8px', flexWrap: 'wrap' },
|
||||
actionSpacer: { flexGrow: 1 },
|
||||
group: { display: 'flex', flexDirection: 'column' },
|
||||
groupHead: {
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
gap: '8px',
|
||||
padding: '6px 4px',
|
||||
color: tokens.colorNeutralForeground2
|
||||
},
|
||||
groupTitle: { fontWeight: tokens.fontWeightSemibold, flexGrow: 1, minWidth: 0 },
|
||||
rows: { display: 'flex', flexDirection: 'column' },
|
||||
row: {
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
gap: '10px',
|
||||
padding: '5px 4px 5px 18px'
|
||||
},
|
||||
rowMain: { display: 'flex', flexDirection: 'column', minWidth: 0, flexGrow: 1 },
|
||||
rowTitle: {
|
||||
whiteSpace: 'nowrap',
|
||||
overflow: 'hidden',
|
||||
textOverflow: 'ellipsis',
|
||||
color: tokens.colorNeutralForeground1
|
||||
},
|
||||
rowMeta: { color: tokens.colorNeutralForeground3 },
|
||||
pill: {
|
||||
flexShrink: 0,
|
||||
fontSize: tokens.fontSizeBase200,
|
||||
padding: '1px 8px',
|
||||
...shorthands.borderRadius(tokens.borderRadiusCircular),
|
||||
backgroundColor: tokens.colorNeutralBackground3,
|
||||
color: tokens.colorNeutralForeground3
|
||||
},
|
||||
pillDownloading: {
|
||||
backgroundColor: tokens.colorBrandBackground2,
|
||||
color: tokens.colorBrandForeground2
|
||||
},
|
||||
pillCompleted: {
|
||||
backgroundColor: tokens.colorPaletteGreenBackground2,
|
||||
color: tokens.colorPaletteGreenForeground2
|
||||
},
|
||||
pillError: {
|
||||
backgroundColor: tokens.colorPaletteRedBackground2,
|
||||
color: tokens.colorPaletteRedForeground2
|
||||
}
|
||||
})
|
||||
|
||||
/** Group items by playlist, sorted by index within a group; 'Uploads' sinks last. */
|
||||
function groupByPlaylist(items: MediaItem[]): { title: string; items: MediaItem[] }[] {
|
||||
const map = new Map<string, MediaItem[]>()
|
||||
for (const it of items) {
|
||||
const arr = map.get(it.playlistTitle) ?? []
|
||||
arr.push(it)
|
||||
map.set(it.playlistTitle, arr)
|
||||
}
|
||||
const groups = [...map.entries()].map(([title, its]) => ({
|
||||
title,
|
||||
items: [...its].sort((a, b) => a.playlistIndex - b.playlistIndex)
|
||||
}))
|
||||
groups.sort(
|
||||
(a, b) =>
|
||||
(a.title === 'Uploads' ? 1 : 0) - (b.title === 'Uploads' ? 1 : 0) ||
|
||||
a.title.localeCompare(b.title)
|
||||
)
|
||||
return groups
|
||||
}
|
||||
|
||||
function relTime(ms?: number): string {
|
||||
if (!ms) return 'never'
|
||||
const mins = Math.round((Date.now() - ms) / 60000)
|
||||
if (mins < 1) return 'just now'
|
||||
if (mins < 60) return `${mins} min ago`
|
||||
const hrs = Math.round(mins / 60)
|
||||
if (hrs < 24) return `${hrs} h ago`
|
||||
return `${Math.round(hrs / 24)} d ago`
|
||||
}
|
||||
|
||||
export function LibraryView(): React.JSX.Element {
|
||||
const styles = useStyles()
|
||||
const sources = useSources((s) => s.sources)
|
||||
const itemsBySource = useSources((s) => s.itemsBySource)
|
||||
const selectedSourceId = useSources((s) => s.selectedSourceId)
|
||||
const indexing = useSources((s) => s.indexing)
|
||||
const selectSource = useSources((s) => s.selectSource)
|
||||
const indexSource = useSources((s) => s.indexSource)
|
||||
const reindexSource = useSources((s) => s.reindexSource)
|
||||
const removeSource = useSources((s) => s.removeSource)
|
||||
const enqueueItems = useSources((s) => s.enqueueItems)
|
||||
const setWatched = useSources((s) => s.setWatched)
|
||||
const syncWatched = useSources((s) => s.syncWatched)
|
||||
const syncing = useSources((s) => s.syncing)
|
||||
const downloadItems = useDownloads((s) => s.items)
|
||||
const autoDownloadNew = useSettings((s) => s.autoDownloadNew)
|
||||
const updateSettings = useSettings((s) => s.update)
|
||||
|
||||
const [url, setUrl] = useState('')
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
const [selected, setSelected] = useState<Set<string>>(new Set())
|
||||
const [syncNote, setSyncNote] = useState<string | null>(null)
|
||||
const [scheduled, setScheduled] = useState(false)
|
||||
|
||||
const watchedCount = sources.filter((s) => s.watched).length
|
||||
|
||||
// Load the current scheduled-sync (Task Scheduler) state once.
|
||||
useEffect(() => {
|
||||
if (PREVIEW) return
|
||||
window.api.getScheduledSync().then((s) => setScheduled(s.enabled)).catch(() => {})
|
||||
}, [])
|
||||
|
||||
async function onCheckNew(): Promise<void> {
|
||||
setSyncNote(null)
|
||||
const n = await syncWatched()
|
||||
setSyncNote(n > 0 ? `Found ${n} new video${n === 1 ? '' : 's'}.` : 'No new videos.')
|
||||
}
|
||||
|
||||
async function toggleScheduled(next: boolean): Promise<void> {
|
||||
setScheduled(next) // optimistic
|
||||
if (PREVIEW) return
|
||||
const res = await window.api.setScheduledSync(next)
|
||||
setScheduled(res.enabled)
|
||||
if (res.error) setError(res.error)
|
||||
}
|
||||
|
||||
// Reset the selection whenever the expanded source changes.
|
||||
useEffect(() => setSelected(new Set()), [selectedSourceId])
|
||||
|
||||
// Live per-URL queue status so a video row reflects its real download state.
|
||||
const statusByUrl = useMemo(() => {
|
||||
const m = new Map<string, DownloadStatus>()
|
||||
for (const d of downloadItems) m.set(d.url, d.status)
|
||||
return m
|
||||
}, [downloadItems])
|
||||
|
||||
const items = selectedSourceId ? (itemsBySource[selectedSourceId] ?? []) : []
|
||||
const groups = useMemo(() => groupByPlaylist(items), [items])
|
||||
|
||||
const effStatus = (it: MediaItem): ItemStatus =>
|
||||
statusByUrl.get(it.url) ?? (it.downloaded ? 'completed' : 'pending')
|
||||
|
||||
const pendingItems = items.filter((it) => effStatus(it) === 'pending')
|
||||
|
||||
async function onIndex(): Promise<void> {
|
||||
setError(null)
|
||||
const u = url.trim()
|
||||
if (!u || indexing.active) return
|
||||
const res = await indexSource(u)
|
||||
if (res.ok) setUrl('')
|
||||
else setError(res.error ?? 'Could not index that link.')
|
||||
}
|
||||
|
||||
function toggle(id: string, on: boolean): void {
|
||||
setSelected((prev) => {
|
||||
const next = new Set(prev)
|
||||
if (on) next.add(id)
|
||||
else next.delete(id)
|
||||
return next
|
||||
})
|
||||
}
|
||||
|
||||
function toggleGroup(groupItems: MediaItem[], on: boolean): void {
|
||||
setSelected((prev) => {
|
||||
const next = new Set(prev)
|
||||
for (const it of groupItems) {
|
||||
if (on) next.add(it.id)
|
||||
else next.delete(it.id)
|
||||
}
|
||||
return next
|
||||
})
|
||||
}
|
||||
|
||||
function downloadSelected(): void {
|
||||
if (!selectedSourceId) return
|
||||
const chosen = items.filter((it) => selected.has(it.id))
|
||||
enqueueItems(selectedSourceId, chosen)
|
||||
setSelected(new Set())
|
||||
}
|
||||
|
||||
function downloadPending(): void {
|
||||
if (!selectedSourceId) return
|
||||
enqueueItems(selectedSourceId, pendingItems)
|
||||
}
|
||||
|
||||
function pillClass(status: ItemStatus): string {
|
||||
if (status === 'downloading' || status === 'queued') return mergeClasses(styles.pill, styles.pillDownloading)
|
||||
if (status === 'completed') return mergeClasses(styles.pill, styles.pillCompleted)
|
||||
if (status === 'error') return mergeClasses(styles.pill, styles.pillError)
|
||||
return styles.pill
|
||||
}
|
||||
|
||||
return (
|
||||
<div className={styles.root}>
|
||||
<div className={styles.header}>
|
||||
<Subtitle2>Library</Subtitle2>
|
||||
<Caption1 className={styles.sub}>
|
||||
Index a channel or playlist once, then download it into organized folders.
|
||||
</Caption1>
|
||||
</div>
|
||||
|
||||
<div className={styles.addRow}>
|
||||
<Input
|
||||
className={styles.addInput}
|
||||
value={url}
|
||||
onChange={(_, d) => setUrl(d.value)}
|
||||
onKeyDown={(e) => e.key === 'Enter' && onIndex()}
|
||||
placeholder="Paste a channel or playlist URL…"
|
||||
size="large"
|
||||
contentBefore={<LibraryRegular />}
|
||||
disabled={indexing.active}
|
||||
/>
|
||||
<Button
|
||||
size="large"
|
||||
appearance="primary"
|
||||
icon={indexing.active ? <Spinner size="tiny" /> : <SearchRegular />}
|
||||
onClick={onIndex}
|
||||
disabled={!url.trim() || indexing.active}
|
||||
>
|
||||
Index
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
<div className={styles.toolbar}>
|
||||
<Button
|
||||
size="small"
|
||||
appearance="secondary"
|
||||
icon={syncing ? <Spinner size="tiny" /> : <ArrowClockwiseRegular />}
|
||||
onClick={onCheckNew}
|
||||
disabled={syncing || watchedCount === 0}
|
||||
>
|
||||
Check {watchedCount > 0 ? `${watchedCount} watched` : 'watched'} for new
|
||||
</Button>
|
||||
{syncNote && <Caption1 className={styles.sub}>{syncNote}</Caption1>}
|
||||
<div className={styles.toolbarSpacer} />
|
||||
<span className={styles.switchRow}>
|
||||
<Caption1>Auto-download new</Caption1>
|
||||
<Switch
|
||||
checked={autoDownloadNew}
|
||||
onChange={(_, d) => updateSettings({ autoDownloadNew: d.checked })}
|
||||
aria-label="Auto-download new uploads"
|
||||
/>
|
||||
</span>
|
||||
<span className={styles.switchRow}>
|
||||
<Caption1>Daily sync</Caption1>
|
||||
<Switch
|
||||
checked={scheduled}
|
||||
onChange={(_, d) => toggleScheduled(d.checked)}
|
||||
aria-label="Daily background sync"
|
||||
/>
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{indexing.active && (
|
||||
<div className={styles.progress}>
|
||||
<Spinner size="tiny" />
|
||||
<Text>
|
||||
{indexing.message ?? 'Indexing…'}
|
||||
{indexing.current && indexing.total ? ` (${indexing.current}/${indexing.total})` : ''}
|
||||
</Text>
|
||||
</div>
|
||||
)}
|
||||
{error && <Caption1 className={styles.error}>{error}</Caption1>}
|
||||
|
||||
{sources.length === 0 && !indexing.active ? (
|
||||
<div className={styles.empty}>
|
||||
<LibraryRegular fontSize={40} />
|
||||
<Body1>No channels or playlists yet. Paste one above to index it.</Body1>
|
||||
</div>
|
||||
) : (
|
||||
<div className={styles.list}>
|
||||
{sources.map((src) => (
|
||||
<SourceCard
|
||||
key={src.id}
|
||||
styles={styles}
|
||||
source={src}
|
||||
expanded={selectedSourceId === src.id}
|
||||
onToggleExpand={() =>
|
||||
selectSource(selectedSourceId === src.id ? null : src.id)
|
||||
}
|
||||
>
|
||||
<div className={styles.detail}>
|
||||
<div className={styles.actionRow}>
|
||||
<Caption1 className={styles.srcSub}>
|
||||
{items.length} videos · {pendingItems.length} pending · indexed{' '}
|
||||
{relTime(src.lastIndexedAt)}
|
||||
</Caption1>
|
||||
<div className={styles.actionSpacer} />
|
||||
{selected.size > 0 ? (
|
||||
<>
|
||||
<Button size="small" appearance="subtle" onClick={() => setSelected(new Set())}>
|
||||
Clear
|
||||
</Button>
|
||||
<Button
|
||||
size="small"
|
||||
appearance="primary"
|
||||
icon={<ArrowDownloadRegular />}
|
||||
onClick={downloadSelected}
|
||||
>
|
||||
Download {Math.min(selected.size, MAX_ENQUEUE_BATCH)} selected
|
||||
</Button>
|
||||
</>
|
||||
) : (
|
||||
<Button
|
||||
size="small"
|
||||
appearance="primary"
|
||||
icon={<ArrowDownloadRegular />}
|
||||
onClick={downloadPending}
|
||||
disabled={pendingItems.length === 0}
|
||||
>
|
||||
Download {Math.min(pendingItems.length, MAX_ENQUEUE_BATCH)} pending
|
||||
</Button>
|
||||
)}
|
||||
<span className={styles.switchRow}>
|
||||
<Caption1>Watch</Caption1>
|
||||
<Switch
|
||||
checked={!!src.watched}
|
||||
onChange={(_, d) => setWatched(src.id, !!d.checked)}
|
||||
aria-label={`Watch ${src.title} for new uploads`}
|
||||
/>
|
||||
</span>
|
||||
<Button
|
||||
size="small"
|
||||
appearance="subtle"
|
||||
icon={<ArrowSyncRegular />}
|
||||
onClick={() => reindexSource(src.id)}
|
||||
disabled={indexing.active}
|
||||
>
|
||||
Re-index
|
||||
</Button>
|
||||
<Button
|
||||
size="small"
|
||||
appearance="subtle"
|
||||
icon={<DeleteRegular />}
|
||||
onClick={() => removeSource(src.id)}
|
||||
>
|
||||
Remove
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{groups.map((g) => {
|
||||
const allOn = g.items.every((it) => selected.has(it.id))
|
||||
return (
|
||||
<div key={g.title} className={styles.group}>
|
||||
<div className={styles.groupHead}>
|
||||
<AppsListRegular />
|
||||
<span className={styles.groupTitle}>{g.title}</span>
|
||||
<Caption1 className={styles.srcSub}>{g.items.length}</Caption1>
|
||||
<Button
|
||||
size="small"
|
||||
appearance="subtle"
|
||||
onClick={() => toggleGroup(g.items, !allOn)}
|
||||
>
|
||||
{allOn ? 'None' : 'All'}
|
||||
</Button>
|
||||
</div>
|
||||
<div className={styles.rows}>
|
||||
{g.items.map((it) => {
|
||||
const status = effStatus(it)
|
||||
return (
|
||||
<div key={it.id} className={styles.row}>
|
||||
<Checkbox
|
||||
checked={selected.has(it.id)}
|
||||
onChange={(_, d) => toggle(it.id, !!d.checked)}
|
||||
aria-label={`Select ${it.title}`}
|
||||
/>
|
||||
<div className={styles.rowMain}>
|
||||
<span className={styles.rowTitle}>
|
||||
{it.playlistIndex}. {it.title}
|
||||
</span>
|
||||
{it.durationLabel && (
|
||||
<Caption1 className={styles.rowMeta}>{it.durationLabel}</Caption1>
|
||||
)}
|
||||
</div>
|
||||
<span className={pillClass(status)}>{STATUS_LABEL[status]}</span>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</SourceCard>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
/** One collapsible source card (header always shown; children render when expanded). */
|
||||
function SourceCard({
|
||||
styles,
|
||||
source,
|
||||
expanded,
|
||||
onToggleExpand,
|
||||
children
|
||||
}: {
|
||||
styles: ReturnType<typeof useStyles>
|
||||
source: Source
|
||||
expanded: boolean
|
||||
onToggleExpand: () => void
|
||||
children: React.ReactNode
|
||||
}): React.JSX.Element {
|
||||
return (
|
||||
<div className={styles.card}>
|
||||
<div
|
||||
className={styles.cardHead}
|
||||
onClick={onToggleExpand}
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
onKeyDown={(e) => (e.key === 'Enter' || e.key === ' ') && onToggleExpand()}
|
||||
aria-expanded={expanded}
|
||||
>
|
||||
{expanded ? <ChevronDownRegular /> : <ChevronRightRegular />}
|
||||
<div className={styles.srcIcon}>
|
||||
<VideoClipMultipleRegular />
|
||||
</div>
|
||||
<div className={styles.srcMeta}>
|
||||
<span className={styles.srcTitleRow}>
|
||||
<span className={styles.srcTitle}>{source.title}</span>
|
||||
{source.watched && (
|
||||
<span className={styles.watchBadge}>
|
||||
<AlertRegular fontSize={11} /> Watching
|
||||
</span>
|
||||
)}
|
||||
</span>
|
||||
<Caption1 className={styles.srcSub}>
|
||||
{source.kind === 'channel' ? 'Channel' : 'Playlist'} · {source.itemCount} videos
|
||||
{source.channel && source.channel !== source.title ? ` · ${source.channel}` : ''}
|
||||
</Caption1>
|
||||
</div>
|
||||
</div>
|
||||
{expanded && children}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -10,12 +10,13 @@ import {
|
||||
ArrowDownloadFilled,
|
||||
ArrowDownloadRegular,
|
||||
HistoryRegular,
|
||||
LibraryRegular,
|
||||
SettingsRegular,
|
||||
WeatherMoonRegular,
|
||||
WeatherSunnyRegular
|
||||
} from '@fluentui/react-icons'
|
||||
|
||||
export type TabValue = 'downloads' | 'history' | 'settings'
|
||||
export type TabValue = 'downloads' | 'library' | 'history' | 'settings'
|
||||
|
||||
const useStyles = makeStyles({
|
||||
root: {
|
||||
@@ -114,6 +115,7 @@ const useStyles = makeStyles({
|
||||
|
||||
const NAV: { value: TabValue; label: string; icon: React.JSX.Element }[] = [
|
||||
{ value: 'downloads', label: 'Downloads', icon: <ArrowDownloadRegular /> },
|
||||
{ value: 'library', label: 'Library', icon: <LibraryRegular /> },
|
||||
{ value: 'history', label: 'History', icon: <HistoryRegular /> },
|
||||
{ value: 'settings', label: 'Settings', icon: <SettingsRegular /> }
|
||||
]
|
||||
|
||||
@@ -31,6 +31,7 @@ if (import.meta.env.DEV && !window.api) {
|
||||
customCommandEnabled: false,
|
||||
defaultTemplateId: null,
|
||||
notifyOnComplete: true,
|
||||
autoDownloadNew: true,
|
||||
hasCompletedOnboarding: true
|
||||
}
|
||||
// Stands in for the cookies.txt file's mtime — lets the Cookies card's
|
||||
@@ -139,7 +140,58 @@ if (import.meta.env.DEV && !window.api) {
|
||||
getSystemTheme: async () => ({ shouldUseDarkColors: false, shouldUseHighContrastColors: false }),
|
||||
onSystemThemeUpdate: () => () => {},
|
||||
openHighContrastSettings: async () => {},
|
||||
onExternalUrl: () => () => {}
|
||||
onExternalUrl: () => () => {},
|
||||
// Media-manager sources — a seeded channel so the (Phase H) Library view has
|
||||
// demo data in this browser-only preview.
|
||||
listSources: async () => [
|
||||
{
|
||||
id: 'src-demo',
|
||||
url: 'https://www.youtube.com/@DevChannel',
|
||||
kind: 'channel',
|
||||
title: 'DevChannel',
|
||||
channel: 'DevChannel',
|
||||
addedAt: Date.now() - 86_400_000,
|
||||
lastIndexedAt: Date.now() - 3_600_000,
|
||||
itemCount: 7
|
||||
}
|
||||
],
|
||||
indexSource: async (url) => {
|
||||
await new Promise((r) => setTimeout(r, 800)) // simulate the index walk
|
||||
return {
|
||||
ok: true,
|
||||
source: {
|
||||
id: 'src-demo',
|
||||
url,
|
||||
kind: 'channel',
|
||||
title: 'DevChannel',
|
||||
channel: 'DevChannel',
|
||||
addedAt: Date.now(),
|
||||
lastIndexedAt: Date.now(),
|
||||
itemCount: 7
|
||||
},
|
||||
itemCount: 7
|
||||
}
|
||||
},
|
||||
reindexSource: async () => ({ ok: true, itemCount: 7, newCount: 0 }),
|
||||
removeSource: async () => [],
|
||||
markSourceItemDownloaded: async () => [],
|
||||
setSourceWatched: async () => [],
|
||||
syncSources: async () => ({ ok: true, newItems: [] }),
|
||||
getScheduledSync: async () => ({ enabled: false }),
|
||||
setScheduledSync: async (enabled: boolean) => ({ enabled }),
|
||||
listSourceItems: async (sourceId) =>
|
||||
Array.from({ length: 7 }, (_, i) => ({
|
||||
id: `${sourceId}:vid${i + 1}`,
|
||||
sourceId,
|
||||
videoId: `vid${i + 1}`,
|
||||
title: `Episode ${i + 1}`,
|
||||
url: `https://youtube.com/watch?v=vid${i + 1}`,
|
||||
playlistTitle: i < 5 ? 'Full Electron Course' : 'Uploads',
|
||||
playlistIndex: i < 5 ? i + 1 : i - 4,
|
||||
durationLabel: `${10 + i}:0${i}`,
|
||||
downloaded: i < 2
|
||||
})),
|
||||
onIndexProgress: () => () => {}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import { create } from 'zustand'
|
||||
import type { DownloadEvent, DownloadMeta, DownloadOptions } from '@shared/ipc'
|
||||
import type { DownloadEvent, DownloadMeta, DownloadOptions, CollectionContext } from '@shared/ipc'
|
||||
import { useSettings } from './settings'
|
||||
import { useHistory } from './history'
|
||||
import { useSources } from './sources'
|
||||
|
||||
export type MediaKind = 'video' | 'audio'
|
||||
|
||||
@@ -42,6 +43,10 @@ export interface DownloadItem {
|
||||
incognito?: boolean
|
||||
/** metadata already fetched at probe time; forwarded to main so it skips a redundant probe */
|
||||
probedMeta?: DownloadMeta
|
||||
/** media-manager folder context — files this into <channel>/<playlist>/<NNN> - <title> */
|
||||
collection?: CollectionContext
|
||||
/** when set, the source MediaItem id this download came from — marked downloaded on completion */
|
||||
mediaItemId?: string
|
||||
}
|
||||
|
||||
export const QUALITY_OPTIONS: Record<MediaKind, string[]> = {
|
||||
@@ -68,6 +73,8 @@ interface DownloadState {
|
||||
options?: DownloadOptions
|
||||
extraArgs?: string
|
||||
incognito?: boolean
|
||||
collection?: CollectionContext
|
||||
mediaItemId?: string
|
||||
}
|
||||
) => void
|
||||
retry: (id: string) => void
|
||||
@@ -210,6 +217,11 @@ function startFakeTicker(
|
||||
completedAt: Date.now()
|
||||
})
|
||||
}
|
||||
// Mirror the real applyEvent path so collection completions mark their
|
||||
// source item downloaded in the preview too.
|
||||
if (finished?.mediaItemId) {
|
||||
useSources.getState().markDownloaded(finished.mediaItemId, finished.filePath)
|
||||
}
|
||||
get().pump() // a slot just freed — promote the next queued item
|
||||
}
|
||||
}, 650)
|
||||
@@ -240,7 +252,8 @@ export const useDownloads = create<DownloadState>((set, get) => {
|
||||
formatHasAudio: item.formatHasAudio,
|
||||
options: item.options,
|
||||
extraArgs: item.extraArgs,
|
||||
meta: item.probedMeta
|
||||
meta: item.probedMeta,
|
||||
collection: item.collection
|
||||
})
|
||||
.then((res) => {
|
||||
if (!res.ok) markError(item.id, res.error)
|
||||
@@ -301,6 +314,8 @@ export const useDownloads = create<DownloadState>((set, get) => {
|
||||
options: opts?.options,
|
||||
extraArgs: opts?.extraArgs,
|
||||
incognito: opts?.incognito,
|
||||
collection: opts?.collection,
|
||||
mediaItemId: opts?.mediaItemId,
|
||||
probedMeta
|
||||
}
|
||||
set((s) => ({ items: [item, ...s.items] }))
|
||||
@@ -411,6 +426,10 @@ export const useDownloads = create<DownloadState>((set, get) => {
|
||||
completedAt: Date.now()
|
||||
})
|
||||
}
|
||||
// Persist media-manager completion so an incremental re-sync skips it.
|
||||
if (item?.mediaItemId) {
|
||||
useSources.getState().markDownloaded(item.mediaItemId, item.filePath)
|
||||
}
|
||||
}
|
||||
// A finished item frees a slot — promote whatever is queued next.
|
||||
if (ev.type === 'done' || ev.type === 'error') pump()
|
||||
|
||||
@@ -25,6 +25,7 @@ const FALLBACK: Settings = {
|
||||
customCommandEnabled: false,
|
||||
defaultTemplateId: null,
|
||||
notifyOnComplete: true,
|
||||
autoDownloadNew: true,
|
||||
// True in preview so design work isn't blocked behind the welcome screen;
|
||||
// a real first launch gets `false` from main/settings.ts's DEFAULTS instead.
|
||||
hasCompletedOnboarding: PREVIEW
|
||||
|
||||
@@ -0,0 +1,288 @@
|
||||
import { create } from 'zustand'
|
||||
import type { Source, MediaItem, IndexProgress } from '@shared/ipc'
|
||||
import { useDownloads } from './downloads'
|
||||
import { useSettings } from './settings'
|
||||
|
||||
// True in the standalone browser preview (no Electron preload).
|
||||
const PREVIEW = typeof window === 'undefined' || !window.electron
|
||||
|
||||
// Cap on how many items one "Download" click enqueues, so an entire channel
|
||||
// can't flood the live queue at once — the rest stay pending for a follow-up
|
||||
// click. (The automatic incremental feeder is Phase I; see ROADMAP-PINCHFLAT.md.)
|
||||
export const MAX_ENQUEUE_BATCH = 100
|
||||
|
||||
// --- Preview seed data ------------------------------------------------------
|
||||
|
||||
function seedItems(sourceId: string, playlist: string, n: number, downloadedUpTo = 0): MediaItem[] {
|
||||
return Array.from({ length: n }, (_, i) => ({
|
||||
id: `${sourceId}:${playlist}-${i + 1}`,
|
||||
sourceId,
|
||||
videoId: `${playlist}-${i + 1}`,
|
||||
title: `${playlist} — part ${i + 1}`,
|
||||
url: `https://youtube.com/watch?v=${sourceId}${i + 1}`,
|
||||
playlistTitle: playlist,
|
||||
playlistIndex: i + 1,
|
||||
durationLabel: `${10 + i}:${String(i * 7).padStart(2, '0')}`,
|
||||
downloaded: i < downloadedUpTo
|
||||
}))
|
||||
}
|
||||
|
||||
const seedSources: Source[] = PREVIEW
|
||||
? [
|
||||
{
|
||||
id: 'src-dev',
|
||||
url: 'https://www.youtube.com/@DevChannel',
|
||||
kind: 'channel',
|
||||
title: 'DevChannel',
|
||||
channel: 'DevChannel',
|
||||
addedAt: Date.now() - 86_400_000,
|
||||
lastIndexedAt: Date.now() - 3_600_000,
|
||||
itemCount: 9
|
||||
},
|
||||
{
|
||||
id: 'src-mix',
|
||||
url: 'https://www.youtube.com/playlist?list=PLmix',
|
||||
kind: 'playlist',
|
||||
title: 'Lo-fi Coding Mixes',
|
||||
channel: 'ChillStudio',
|
||||
addedAt: Date.now() - 2 * 86_400_000,
|
||||
lastIndexedAt: Date.now() - 7_200_000,
|
||||
itemCount: 5
|
||||
}
|
||||
]
|
||||
: []
|
||||
|
||||
const seedItemsBySource: Record<string, MediaItem[]> = PREVIEW
|
||||
? {
|
||||
'src-dev': [
|
||||
...seedItems('src-dev', 'Full Electron Course', 6, 2),
|
||||
...seedItems('src-dev', 'Uploads', 3)
|
||||
],
|
||||
'src-mix': seedItems('src-mix', 'Lo-fi Coding Mixes', 5, 1)
|
||||
}
|
||||
: {}
|
||||
|
||||
// --- Store ------------------------------------------------------------------
|
||||
|
||||
/** Live indexing status for the add-source bar. */
|
||||
interface IndexingState {
|
||||
active: boolean
|
||||
url?: string
|
||||
message?: string
|
||||
current?: number
|
||||
total?: number
|
||||
}
|
||||
|
||||
interface SourcesState {
|
||||
sources: Source[]
|
||||
/** media items per source, lazily loaded when a source is expanded */
|
||||
itemsBySource: Record<string, MediaItem[]>
|
||||
/** which source is expanded in the library view (null = none) */
|
||||
selectedSourceId: string | null
|
||||
indexing: IndexingState
|
||||
loadSources: () => void
|
||||
selectSource: (id: string | null) => void
|
||||
/** index a pasted channel/playlist URL; resolves with ok + an error message */
|
||||
indexSource: (url: string) => Promise<{ ok: boolean; error?: string }>
|
||||
reindexSource: (id: string) => Promise<void>
|
||||
removeSource: (id: string) => void
|
||||
/**
|
||||
* Enqueue the given media items into the download queue with folder context,
|
||||
* capped at MAX_ENQUEUE_BATCH. Returns how many were actually enqueued.
|
||||
*/
|
||||
enqueueItems: (sourceId: string, items: MediaItem[]) => number
|
||||
/** mark an item downloaded once its queue download completes (called by the downloads store) */
|
||||
markDownloaded: (itemId: string, filePath?: string) => void
|
||||
/** toggle a source's watched-for-new-uploads flag (Phase J) */
|
||||
setWatched: (id: string, watched: boolean) => void
|
||||
/** whether a sync is currently running (the "Check for new" button's busy state) */
|
||||
syncing: boolean
|
||||
/**
|
||||
* Re-index all watched sources and (when autoDownloadNew is on) enqueue the new
|
||||
* videos. Resolves with how many new videos were found.
|
||||
*/
|
||||
syncWatched: () => Promise<number>
|
||||
}
|
||||
|
||||
export const useSources = create<SourcesState>((set, get) => ({
|
||||
sources: seedSources,
|
||||
itemsBySource: seedItemsBySource,
|
||||
selectedSourceId: null,
|
||||
indexing: { active: false },
|
||||
syncing: false,
|
||||
|
||||
loadSources: () => {
|
||||
if (PREVIEW) return
|
||||
window.api
|
||||
.listSources()
|
||||
.then((sources) => set({ sources }))
|
||||
.catch(() => {})
|
||||
},
|
||||
|
||||
selectSource: (id) => {
|
||||
set({ selectedSourceId: id })
|
||||
// Lazily fetch a source's items the first time it's expanded.
|
||||
if (id && !get().itemsBySource[id] && !PREVIEW) {
|
||||
window.api
|
||||
.listSourceItems(id)
|
||||
.then((items) => set((s) => ({ itemsBySource: { ...s.itemsBySource, [id]: items } })))
|
||||
.catch(() => {})
|
||||
}
|
||||
},
|
||||
|
||||
indexSource: async (url) => {
|
||||
set({ indexing: { active: true, url, message: 'Reading source…' } })
|
||||
if (PREVIEW) {
|
||||
await new Promise((r) => setTimeout(r, 900)) // simulate the index walk
|
||||
set({ indexing: { active: false } })
|
||||
return { ok: true }
|
||||
}
|
||||
try {
|
||||
const res = await window.api.indexSource(url)
|
||||
set({ indexing: { active: false } })
|
||||
if (res.ok) {
|
||||
get().loadSources()
|
||||
if (res.source) get().selectSource(res.source.id)
|
||||
}
|
||||
return { ok: res.ok, error: res.error }
|
||||
} catch (e) {
|
||||
set({ indexing: { active: false } })
|
||||
return { ok: false, error: e instanceof Error ? e.message : String(e) }
|
||||
}
|
||||
},
|
||||
|
||||
reindexSource: async (id) => {
|
||||
const src = get().sources.find((s) => s.id === id)
|
||||
set({ indexing: { active: true, url: src?.url, message: 'Re-indexing…' } })
|
||||
if (PREVIEW) {
|
||||
await new Promise((r) => setTimeout(r, 700))
|
||||
set({ indexing: { active: false } })
|
||||
return
|
||||
}
|
||||
try {
|
||||
const res = await window.api.reindexSource(id)
|
||||
set({ indexing: { active: false } })
|
||||
if (res.ok) {
|
||||
get().loadSources()
|
||||
const items = await window.api.listSourceItems(id)
|
||||
set((s) => ({ itemsBySource: { ...s.itemsBySource, [id]: items } }))
|
||||
}
|
||||
} catch {
|
||||
set({ indexing: { active: false } })
|
||||
}
|
||||
},
|
||||
|
||||
removeSource: (id) => {
|
||||
set((s) => ({
|
||||
sources: s.sources.filter((x) => x.id !== id),
|
||||
selectedSourceId: s.selectedSourceId === id ? null : s.selectedSourceId
|
||||
}))
|
||||
if (!PREVIEW) window.api.removeSource(id).catch(() => {})
|
||||
},
|
||||
|
||||
enqueueItems: (sourceId, items) => {
|
||||
const src = get().sources.find((s) => s.id === sourceId)
|
||||
const settings = useSettings.getState()
|
||||
const kind = settings.defaultKind
|
||||
const quality = kind === 'video' ? settings.defaultVideoQuality : settings.defaultAudioQuality
|
||||
const batch = items.slice(0, MAX_ENQUEUE_BATCH)
|
||||
const add = useDownloads.getState().addFromUrl
|
||||
for (const it of batch) {
|
||||
add(it.url, kind, quality, {
|
||||
title: it.title,
|
||||
channel: src?.channel,
|
||||
durationLabel: it.durationLabel,
|
||||
collection: {
|
||||
channel: src?.channel || src?.title || 'Channel',
|
||||
playlist: it.playlistTitle,
|
||||
index: it.playlistIndex
|
||||
},
|
||||
mediaItemId: it.id
|
||||
})
|
||||
}
|
||||
return batch.length
|
||||
},
|
||||
|
||||
markDownloaded: (itemId, filePath) => {
|
||||
set((s) => {
|
||||
const next: Record<string, MediaItem[]> = {}
|
||||
let changed = false
|
||||
for (const [sid, list] of Object.entries(s.itemsBySource)) {
|
||||
if (list.some((m) => m.id === itemId)) {
|
||||
changed = true
|
||||
next[sid] = list.map((m) =>
|
||||
m.id === itemId ? { ...m, downloaded: true, downloadedAt: Date.now(), filePath } : m
|
||||
)
|
||||
} else {
|
||||
next[sid] = list
|
||||
}
|
||||
}
|
||||
return changed ? { itemsBySource: next } : {}
|
||||
})
|
||||
if (!PREVIEW) window.api.markSourceItemDownloaded(itemId, filePath).catch(() => {})
|
||||
},
|
||||
|
||||
setWatched: (id, watched) => {
|
||||
set((s) => ({ sources: s.sources.map((x) => (x.id === id ? { ...x, watched } : x)) }))
|
||||
if (!PREVIEW) window.api.setSourceWatched(id, watched).catch(() => {})
|
||||
},
|
||||
|
||||
syncWatched: async () => {
|
||||
if (get().syncing) return 0
|
||||
set({ syncing: true })
|
||||
try {
|
||||
if (PREVIEW) {
|
||||
await new Promise((r) => setTimeout(r, 700))
|
||||
return 0 // preview has no real feed to poll
|
||||
}
|
||||
const res = await window.api.syncSources()
|
||||
if (!res.ok) return 0
|
||||
get().loadSources()
|
||||
// Refresh any expanded source's items so new videos appear in the tree.
|
||||
const sel = get().selectedSourceId
|
||||
if (sel) {
|
||||
const items = await window.api.listSourceItems(sel)
|
||||
set((s) => ({ itemsBySource: { ...s.itemsBySource, [sel]: items } }))
|
||||
}
|
||||
// Auto-enqueue the new videos, grouped by source, when the setting is on.
|
||||
if (useSettings.getState().autoDownloadNew && res.newItems.length > 0) {
|
||||
const bySource = new Map<string, MediaItem[]>()
|
||||
for (const it of res.newItems) {
|
||||
const arr = bySource.get(it.sourceId) ?? []
|
||||
arr.push(it)
|
||||
bySource.set(it.sourceId, arr)
|
||||
}
|
||||
for (const [sid, items] of bySource) get().enqueueItems(sid, items)
|
||||
}
|
||||
return res.newItems.length
|
||||
} finally {
|
||||
set({ syncing: false })
|
||||
}
|
||||
}
|
||||
}))
|
||||
|
||||
// Load persisted sources on startup, and subscribe to live indexing progress.
|
||||
if (!PREVIEW) {
|
||||
window.api
|
||||
.listSources()
|
||||
.then((sources) => {
|
||||
useSources.setState({ sources })
|
||||
// If any source is watched, kick off a sync shortly after launch (covers the
|
||||
// scheduled `--sync` launch and a normal launch alike).
|
||||
if (sources.some((s) => s.watched)) {
|
||||
setTimeout(() => useSources.getState().syncWatched(), 1500)
|
||||
}
|
||||
})
|
||||
.catch(() => {})
|
||||
window.api.onIndexProgress((p: IndexProgress) => {
|
||||
useSources.setState({
|
||||
indexing: {
|
||||
active: p.phase !== 'done' && p.phase !== 'error',
|
||||
url: p.url,
|
||||
message: p.message,
|
||||
current: p.current,
|
||||
total: p.total
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
+147
-2
@@ -40,7 +40,24 @@ export const IpcChannels = {
|
||||
openHighContrastSettings: 'shell:open-high-contrast-settings',
|
||||
/** main → renderer push channel: a URL handed to AeroFetch from outside the app
|
||||
* (the aerofetch:// protocol, or a .url file via Explorer's "Send to" menu) */
|
||||
externalUrl: 'external-url'
|
||||
externalUrl: 'external-url',
|
||||
// --- Sources & media-manager indexing (Pinchflat-style, see ROADMAP-PINCHFLAT.md) ---
|
||||
sourcesList: 'sources:list',
|
||||
sourceIndex: 'sources:index',
|
||||
sourceReindex: 'sources:reindex',
|
||||
sourceRemove: 'sources:remove',
|
||||
sourceItems: 'sources:items',
|
||||
/** mark a media item downloaded once its collection download completes */
|
||||
sourceItemDownloaded: 'sources:item-downloaded',
|
||||
/** toggle whether a source is watched for new uploads (Phase J) */
|
||||
sourceSetWatched: 'sources:set-watched',
|
||||
/** re-index all watched sources and return the videos that are new */
|
||||
sourcesSync: 'sources:sync',
|
||||
/** get / set the Windows scheduled-sync task (Task Scheduler) */
|
||||
scheduledSyncGet: 'sources:scheduled-sync-get',
|
||||
scheduledSyncSet: 'sources:scheduled-sync-set',
|
||||
/** main → renderer push channel for live indexing progress */
|
||||
indexProgress: 'sources:index-progress'
|
||||
} as const
|
||||
|
||||
/** Sentinel format id meaning "let yt-dlp pick the best video+audio". */
|
||||
@@ -134,6 +151,12 @@ export interface DownloadOptions {
|
||||
embedThumbnail: boolean
|
||||
/** crop embedded audio artwork to a centred square (Seal's "crop artwork") */
|
||||
cropThumbnail: boolean
|
||||
/** write a .info.json metadata sidecar (Jellyfin/Plex/Kodi ingest — Phase K) */
|
||||
writeInfoJson: boolean
|
||||
/** write the thumbnail as a separate image file alongside the video */
|
||||
writeThumbnailFile: boolean
|
||||
/** write the video description to a .description sidecar file */
|
||||
writeDescription: boolean
|
||||
}
|
||||
|
||||
export const DEFAULT_DOWNLOAD_OPTIONS: DownloadOptions = {
|
||||
@@ -149,7 +172,10 @@ export const DEFAULT_DOWNLOAD_OPTIONS: DownloadOptions = {
|
||||
embedChapters: false,
|
||||
embedMetadata: true,
|
||||
embedThumbnail: true,
|
||||
cropThumbnail: false
|
||||
cropThumbnail: false,
|
||||
writeInfoJson: false,
|
||||
writeThumbnailFile: false,
|
||||
writeDescription: false
|
||||
}
|
||||
|
||||
export interface YtdlpVersionResult {
|
||||
@@ -225,6 +251,24 @@ export interface ProbeResult {
|
||||
error?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Media-manager folder context for a collection download (see ROADMAP-PINCHFLAT.md
|
||||
* Phase G). When present on a StartDownloadOptions, the file is filed into
|
||||
* `<outputDir>/<channel>/<playlist>/<NNN> - <title>.<ext>` instead of the flat
|
||||
* filenameTemplate. The directory segments are sanitized for Windows at argv-build
|
||||
* time, and `index` (the 1-based playlist position from the persisted MediaItem)
|
||||
* drives the NNN prefix — not yt-dlp's %(playlist_index)s, which is empty under
|
||||
* the per-video `--no-playlist` path these downloads still take.
|
||||
*/
|
||||
export interface CollectionContext {
|
||||
/** channel / uploader name — the top folder level */
|
||||
channel: string
|
||||
/** playlist title — the second folder level ('Uploads' for the catch-all) */
|
||||
playlist: string
|
||||
/** 1-based position within the playlist, for the NNN filename prefix */
|
||||
index: number
|
||||
}
|
||||
|
||||
/** Sent renderer → main to kick off a download. The renderer owns the id. */
|
||||
export interface StartDownloadOptions {
|
||||
id: string
|
||||
@@ -255,6 +299,12 @@ export interface StartDownloadOptions {
|
||||
* race where the late probe overwrites a good title the renderer supplied.
|
||||
*/
|
||||
meta?: DownloadMeta
|
||||
/**
|
||||
* When set, this is a media-manager (collection) download — file it into
|
||||
* `<channel>/<playlist>/<NNN> - <title>` folders instead of the flat
|
||||
* filenameTemplate. See CollectionContext.
|
||||
*/
|
||||
collection?: CollectionContext
|
||||
}
|
||||
|
||||
export interface StartDownloadResult {
|
||||
@@ -327,6 +377,8 @@ export interface Settings {
|
||||
defaultTemplateId: string | null
|
||||
/** show a native OS notification when a download finishes or fails */
|
||||
notifyOnComplete: boolean
|
||||
/** auto-enqueue newly-found videos from watched sources during a sync (Phase J) */
|
||||
autoDownloadNew: boolean
|
||||
/** false until the first-run welcome screen has been dismissed */
|
||||
hasCompletedOnboarding: boolean
|
||||
}
|
||||
@@ -410,3 +462,96 @@ export interface BackupImportResult {
|
||||
ok: boolean
|
||||
error?: string
|
||||
}
|
||||
|
||||
// --- Sources & media-manager indexing (Pinchflat-style) ---------------------
|
||||
// See ROADMAP-PINCHFLAT.md. A Source (channel/playlist) is indexed once into a
|
||||
// persisted list of MediaItem records; the live download queue then pulls from
|
||||
// that list a batch at a time, so "download an entire channel" never has to hold
|
||||
// the whole collection in the queue at once.
|
||||
|
||||
/** Whether a Source is a whole channel or a single playlist. */
|
||||
export type SourceKind = 'channel' | 'playlist'
|
||||
|
||||
/**
|
||||
* A monitored channel or playlist that AeroFetch has indexed. Its full video
|
||||
* list lives as MediaItem records (one JSON store), keyed back by `id`.
|
||||
*/
|
||||
export interface Source {
|
||||
id: string
|
||||
/** the URL the user added (used for re-indexing) */
|
||||
url: string
|
||||
kind: SourceKind
|
||||
title: string
|
||||
/** channel / uploader name, when known */
|
||||
channel?: string
|
||||
/** epoch ms the source was first added */
|
||||
addedAt: number
|
||||
/** epoch ms of the last successful (re)index; undefined if never indexed */
|
||||
lastIndexedAt?: number
|
||||
/** number of MediaItems at the last index (cached for list display) */
|
||||
itemCount: number
|
||||
/** watched for new uploads — included in scheduled / startup sync (Phase J) */
|
||||
watched?: boolean
|
||||
/** YouTube RSS feed URL for cheap "anything new?" checks; undefined if unknown */
|
||||
feedUrl?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* One video discovered while indexing a Source. `playlistTitle`/`playlistIndex`
|
||||
* drive the on-disk folder layout (<Channel>/<Playlist>/<NNN> - <Title>); the
|
||||
* `downloaded` flag + `filePath` let the library view show per-item state and
|
||||
* let re-syncs skip what's already on disk.
|
||||
*/
|
||||
export interface MediaItem {
|
||||
/** globally unique, formed as `${sourceId}:${videoId}` */
|
||||
id: string
|
||||
sourceId: string
|
||||
/** the yt-dlp video id — the dedup key within a source */
|
||||
videoId: string
|
||||
title: string
|
||||
url: string
|
||||
/** the playlist this item is filed under; 'Uploads' for videos in no playlist */
|
||||
playlistTitle: string
|
||||
/** 1-based position within its playlist, for NNN numbering */
|
||||
playlistIndex: number
|
||||
durationLabel?: string
|
||||
downloaded: boolean
|
||||
downloadedAt?: number
|
||||
filePath?: string
|
||||
}
|
||||
|
||||
/** Live progress pushed while a Source is being indexed (main → renderer). */
|
||||
export interface IndexProgress {
|
||||
/** the Source URL being indexed (correlates events back to the request) */
|
||||
url: string
|
||||
phase: 'start' | 'playlists' | 'playlist' | 'uploads' | 'done' | 'error'
|
||||
/** human-readable status line */
|
||||
message: string
|
||||
/** for the per-playlist phase: playlists processed so far / total */
|
||||
current?: number
|
||||
total?: number
|
||||
}
|
||||
|
||||
/** Result of indexing (or re-indexing) a Source. */
|
||||
export interface IndexSourceResult {
|
||||
ok: boolean
|
||||
source?: Source
|
||||
itemCount?: number
|
||||
/** how many videos were new since the previous index (all of them on first index) */
|
||||
newCount?: number
|
||||
error?: string
|
||||
}
|
||||
|
||||
/** Result of syncing all watched sources: the videos found new across them. */
|
||||
export interface SyncResult {
|
||||
ok: boolean
|
||||
/** newly-discovered media items across all watched sources (empty if nothing new) */
|
||||
newItems: MediaItem[]
|
||||
error?: string
|
||||
}
|
||||
|
||||
/** Whether the Windows scheduled-sync task is currently registered. */
|
||||
export interface ScheduledSyncStatus {
|
||||
enabled: boolean
|
||||
error?: string
|
||||
}
|
||||
|
||||
@@ -3,6 +3,8 @@ import {
|
||||
buildArgs,
|
||||
parseExtraArgs,
|
||||
formatCommandLine,
|
||||
sanitizeDirSegment,
|
||||
collectionOutputTemplate,
|
||||
CROP_SQUARE_PPA,
|
||||
ARIA2C_ARGS,
|
||||
type AccessOptions
|
||||
@@ -360,3 +362,91 @@ describe('buildArgs extraArgs', () => {
|
||||
expect(withEmpty).toEqual(withoutParam)
|
||||
})
|
||||
})
|
||||
|
||||
// --- Media-server sidecar files (Phase K) -----------------------------------
|
||||
|
||||
describe('sidecar files', () => {
|
||||
it('emits --write-info-json / --write-thumbnail / --write-description when enabled', () => {
|
||||
const argv = build(
|
||||
opts(),
|
||||
dlo({ writeInfoJson: true, writeThumbnailFile: true, writeDescription: true })
|
||||
)
|
||||
expect(argv).toContain('--write-info-json')
|
||||
expect(argv).toContain('--write-thumbnail')
|
||||
expect(argv).toContain('--write-description')
|
||||
})
|
||||
|
||||
it('emits none of them by default', () => {
|
||||
const argv = build(opts(), dlo())
|
||||
expect(argv).not.toContain('--write-info-json')
|
||||
expect(argv).not.toContain('--write-description')
|
||||
// (--write-thumbnail is sidecar-only here; embedThumbnail uses --embed-thumbnail)
|
||||
expect(argv).not.toContain('--write-thumbnail')
|
||||
})
|
||||
})
|
||||
|
||||
// --- Collection folder paths (Phase G, media-manager) -----------------------
|
||||
|
||||
describe('sanitizeDirSegment', () => {
|
||||
it('keeps an ordinary name (incl. spaces and hyphens) intact', () => {
|
||||
expect(sanitizeDirSegment('Linus Tech Tips')).toBe('Linus Tech Tips')
|
||||
expect(sanitizeDirSegment('Two-Minute Papers')).toBe('Two-Minute Papers')
|
||||
})
|
||||
|
||||
it('replaces Windows-illegal characters with a space and collapses runs', () => {
|
||||
expect(sanitizeDirSegment('A/B\\C:D*E?F')).toBe('A B C D E F')
|
||||
expect(sanitizeDirSegment('Best of 2024 <Live>')).toBe('Best of 2024 Live')
|
||||
})
|
||||
|
||||
it('strips leading/trailing dots and spaces, neutralising `..` traversal', () => {
|
||||
expect(sanitizeDirSegment('..')).toBe('Untitled')
|
||||
expect(sanitizeDirSegment('../../etc')).toBe('etc')
|
||||
expect(sanitizeDirSegment(' trailing. ')).toBe('trailing')
|
||||
expect(sanitizeDirSegment('My Playlist.')).toBe('My Playlist')
|
||||
})
|
||||
|
||||
it('prefixes reserved Windows device names', () => {
|
||||
expect(sanitizeDirSegment('CON')).toBe('_CON')
|
||||
expect(sanitizeDirSegment('com1')).toBe('_com1')
|
||||
})
|
||||
|
||||
it('falls back to Untitled for empty / all-illegal input', () => {
|
||||
expect(sanitizeDirSegment('')).toBe('Untitled')
|
||||
expect(sanitizeDirSegment('???')).toBe('Untitled')
|
||||
})
|
||||
|
||||
it('caps length to keep the overall path bounded', () => {
|
||||
expect(sanitizeDirSegment('x'.repeat(200)).length).toBe(80)
|
||||
})
|
||||
})
|
||||
|
||||
describe('collectionOutputTemplate', () => {
|
||||
const ctx = { channel: 'DevChannel', playlist: 'Full Course', index: 3 }
|
||||
|
||||
it('files into <out>/<channel>/<playlist>/<NNN> - <filename>', () => {
|
||||
const t = collectionOutputTemplate('C:/out', ctx, '%(title)s.%(ext)s')
|
||||
// normalise separators so the assertion is OS-independent
|
||||
expect(t.replace(/\\/g, '/')).toBe('C:/out/DevChannel/Full Course/003 - %(title)s.%(ext)s')
|
||||
})
|
||||
|
||||
it('zero-pads the index to three digits and clamps bad values to 001', () => {
|
||||
expect(collectionOutputTemplate('C:/o', { ...ctx, index: 42 }, 'f').replace(/\\/g, '/')).toContain(
|
||||
'/042 - f'
|
||||
)
|
||||
expect(collectionOutputTemplate('C:/o', { ...ctx, index: 0 }, 'f').replace(/\\/g, '/')).toContain(
|
||||
'/001 - f'
|
||||
)
|
||||
expect(
|
||||
collectionOutputTemplate('C:/o', { ...ctx, index: NaN }, 'f').replace(/\\/g, '/')
|
||||
).toContain('/001 - f')
|
||||
})
|
||||
|
||||
it('sanitizes the channel and playlist folder names', () => {
|
||||
const t = collectionOutputTemplate(
|
||||
'C:/out',
|
||||
{ channel: 'A/B', playlist: '..', index: 1 },
|
||||
'%(title)s.%(ext)s'
|
||||
)
|
||||
expect(t.replace(/\\/g, '/')).toBe('C:/out/A B/Untitled/001 - %(title)s.%(ext)s')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
import { describe, it, expect } from 'vitest'
|
||||
import {
|
||||
classifySource,
|
||||
buildMediaItems,
|
||||
mergeItemsPreservingState,
|
||||
buildFeedUrl,
|
||||
parseRssVideoIds,
|
||||
entryUrl,
|
||||
fmtDuration,
|
||||
stripTabSuffix,
|
||||
stableSourceId,
|
||||
type RawEntry
|
||||
} from '../src/main/indexerCore'
|
||||
import type { MediaItem } from '@shared/ipc'
|
||||
|
||||
describe('classifySource', () => {
|
||||
it('classifies channel handle / id / c / user forms and strips the tab', () => {
|
||||
expect(classifySource('https://www.youtube.com/@LinusTechTips')).toEqual({
|
||||
kind: 'channel',
|
||||
base: 'https://www.youtube.com/@LinusTechTips'
|
||||
})
|
||||
expect(classifySource('https://youtube.com/@Foo/videos')).toEqual({
|
||||
kind: 'channel',
|
||||
base: 'https://www.youtube.com/@Foo'
|
||||
})
|
||||
expect(classifySource('https://www.youtube.com/channel/UC123/playlists')?.base).toBe(
|
||||
'https://www.youtube.com/channel/UC123'
|
||||
)
|
||||
expect(classifySource('https://www.youtube.com/c/SomeName')?.kind).toBe('channel')
|
||||
expect(classifySource('https://www.youtube.com/user/Legacy')?.kind).toBe('channel')
|
||||
})
|
||||
|
||||
it('classifies a playlist by its list= param and canonicalises it', () => {
|
||||
expect(classifySource('https://www.youtube.com/playlist?list=PL_abc')).toEqual({
|
||||
kind: 'playlist',
|
||||
base: 'https://www.youtube.com/playlist?list=PL_abc'
|
||||
})
|
||||
// a watch URL that also carries list= is treated as the playlist
|
||||
expect(classifySource('https://www.youtube.com/watch?v=xyz&list=PLfoo')?.kind).toBe('playlist')
|
||||
})
|
||||
|
||||
it('returns null for a lone video, a bad URL, or a non-YouTube host', () => {
|
||||
expect(classifySource('https://www.youtube.com/watch?v=dQw4w9WgXcQ')).toBeNull()
|
||||
expect(classifySource('not a url')).toBeNull()
|
||||
expect(classifySource('ftp://example.com/@chan')).toBeNull()
|
||||
expect(classifySource('https://vimeo.com/channels/staffpicks')).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('entryUrl', () => {
|
||||
it('prefers an explicit http(s) url, else builds a watch url from the id', () => {
|
||||
expect(entryUrl({ url: 'https://youtu.be/abc' })).toBe('https://youtu.be/abc')
|
||||
expect(entryUrl({ webpage_url: 'https://x/y' })).toBe('https://x/y')
|
||||
expect(entryUrl({ id: 'vid123' })).toBe('https://www.youtube.com/watch?v=vid123')
|
||||
})
|
||||
it('rejects a non-http url and returns null when nothing usable is present', () => {
|
||||
expect(entryUrl({ url: 'javascript:alert(1)' })).toBeNull()
|
||||
expect(entryUrl({})).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('fmtDuration', () => {
|
||||
it('formats seconds as M:SS or H:MM:SS', () => {
|
||||
expect(fmtDuration(0)).toBe('0:00')
|
||||
expect(fmtDuration(75)).toBe('1:15')
|
||||
expect(fmtDuration(3661)).toBe('1:01:01')
|
||||
expect(fmtDuration(undefined)).toBeUndefined()
|
||||
expect(fmtDuration(NaN)).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
describe('stripTabSuffix', () => {
|
||||
it('strips a trailing " - <Tab>" suffix only', () => {
|
||||
expect(stripTabSuffix('Creator - Videos')).toBe('Creator')
|
||||
expect(stripTabSuffix('Creator - Playlists')).toBe('Creator')
|
||||
expect(stripTabSuffix('Just A Name')).toBe('Just A Name')
|
||||
expect(stripTabSuffix('Tech - Tips')).toBe('Tech - Tips') // 'Tips' isn't a tab
|
||||
expect(stripTabSuffix(undefined)).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
describe('stableSourceId', () => {
|
||||
it('is deterministic and 8 hex chars', () => {
|
||||
const a = stableSourceId('https://www.youtube.com/@Foo')
|
||||
expect(a).toMatch(/^[0-9a-f]{8}$/)
|
||||
expect(stableSourceId('https://www.youtube.com/@Foo')).toBe(a)
|
||||
expect(stableSourceId('https://www.youtube.com/@Bar')).not.toBe(a)
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildMediaItems', () => {
|
||||
const vids = (...ids: string[]): RawEntry[] => ids.map((id) => ({ id, title: `T-${id}` }))
|
||||
|
||||
it('files videos under their playlist with 1-based per-playlist index', () => {
|
||||
const items = buildMediaItems('src1', [{ title: 'Series A', entries: vids('a', 'b') }], [])
|
||||
expect(items).toHaveLength(2)
|
||||
expect(items[0]).toMatchObject({
|
||||
id: 'src1:a',
|
||||
sourceId: 'src1',
|
||||
videoId: 'a',
|
||||
playlistTitle: 'Series A',
|
||||
playlistIndex: 1,
|
||||
downloaded: false
|
||||
})
|
||||
expect(items[1]).toMatchObject({ videoId: 'b', playlistIndex: 2 })
|
||||
})
|
||||
|
||||
it('dedups across playlists — the first playlist a video appears in wins', () => {
|
||||
const items = buildMediaItems(
|
||||
'src1',
|
||||
[
|
||||
{ title: 'First', entries: vids('x', 'y') },
|
||||
{ title: 'Second', entries: vids('y', 'z') }
|
||||
],
|
||||
[]
|
||||
)
|
||||
expect(items.map((i) => i.videoId)).toEqual(['x', 'y', 'z'])
|
||||
expect(items.find((i) => i.videoId === 'y')?.playlistTitle).toBe('First')
|
||||
})
|
||||
|
||||
it('falls back to the synthetic Uploads folder for videos in no playlist', () => {
|
||||
const items = buildMediaItems('src1', [{ title: 'P', entries: vids('a') }], vids('a', 'b'))
|
||||
// 'a' already filed under P; only 'b' becomes an Uploads item
|
||||
expect(items).toHaveLength(2)
|
||||
expect(items.find((i) => i.videoId === 'a')?.playlistTitle).toBe('P')
|
||||
const b = items.find((i) => i.videoId === 'b')
|
||||
expect(b).toMatchObject({ playlistTitle: 'Uploads', playlistIndex: 2 })
|
||||
})
|
||||
|
||||
it('skips entries with no id or no resolvable URL', () => {
|
||||
const items = buildMediaItems('src1', [{ title: 'P', entries: [{ title: 'no id' }] }], [])
|
||||
expect(items).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('mergeItemsPreservingState', () => {
|
||||
const item = (videoId: string, over: Partial<MediaItem> = {}): MediaItem => ({
|
||||
id: `s:${videoId}`,
|
||||
sourceId: 's',
|
||||
videoId,
|
||||
title: videoId,
|
||||
url: `https://youtube.com/watch?v=${videoId}`,
|
||||
playlistTitle: 'P',
|
||||
playlistIndex: 1,
|
||||
downloaded: false,
|
||||
...over
|
||||
})
|
||||
|
||||
it('carries downloaded state forward for videos already on disk', () => {
|
||||
const existing = [item('a', { downloaded: true, downloadedAt: 111, filePath: 'C:/a.mp4' })]
|
||||
const fresh = [item('a'), item('b')]
|
||||
const { items, newCount } = mergeItemsPreservingState(existing, fresh)
|
||||
expect(newCount).toBe(1) // only 'b' is new
|
||||
expect(items.find((i) => i.videoId === 'a')).toMatchObject({
|
||||
downloaded: true,
|
||||
downloadedAt: 111,
|
||||
filePath: 'C:/a.mp4'
|
||||
})
|
||||
expect(items.find((i) => i.videoId === 'b')?.downloaded).toBe(false)
|
||||
})
|
||||
|
||||
it('adopts the fresh membership/order and drops vanished videos', () => {
|
||||
const existing = [item('a', { downloaded: true }), item('gone', { downloaded: true })]
|
||||
const fresh = [item('a', { playlistTitle: 'Moved', playlistIndex: 5 })]
|
||||
const { items, newCount } = mergeItemsPreservingState(existing, fresh)
|
||||
expect(items).toHaveLength(1) // 'gone' dropped
|
||||
expect(newCount).toBe(0)
|
||||
// fresh playlist assignment wins, but downloaded state is preserved
|
||||
expect(items[0]).toMatchObject({ playlistTitle: 'Moved', playlistIndex: 5, downloaded: true })
|
||||
})
|
||||
|
||||
it('counts every video as new on a first index (empty existing)', () => {
|
||||
const { newCount } = mergeItemsPreservingState([], [item('a'), item('b'), item('c')])
|
||||
expect(newCount).toBe(3)
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildFeedUrl', () => {
|
||||
it('builds a channel feed by channel_id and a playlist feed by playlist_id', () => {
|
||||
expect(buildFeedUrl('channel', 'UCabc')).toBe(
|
||||
'https://www.youtube.com/feeds/videos.xml?channel_id=UCabc'
|
||||
)
|
||||
expect(buildFeedUrl('playlist', 'PLxyz')).toBe(
|
||||
'https://www.youtube.com/feeds/videos.xml?playlist_id=PLxyz'
|
||||
)
|
||||
})
|
||||
it('returns undefined when the id is missing', () => {
|
||||
expect(buildFeedUrl('channel', undefined)).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
describe('parseRssVideoIds', () => {
|
||||
it('extracts every <yt:videoId> from a feed body, in order', () => {
|
||||
const xml = `
|
||||
<feed><entry><yt:videoId>aaa111</yt:videoId></entry>
|
||||
<entry><yt:videoId> bbb222 </yt:videoId></entry></feed>`
|
||||
expect(parseRssVideoIds(xml)).toEqual(['aaa111', 'bbb222'])
|
||||
})
|
||||
it('returns an empty array for a feed with no entries', () => {
|
||||
expect(parseRssVideoIds('<feed></feed>')).toEqual([])
|
||||
})
|
||||
})
|
||||
Reference in New Issue
Block a user