1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
|
.Dd August 28, 2026
.Dt OBSERVATIONS 1
.Os
.Sh NAME
.Nm observations
.Nd publish audio readings of other people's texts
.Sh SYNOPSIS
.Nm
.Cm add
.Op Fl -episodes Ar dir
.Op Fl -audio Ar dir
.Op Fl -out Ar dir
.Fl -source-title Ar title
.Fl -source-url Ar url
.Fl -source-author Ar author
.Fl -source-published Ar date
.Op Fl -source-site Ar site
.Op Fl -source-note Ar note
.Op Fl -title Ar title
.Op Fl -recorded Ar date
.Op Fl -notes Ar text
.Op Fl -number Ar n
.Op Fl -ffprobe Ar path
.Op Fl -ffmpeg Ar path
.Ar recording
.Nm
.Cm render
.Op Fl -episodes Ar dir
.Fl -out Ar dir
.Nm
.Cm list
.Op Fl -episodes Ar dir
.Nm
.Cm transcript
.Op Fl -format Ar what
.Op Fl -ffprobe Ar path
.Op Fl -ffmpeg Ar path
.Ar recording
.Nm
.Cm listen
.Op Fl -episodes Ar dir
.Op Fl -audio Ar dir
.Op Fl -gemini-key-file Ar path
.Op Fl -pad Ar seconds
.Op Fl -no-gemini-call
.Op Fl -ffmpeg Ar path
.Ar episode from to
.Nm
.Cm convert
.Op Fl -episodes Ar dir
.Op Fl -force
.Sh DESCRIPTION
.Nm
ingests a recording of someone reading a text out loud and renders a static
website and podcast feed from it.
For what the format is and why the pipeline is shaped this way, see
.Xr observations 7 .
.Ss add
Ingests
.Ar recording ,
which must be the file as it came off the recorder.
.Pp
The recording is probed for its duration and byte size (both needed by the
feed), the transcript track is extracted and decoded if there is one, the
audio is copied to the audio directory as
.Pa NNN.m4a ,
and the metadata is written to
.Pa episodes/NNN.json .
.Pp
.Cm add
records
.Sy one
source, which is the common case.
An episode that reads several texts gets the rest by appending them to the
.Ql sources
array in
.Pa episodes/NNN.json
and running
.Cm render
again; give it a
.Fl -title
at ingest, since such an episode is not named after any one of its texts.
The flags are deliberately not repeatable: zipping parallel flag lists by
position misaligns silently as soon as one source has a
.Fl -source-site
and another does not, and quietly attributing a text to the wrong author is a
worse failure than typing six lines of JSON.
.Bl -tag -width "-source-published date"
.It Fl -source-title Ar title
Title of the text being read.
Required.
.It Fl -source-url Ar url
URL of the text being read.
Must be http or https.
Required.
.It Fl -source-author Ar author
Who wrote the text.
Required.
.It Fl -source-published Ar date
When the text was published, as
.Ar YYYY-MM-DD .
Required.
.It Fl -source-site Ar site
The publication the text appeared in, when it differs from the author.
.It Fl -source-note Ar note
What this text is doing in the episode \(en
.Qq intro only ,
.Qq the main piece .
Only useful for an episode that reads more than one.
.It Fl -title Ar title
Title of the episode.
Defaults to the title of its source, which is what a single-source episode
wants.
Set this for an episode built around a subject rather than one text.
.It Fl -recorded Ar date
Date of the recording, as
.Ar YYYY-MM-DD .
This is also the publication date of the episode.
Defaults to the recording's modification time.
.It Fl -notes Ar text
Prose shown on the episode page and included in the feed entry.
.It Fl -number Ar n
Episode number.
Defaults to one past the highest existing episode.
.It Fl -ffprobe Ar path , Fl -ffmpeg Ar path
Paths to the ffmpeg tools.
Default to
.Ic ffprobe
and
.Ic ffmpeg
on
.Ev PATH .
.El
.Ss render
Writes the site into the
.Fl -out
directory: an
.Pa index.html ,
a
.Pa NNN/index.html
per episode, and
.Pa feed.xml .
An episode page carries the player, the sources, the submission form and the
episode's transcript, when it has one; the feed carries the summary but not
the transcript.
What the page does with the transcript, and why, is in
.Xr observations 7 .
.Pp
Rendering is a pure function of the episodes directory and can be re-run at
any time.
.Fl -out
is required.
.Ss list
Prints one line per episode: number, recording date, running time, transcript
word count and source title.
.Ss transcript
Prints the transcript embedded in
.Ar recording ,
without ingesting it.
Reads the file and writes nothing, so it is safe to point at anything,
including an episode already published.
.Bl -tag -width "-format what"
.It Fl -format Ar what
.Bl -tag -width transcript -compact
.It Cm transcript
The stored format, on standard output.
The default.
.It Cm text
The transcript as prose, for reading or grepping.
.It Cm streams
What the container holds: duration, audio codec, and every data track with
the transcript marked.
This is what answers
.Dq why is the transcript missing .
.El
.It Fl -ffprobe Ar path , Fl -ffmpeg Ar path
Paths to the ffmpeg tools, as for
.Cm add .
.El
.Pp
Exits non-zero when the recording carries no transcript, so it can gate a
script that is about to overwrite the original.
.Ss listen
Prints what a speech model hears in one passage of an episode, beside what the
source says and what the transcript currently says.
It is for the passages a correction pass cannot settle from the text \(en see
.Xr observations-transcripts 7 ,
which is where the judgement rules live.
.Pp
.Ar from
and
.Ar to
may be written as
.Ar HH:MM:SS.mmm
as the transcript stores them,
.Ar MM:SS
as a player shows them, or as bare seconds.
.Ar episode
may be written 001 or 1.
.Pp
Three views are printed, none of them authoritative:
.Bl -tag -width TRANSCRIPT -compact
.It Cm GEMINI
What the model heard.
It is a second recogniser, wrong in different places than the first.
.It Cm SOURCE
The source's own wording, when the span falls inside a
.Ic .quote .
Where there is one this settles the passage and the model is not needed.
.It Cm TRANSCRIPT
What the file says now, printed last and labelled, because it is the thing
being checked.
.El
.Pp
It never reports a timestamp.
The model answers to the millisecond and is late by 250 to 450 ms, which is
worse than interpolating within a line; a timestamp for splitting a line comes
from the per-word data in git history
.Pq Xr observations-transcripts 7 .
.Pp
Reads the episode and its audio and writes nothing.
This is the only subcommand that uses the network or wants a credential;
.Cm add ,
.Cm render
and
.Cm list
never reach it.
.Bl -tag -width "-gemini-key-file path"
.It Fl -gemini-key-file Ar path
File holding a Gemini API key.
Without it the key is taken from
.Ev GEMINI_API_KEY ,
and failing that from
.Xr pass 1 .
The password store is tried last because it is the one source that can block
on a
.Xr gpg-agent 1
prompt.
Consent is not at issue here, unlike
.Xr observations-inbox 1 :
the audio is a published episode, not someone else's voice.
.It Fl -pad Ar seconds
Audio to include either side of the span, default 2.
A word at the very edge of a clip is the one most likely to be misheard.
The padding is stated in the output, so the extra words are known to be
context rather than part of the passage.
.It Fl -no-gemini-call
Print the source and transcript views only.
Needs no key, and answers
.Dq what does the article actually say here
on its own.
.It Fl -ffmpeg Ar path
Path to ffmpeg, used to cut the clip.
.El
.Ss convert
Migrates
.Pa episodes/NNN.transcript.json ,
the shape the recogniser's protobuf decoded into, to
.Pa episodes/NNN.transcript ,
the shape people edit.
.Pp
This is a one-off: it exists for the episodes ingested before the stored
format changed, and is kept because it is the only route back should a
conversion ever have to be redone from the JSON in git history.
It refuses to overwrite an existing transcript unless given
.Fl -force ,
since that transcript has probably been corrected by hand and the JSON has
not.
.Ss Common flags
.Bl -tag -width "-episodes dir"
.It Fl -episodes Ar dir
Episode metadata, which lives in git.
Default
.Pa episodes .
.It Fl -audio Ar dir
Audio files, which do
.Sy not
live in git.
Default
.Pa audio .
.It Fl -out Ar dir
Where to render the site.
For
.Cm add
this is optional and renders after ingesting.
.El
.Sh FILES
.Bl -tag -width "users/Profpatsch/web/observations/"
.It Pa episodes/NNN.json
One episode: its optional title, the
.Ql sources
array, and the audio's size and duration.
A couple of dozen lines, meant to be read and hand-edited; this is the place
to add a second source.
Committed to git.
.It Pa episodes/NNN.transcript
The transcript for that episode: one timed line of speech per subtitle-shaped
line, with paragraph breaks and quotations on their own.
In its own file because it is hundreds of lines and would otherwise bury the
metadata above in every diff.
Written by
.Cm add ,
then corrected by hand \(en raw recogniser output has a mishearing in every
proper noun.
See
.Xr observations-transcripts 5
for the format and
.Xr observations-transcripts 7
for how to correct one.
Committed to git.
.It Pa audio/NNN.m4a
The recording, byte for byte as it came off the recorder.
Never committed; see
.Xr observations 7 .
.It Pa users/Profpatsch/web/observations/
Rendered site, committed and published with the rest of the website.
.Sy Generated \(en do not edit .
Change the episode JSON or the transcript for content, or the templates and
the
.Va styleCSS ,
.Va replyCSS
and
.Va transcriptCSS
stylesheets in
.Pa render.go
for layout, then re-run
.Cm render .
Every generated file says so in a comment at the top.
The page embeds the transcript, so a correction to
.Pa episodes/NNN.transcript
does not reach the site until it is rendered again.
.It Pa /var/lib/observations/audio/
Where the audio is served from on the server.
.El
.Sh EXAMPLES
Check that a recording still carries its transcript, before anything is done
to it:
.Bd -literal -offset indent
observations transcript --format streams ~/Downloads/recording.m4a
.Ed
.Pp
Ingest a recording and render the site:
.Bd -literal -offset indent
observations add \e
--source-title "Why We're Dropping Basecamp" \e
--source-url https://blogs.library.duke.edu/blog/2023/11/30/why-were-dropping-basecamp/ \e
--source-author "Will Sexton" \e
--source-site "Duke University Libraries Blog" \e
--source-published 2023-11-30 \e
--out ../web/observations \e
~/Downloads/recording.m4a
.Ed
.Pp
An episode built around a subject, reading two texts.
Ingest names the first and gives the episode its own title:
.Bd -literal -offset indent
observations add \e
--title "Commoditizing Your Complement" \e
--source-title "Laws of Tech: Commoditize Your Complement" \e
--source-url https://gwern.net/complement \e
--source-author "Gwern Branwen" \e
--source-site gwern.net \e
--source-published 2018-03-17 \e
--source-note "intro only" \e
~/Downloads/recording.m4a
.Ed
.Pp
then the second is appended to
.Ql sources
in
.Pa episodes/002.json :
.Bd -literal -offset indent
{
"title": "Strategy Letter V: The Economics of Open Source",
"url": "https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/",
"author": "Joel Spolsky",
"published": "2002-06-12",
"site": "Joel on Software",
"note": "the main piece"
}
.Ed
.Pp
Publish it.
The two halves go by two different routes, because the audio is not in git:
.Bd -literal -offset indent
git add users/Profpatsch/observations users/Profpatsch/web/observations
git commit -m "observations: episode 001"
git push legosi-web canon
rsync -av users/Profpatsch/observations/audio/ \e
root@legosi:/var/lib/observations/audio/
.Ed
.Sh SEE ALSO
.Xr ffmpeg 1 ,
.Xr pass 1 ,
.Xr observations-transcripts 5 ,
.Xr observations 7 ,
.Xr observations-transcripts 7
.Sh AUTHORS
.An Profpatsch
.Sh CAVEATS
The audio directory is deliberately outside git, which means it is
.Sy not backed up by pushing .
The recordings are the one part of an episode that cannot be regenerated;
keep them somewhere durable.
.Pp
The feed carries no
.Li itunes:image ,
because there is no cover art yet.
The feed is valid and works in podcast clients, but Apple Podcasts will not
accept a directory submission without artwork.
.Pp
Transcripts are recogniser output as it came out of the recording, and stay
that way until someone corrects them by hand.
An uncorrected one mishears every proper noun in the episode and punctuates
by guessing at intonation; a corrected one has been checked against the
sources it quotes.
.Cm render
publishes whichever of the two it finds, with nothing on the page marking
which it is, so an episode whose transcript has not been through a correction
pass publishes its mishearings under the names of the people it quotes.
See
.Xr observations-transcripts 7
for what correcting one involves and what must not be done to it.
.Pp
Episode numbers and the URLs derived from them are permanent: they appear in
feed GUIDs, and podcast clients remember them.
Renumbering a published episode makes clients treat it as a new one.
|