1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428

.Dd September 3, 2026
.Dt OBSERVATIONS 7
.Os
.Sh NAME
.Nm observations
.Nd an audio format for reading someone else's text out loud, and thinking about it
.Sh DESCRIPTION
This page is the reasoning behind
.Xr observations 1 ,
which is the reference: the flags, the files and the caveats are there, and
nothing here is needed to publish an episode.
What is here is why the format and the publishing pipeline are shaped the way
they are \(en the constraints that would otherwise be rediscovered by
experiment, and the one data-loss trap that is easy to walk into.
.Sh WHAT AN OBSERVATION IS
An observation is a recording of one person reading a short text \(en an
essay, a blog post, a piece of documentation \(en and thinking out loud about
it while doing so.
It is deliberately not a podcast in the interview or panel sense: there is no
guest, no segment structure, and no attempt at production value.
.Pp
The form is fixed by three properties, each of which has consequences for the
software:
.Bl -tag -width indent
.It The text is someone else's
An episode is always
.Em about
one or more specific sources, each identified by a URL, a title, an author and
a publication date.
The sources are the anchor: the episode page links to them and the feed names
them.
This is why a source has four required fields and not a free-form
description, and why the episode title normally carries the source's own title
rather than an invented one \(en the reader should be able to find the thing
being read.
.Pp
Some episodes are built around a subject rather than a single text, and work
through several in one sitting: a short one to introduce the idea, then the
piece that treats it properly.
Such an episode has a title of its own, because neither source's title is an
honest name for it, and each source carries a note saying what it is doing
there.
The single-source case stays the default, and writes no title at all.
.It The commentary is unscripted
The value of the form is that reactions happen while reading, not afterwards
in an edit.
That means recordings are long, single-take, and not worth editing.
A 30-minute episode is normal, and the recording is the artefact rather than
raw material for one.
.It One voice, recorded on a phone
There is no studio and no multitrack session.
The input to the pipeline is whatever the phone produced.
.El
.Sh THE ARTEFACT IS THE RECORDING
.Nm
serves the file that came off the recording device, unmodified.
It does not transcode, normalise, trim, or re-tag it.
.Pp
This is not laziness, it is the consequence of the input.
The recorder produces HE-AAC mono at 48 kbit/s \(en already speech-tuned and
already small, on the order of 11 MB for half an hour.
Any re-encode of that is lossy-to-lossy: it would cost quality to buy
compatibility that AAC in an
.Pa .m4a
container does not actually lack, since every podcast client and every browser
plays it.
Transcoding to MP3 in particular would produce a
.Em larger
file that sounds
.Em worse .
.Pp
The second reason is the one that matters more, and it is the subject of the
next section.
.Sh THE TRANSCRIPT ONLY EXISTS ONCE
The recordings this format is built around are made by Google Recorder, which
performs speech recognition on the device as it records.
It stores the result
.Em inside the .m4a ,
as an additional
.Li mett
data track with the content type
.Li application/transcription_2 ,
alongside two further tracks
.Pq Li audio_tags_2 , waveform_1
that
.Nm
ignores.
.Pp
That track is a word-level transcript: for the first episode, 308 timed
segments containing 4051 words, each with a start and end offset in
milliseconds.
It is obtained with
.Pp
.Dl ffmpeg -i rec.m4a -map 0:1 -c copy -f data -
.Pp
and a small protobuf decoder \(en no speech recognition, no API, no alignment
pass.
It is, in effect, free.
.Pp
It is also
.Sy unrecoverable .
Nothing outside the original file has it.
Every ordinary audio operation \(en transcoding, remuxing, "cleaning up the
metadata", running the file through an editor \(en drops unknown data tracks
silently and succeeds.
The audio is unharmed, the file plays fine, and the transcript is simply gone,
with no error to notice.
Re-deriving it later means running a speech recogniser over the archive and
accepting a different, worse result.
.Pp
The rule that follows is the least negotiable thing in this program:
.Cm add
extracts the transcript at ingest time and stores it in the repository,
whether or not anything currently displays it.
For the first two episodes nothing did: the data was captured anyway, because
the moment to capture it is the only moment it exists.
The episode pages render it now, which is the argument paying off rather than
changing \(en had it been captured only once there was a use for it, there
would have been nothing to render.
.Pp
.Cm add
does
.Sy not
refuse a file with no transcription track; it ingests it and says the track is
missing.
Refusing was tried, on the argument that a missing track usually means the
file has already been through a tool that stripped it.
The argument is right about the cause and wrong about the remedy:
.Cm add
cannot tell a stripped file from one that never had a transcript, so refusing
turns a fact worth knowing into an obstacle in front of every recording that
was made some other way \(en and by the time
.Cm add
runs, whatever damage there was has already been done.
Ingesting loses nothing that was not lost before, and an episode without a
transcript is still an episode.
.Pp
The moment the check is worth anything is
.Em before
the destructive step, not at ingest, and it does not need a recording to be an
episode:
.Pp
.Dl observations transcript --format streams recording.m4a
.Pp
prints the container's data tracks and marks the transcript, and exits
non-zero when there is none.
That is the check to run before moving, converting or tidying up a recording,
which is the moment the track is usually lost.
.Pp
Storing it in git rather than leaving it in the container is deliberate: it is
small, it diffs, it greps, and it survives the audio file being moved,
re-hosted or lost.
It also makes a transcript view, cross-episode search, or click-to-seek a
later rendering decision rather than a re-recording problem.
The first and third of those are now on the episode pages, and neither
required going back to the audio \(en which is the argument holding: the
seeking was built years after the recording, from a file that was captured
before anyone knew what it would be for.
.Pp
Being in git is also what makes the file safe to
.Em edit .
What the recogniser produced is unrepeatable only up to the moment it is
committed; after that the history holds it, a correction is an ordinary diff,
and a bad one is
.Ic git checkout .
The rule that nothing may touch the transcript belongs to the window between
the recorder and
.Cm add ,
not to the file afterwards \(en which is just as well, because raw recogniser
output is wrong often enough to be worth correcting.
See
.Xr observations-transcripts 7 .
.Pp
It lives in
.Pa episodes/NNN.transcript ,
beside the episode rather than inside it.
The two were one file at first, and the result was that a thirty-line record of
what an episode is sat underneath tens of thousands of lines of timed words:
every diff, every
.Ic git log -p
and every attempt to hand-edit a source had to be done through that.
The split costs one derived path \(en the transcript's name comes from the
episode's, so there is nothing to keep in sync and no way to point at another
episode's transcript \(en and buys back a metadata file that can be read at a
glance.
.Ss The stored shape is not the recogniser's shape
What comes out of the container is a flat list of words, each with its own
start and end.
That is the right shape for a protobuf and the wrong shape for a file someone
edits: encoded as JSON it cost 117 bytes and five lines per word, so a
half-hour episode was twenty-two thousand lines, and correcting a single word
meant finding it in that and then computing an index into a flattened array
the nesting hid.
Every error in the first correction pass came from that arithmetic \(en
deletions shifting the indices of later edits, paragraph breaks riding on
deleted words, one word replacing three.
.Pp
So the stored format is line-oriented and subtitle-shaped instead: about six
words to a line, timestamps in a fixed-width margin, structure on its own
lines.
Correcting a word is editing a line.
The cost is that per-word timings are not kept \(en interpolating within a
line is accurate to about a tenth of a second, which is inside what seeking
audio needs, and the exact values remain in git history.
See
.Xr observations-transcripts 5 .
.Sh THE EPISODE PAGE
The page is the recording plus the text of it, and every decision on it
follows from one fact: leaving the page stops the audio, and no browser comes
back to a position half an hour into a recording.
.Pp
So every link off an episode page \(en the sources, and the
.Sy \(->
at the end of each quotation \(en opens in a new tab, which the rest of the
site does not do.
The arrow's link carries a text fragment taken from the quotation's opening
words, so a browser that supports them scrolls to the sentence and highlights
it and one that does not opens the article.
Those words come from the transcript, which is the second place a correction
pass pays off: a misheard quotation matches nothing.
Where the speaker reworded the run-up, a
.Ic .skip
moves the fragment past it
.Pq Xr observations-transcripts 5 .
.Pp
Where scripting is available the transcript is also an index into the audio:
the line being spoken is highlighted, and clicking a line seeks to its stored
start, which is exact rather than interpolated.
Seeking is offered
.Em only
while the recording plays, and the text is styled as clickable only then \(en
a click that started playback would mean a page being read in quiet can be
made to speak by accident.
Selecting text does not seek, so a passage can be copied out while it plays,
and nothing scrolls on its own.
Without scripting the text is all still there and each paragraph timestamp is
an ordinary link, so a
.Pa #t Ns Ar ms
fragment names a passage in a way that can be shared.
.Pp
The transcript is not in the feed.
An entry carries the episode's summary; a corrected mishearing is not news,
and should not re-notify every subscriber.
.Sh WHY THE AUDIO IS NOT IN GIT
The site is published by pushing a git repository, and it would be simpler for
the audio to ride along in the same push.
It does not, for two reasons.
.Pp
First, this repository is published to
.Lk https://sources.profpatsch.de/Profpatsch/ ,
so anything committed here appears in a public source browser.
Audio files are not source.
.Pp
Second, and less reversibly: git keeps every blob forever, and compressed
audio does not delta-compress.
At roughly 11 MB an episode, a weekly cadence is about half a gigabyte a year
added permanently to a repository that is cloned often and never shrinks.
The cost is not the disk, it is that the mistake cannot be undone later
without rewriting history.
.Pp
So the two halves are published by two different mechanisms, and this is the
one piece of friction the design accepts deliberately:
.Bl -tag -width indent
.It Pages and feed
Rendered into
.Pa users/Profpatsch/web/observations/
and pushed with the rest of the website.
.It Audio
Copied to
.Pa /var/lib/observations/audio/
on the server and served from a separate nginx location.
.El
.Pp
Keeping the audio outside the Nix store is the same argument in a different
key: were the site a store path containing the episodes, every new episode
would require a system rebuild and a closure copy to publish one HTML file and
one audio file.
The site is static output, not a system configuration, so it is deployed like
content.
.Sh WHY A FEED AND NOT AN ACTIVITYPUB ACTOR
The obvious way to be present on the Fediverse is to be an actor, and the
machinery to do that already exists in this repository
.Pq the Li activitypub-go No library .
.Nm
does not use it.
.Pp
An actor is a mailbox: once it can be followed, it can be replied to, and
replies have to be received, moderated and answered.
That is a permanent obligation, and it is a social one rather than a technical
one \(en it cannot be discharged by writing better code.
It is not obviously the right one for a format whose whole point is unhurried
reading.
.Pp
A podcast feed has none of that and loses very little.
It is subscribable in every podcast client, and it is the standard input to
feed-to-Fediverse bridges, so the option of an actor remains open and costs
nothing to keep open.
The bet is that publishing should not create an inbox until there is a reason
to want one.
.Sh THE INBOX THAT WAS NOT SUPPOSED TO EXIST
The section above argued against having an inbox, and then the site grew one:
a drop zone where a listener can send a recording of their own, served by
.Xr observations-inbox 1 .
That is a reversal, and it is worth being precise about what was actually
being argued, because most of it still holds.
.Pp
The objection was never to receiving things.
It was to the
.Em obligation
that a federated actor creates: something followable is something repliable,
and a reply in a conversation expects an answer.
That expectation cannot be bounded by code, it does not stop when attention
runs out, and withdrawing from it after other servers have followed is not
something the protocol makes graceful.
.Pp
A submission slot is a different shape, and the differences are the whole
argument:
.Bl -bullet
.It
It is
.Sy one-way .
There is no thread, and nothing about sending a recording implies a reply.
What arrives is material, not conversation.
.It
It is
.Sy bounded .
Five megabytes at a time, fifty in total; when it is full it closes and says
so.
A federated inbox has no such number.
.It
It is
.Sy switchable .
Turning it off breaks nobody's follow and leaves no dangling relationship \(en
the page simply stops offering it.
Un-federating an actor is not that.
.It
It is
.Sy quiet .
Nothing arrives in public, nothing is addressed to an audience, and there is
no timeline for a bad-faith submission to be visible in.
The failure mode of a submission is that it is deleted and no one ever knows.
.El
.Pp
So the standing bet is unchanged: no actor, no federated inbox, and
conversation still happens on Mastodon under an account that already exists
and is already moderated.
What changed is the recognition that
.Dq send me a recording
does not need any of that machinery, and that asking listeners to record
observations of their own while providing nowhere to put them was the weaker
position.
.Ss What that costs
An endpoint on the public internet that accepts uploads is found by scanners
within days, so the slot is not open: a passphrase spoken at the start of each
episode is what unlocks it.
Reaching for something only listeners have is deliberate.
It cannot be scraped from the page, it costs nothing to change, and the set of
people who have heard it is exactly the set of people the invitation is meant
for.
.Pp
Submissions are held, not forwarded.
The notification carries an unguessable link rather than the audio, because a
recording attached to a mail outlives every deletion, and being able to delete
one is the point.
There is no listing route for the same reason a token is not a filename: one
link exposes one recording, and knowing it says nothing about any other.
.Pp
Transcription is offered because reading is faster than listening, and it is
opt-in because the recording is someone else's voice and sending it to Google
is not a decision this program gets to make for them.
Unticked means the audio never leaves the machine it arrived on.
.Pp
A submission is the one place where the rule above \(en that the artefact is
the file that arrived \(en is bent, and it is worth saying why the exception
does not undermine it.
A recording made in the page comes from MediaRecorder, which cannot write a
duration into a container it is still writing, so the file arrives declaring no
length and no player can offer a scrubber for it until it has been played
through once.
Such a file is remuxed at ingest: the encoded audio is copied into a new
container, not decoded, so the samples are the samples that arrived.
The reason this is safe here and not for an episode is that the thing an
episode would lose to a remux \(en the transcript track, which exists nowhere
else \(en is a property of Google Recorder files, and a submission has no such
track to lose.
See
.Xr observations-inbox 1 .
.Ss Recording in the page
The slot takes a dropped file, and it also records one: the episode page has a
button that captures from the microphone and hands the result to the same file
input a dropped file lands in.
This is the difference between replying being a thing you do now and a thing
you do later with a phone, a file manager and a laptop \(en which is to say,
mostly not at all.
.Pp
It is strictly an addition.
The button ships hidden and is revealed only where
.Li MediaRecorder
exists, so a browser without it, or without scripting at all, is left with a
plain file input that works everywhere.
The recorded blob goes through the same checks as any other upload, and the
server sniffs the bytes rather than believing what the browser called them.
.Sh IDENTIFIERS ARE PERMANENT
Episodes are numbered sequentially and live at
.Pa /NNN/ .
Feed GUIDs are derived from that number.
.Pp
Numbers rather than title slugs because a URL that encodes the source's title
cannot be corrected later without breaking every subscriber and every archive
that recorded it, and because the source's title is not
.Nm Ns 's
to fix.
Numbers rather than dates because two recordings can share a day, and because
the publication date of the episode and the date of the text being read are
different things that a date-shaped URL invites confusing.
.Pp
The title displayed \(en
.Li 001 \(em Why We're Dropping Basecamp
\(en carries both: the number that is stable and the source title that is
searchable.
Retitling an episode is therefore always safe.
.Sh SEE ALSO
.Xr ffmpeg 1 ,
.Xr observations 1 ,
.Xr observations-transcripts 5 ,
.Xr observations-transcripts 7
.Sh AUTHORS
.An Profpatsch