SQLiteFS Filesystem Semantics

This document specifies the observable behavior of the SQLiteFS filesystem and its relationship to SQLite database state. It serves as the authoritative specification for testing and ensuring behavioral consistency across implementations.

Data Model

SQLite Schema

CREATE TABLE files (
    filename TEXT PRIMARY KEY,
    filecontent BLOB
);

Requirement: filecontent must be BLOB type for byte-level operations.

Path-to-Database Mapping

• Filesystem paths are stored in the database without leading slashes
• /foo/bar ↔ database entry with filename = "foo/bar"
• / (root) ↔ queries for all entries
• Empty path "" is not a valid filesystem path

File vs Directory Semantics

File Entries

• Definition: A file exists if there is a database row with filename exactly matching the path
• Example: Database row filename = "test.txt" creates file /test.txt

Directory Entries

• Definition: A directory exists if either:
  1. There are database rows with filename values having that path as a prefix, followed by /
  2. There is a database row with filename exactly matching the path plus trailing /
• Example: Database row filename = "dir/file.txt" creates directory /dir/
• Explicit creation: Database row filename = "dir/" creates empty directory /dir/ (filecontent ignored)
• Precedence hierarchy:
  1. Nested files (highest) - if foo/bar exists, /foo is a directory with contents
  2. Directory marker - if foo/ exists, /foo is an empty directory
  3. Direct file (lowest) - if foo exists, /foo is a regular file

Mixed File/Directory Cases

• If both filename = "foo" and filename = "foo/bar" exist:
  • /foo appears as a directory containing bar
  • The direct file content from filename = "foo" is not accessible via the filesystem
  • Directory takes precedence over file when both exist

Directory Marker Examples

• filename = "empty/" → Creates empty directory /empty/
• filename = "empty/" + filename = "empty/file" → Directory /empty/ contains file, marker ignored
• filename = "mixed" + filename = "mixed/" → /mixed appears as empty directory (directory marker takes precedence over file)

Directory Listing Behavior

Root Directory (/)

When listing /, the filesystem shows:

• Direct files: Any filename without / appears as a file
• Top-level directories: For any filename containing /, only the part before the first / appears as a directory

Example Database State:

filename = "topfile"        → shows as file "topfile"
filename = "a/b"           → shows as directory "a" 
filename = "a/c"           → shows as directory "a" (not duplicated)
filename = "x/y/z"         → shows as directory "x"

Subdirectory Listings

When listing /a/, the filesystem shows:

• Direct children: Any filename starting with a/ but not containing additional / after the prefix
• Subdirectories: Any filename starting with a/ and containing exactly one more /

Example Database State:

filename = "a/file1"       → shows as file "file1"
filename = "a/file2"       → shows as file "file2"  
filename = "a/sub/file3"   → shows as directory "sub"
filename = "a/sub/file4"   → shows as directory "sub" (not duplicated)

Display Name Rules

• Root directory: Strip everything after first / for directories, show files as-is
• Subdirectories: Strip the directory prefix (a/) from all matching entries
• Duplicates: Multiple database entries mapping to the same display name show as a single directory entry

File Operations & SQLite Data

File Reading

• Reading /path queries filename = "path"
• Content returned from filecontent BLOB starting at requested offset
• Offset behavior: Reading beyond file length returns empty data
• Binary data: Files can contain arbitrary binary data including null bytes; all bytes should be preserved

File Writing

• Writing to /path modifies the filecontent BLOB for filename = "path"
• Offset writes: Data is inserted at the specified offset, with existing content preserved before and after
• File extension: Writing beyond current file length extends the file
• New files: Writing to non-existent files creates new database rows

File Creation

• Creating /path inserts filename = "path" with filecontent = "" (empty BLOB)
• Overwrite behavior: Creating existing files replaces content with empty BLOB

File Deletion

• Deleting /path removes the database row where filename = "path"
• Directory implications: May cause parent directories to disappear if no other files remain
• Non-existent files: Deleting non-existent files returns success (POSIX behavior)

File Truncation

• Truncating /path to length N modifies filecontent to contain exactly N bytes
• Shrinking: Truncating to smaller size removes trailing bytes
• Growing: Truncating to larger size extends file with null bytes, preserving all existing content

Access Control & Permissions

Read-Only vs Read-Write Mode

• Table-backed filesystem: Allows both read and write operations
• View-backed filesystem: Allows only read operations; writes return EROFS (Read-only file system)
• Mode detection: Determined by querying sqlite_schema for table vs view type

Path Resolution Rules

Path Normalization

• Leading slashes are stripped before database queries
• /foo/bar → database lookup for filename = "foo/bar"
• Multiple consecutive slashes are NOT normalized (handled by FUSE layer)

Case Sensitivity

• Filesystem paths are case-sensitive (SQLite default TEXT comparison)
• /File and /file are distinct entities

Special Characters

• All Unicode characters allowed in SQLite TEXT are valid in filenames
• Null bytes (\0) are not allowed within filenames (PATH components, not file content)
• Forward slashes (/) have special meaning as path separators

Error Conditions

Standard POSIX Errors

• ENOENT: File or directory not found in database
• ENOTDIR: Attempting directory operations on files
• EISDIR: Attempting file operations on directories
• EROFS: Write operations on read-only filesystem
• EIO: Database connection or SQL execution errors
• ENOMEM: Memory allocation failures

Edge Cases & Invariants

Empty Files vs Missing Files

• Empty file: Database row exists with filecontent = "" (zero-length BLOB)
• Empty file: Database row exists with filecontent = NULL (to be maximally flexible)
• Empty file: When writing an empty file, we always write as filecontent = ""
• Missing file: No database row exists with matching filename
• Behavioral difference: Empty files show in directory listings; missing files return ENOENT

File Size Consistency

• File size reported by stat() must match LENGTH(filecontent) in database
• All binary data including null bytes must be accurately reflected in file size

Testing Implications

Required Test Coverage

1. Path mapping: Verify filesystem paths correctly map to database entries
2. Directory inference: Test directory creation/destruction based on file presence
3. Listing behavior: Verify correct display names and entry types in directory listings
4. File operations: Test read/write/create/delete with various offsets and sizes
5. Permission enforcement: Verify read-only mode restrictions
6. Error handling: Test all documented error conditions
7. Edge cases: Empty files, mixed file/directory names, special characters
8. Binary data handling: Test files with null bytes, non-UTF-8 content, large files

Property-Based Testing

• Path consistency: Any valid filesystem operation sequence should leave database in consistent state
• Bidirectional mapping: Database state should always produce predictable filesystem view
• Permission invariants: Read-only filesystems should never allow modifications

Cross-Implementation Testing

• Haskell and Zig implementations should produce identical filesystem behavior for same database state
• Database modifications through either implementation should be compatible
• Error conditions and errno codes should match across implementations

Fuzz Testing

• We should consider using fuzzing to validate the implementations (zig has builtin fuzzer support)