1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111

# Development Notes

This file tracks observations, potential issues, and areas for improvement discovered during development.

## Pretty Printer Issues

### Leading Empty Line in Multi-field Records  
**Date**: 2025-01-04  
**Issue**: `netencode-pretty` outputs a leading empty line when formatting multi-field records  
**Status**: ✅ **FIXED** - 2025-01-07  
**Fix**: Modified `lib-rust/pretty.rs` line 265 to remove leading newline in multi-line formatting  
**Impact**: Pretty printer now correctly formats multi-field records, nested records, and lists without unwanted leading empty lines  

## Field Ordering

### Python vs JSON-to-netencode Field Order
**Date**: 2025-01-04  
**Observation**: Python `netencode.record()` orders fields alphabetically, while `json-to-netencode` preserves JSON key order  
**Example**: 
- Python: `{49:<4:name|t5:Alice,<3:age|i:30,<6:active|<4:true|u,}`
- JSON: `{49:<6:active|<4:true|u,<3:age|i:30,<4:name|t5:Alice,}`
**Status**: Expected behavior (different ordering strategies)  
**Impact**: Documentation must show correct expected outputs for each method

## Testing Infrastructure

### Test Data Generation
**Date**: 2025-01-04  
**Best Practice**: Always use `json-to-netencode` for generating test data rather than hand-crafting netencode format  
**Reason**: Ensures correct length prefixes and format compliance  
**Reference**: Fixed netencode-filter tests by using proper generation

## Areas for Future Investigation

- [x] ~~Pretty printer leading empty line behavior~~ (Fixed 2025-01-07)
- [ ] Standardize field ordering across implementations
- [ ] Performance benchmarks vs other formats
- [ ] Error message clarity improvements

## Rust Implementation TODOs

### Type System Improvements
**Date**: 2025-01-04  
**Files**: `lib-rust/src/netencode.rs`  
**TODOs Found**: 12 total

**API Design Issues**:
- Lines 19, 24, 27, 167: Convert `String` fields to `&str` for zero-copy parsing
- ✅ Lines 21, 25, 850: ~~Rename `Binary` → `Bytes`, `Sum` → `Tag` for clarity~~ (Completed 2025-01-07)
- Line 894: Rename `Record` type description from "map of Ts" to "map"
- Line 131: Rename type parameter in Tag struct

**Performance Issues**:
- Line 112: `T::from_u()` conversion inefficient, avoid U→T roundtrip
- Line 128: U-recursion prevents breadth-lazy evaluation

**Code Organization**:
- Line 319: Move decoder code to separate module

### Execution Helpers Performance
**Date**: 2025-01-04  
**File**: `lib-rust/exec-helpers/exec_helpers.rs`  
**Lines**: 68, 75

**Issue**: Unnecessary Vec collection for iterator processing  
**Impact**: Memory overhead when processing command arguments and environment  
**Complexity**: Medium - requires careful lifetime management

### Pretty Printer Issues  
**Date**: 2025-01-04  
**File**: `lib-rust/pretty.rs`  
**Lines**: 161, 171

**Issue**: Length information lost during U→Pretty conversion  
**Impact**: Pretty printer can't show original encoded lengths  
**Decision Needed**: Whether to recompute or preserve original lengths

## Lazy Parsing Implementation - Stashed Work

### Nom Parser Upgrade Required
**Date**: 2025-01-07  
**Status**: ⏸️ **PAUSED** - Stashed in git  
**Stash**: `WIP: lazy parsing implementation with compilation errors - needs nom upgrade`

**Issue**: Current lazy parsing implementation has compilation errors due to nom parser function signature incompatibilities. Functions return `FnMut` closures but signatures expect `Fn` closures.

**Root Cause**: Version mismatch between nom parser combinator API expectations and actual return types. The `impl Fn` return types are incompatible with nom's `FnMut` closures.

**Solution Required**: 
1. Upgrade nom dependency to compatible version
2. Update parser function signatures to match nom API
3. Resume lazy parsing implementation from stash
4. Add comprehensive unit tests

**Lazy Parsing Features Implemented (Stashed)**:
- Raw variant added to U<'a> enum
- ParseMode system (Strict, Lazy, MaxDepth)  
- Structural parsing functions (record_g_lazy, list_g_lazy)
- Random access methods (get_field_mut, get_element_mut)
- Force parsing functionality with mem::replace
- Example file with demonstration

**Next Steps After Nom Upgrade**:
1. `git stash pop` to restore work
2. Fix compilation errors with updated nom API
3. Add unit tests to mod tests section
4. Verify all functionality works correctly

## Development Experience Improvements