1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
|
# whatcd-resolver Notes
**IMPORTANT**: Always remove completed items from this file. Completed work should not clutter the active notes.
## Database Schema Cleanup
### TODO: Rename torrents_json to torrents
- Currently using `redacted.torrents_json` table
- Should rename to `redacted.torrents` for cleaner naming
- Remove the old `redacted.torrents` view if it exists
- Update all queries to use the renamed table
This will simplify the schema and make the naming more intuitive.
## Query Optimizations
### Current Issue
- Query optimization introduced complexity that may be causing failures
- Need to verify optimized query works correctly in application context
## Performance Optimization Goals
### Sub-100ms Response Time Target
**Date**: 2025-01-09
**Goal**: Achieve sub-100ms response times for all user interactions using traditional web patterns
**Status**: 🔄 **INVESTIGATION** - Jaeger analysis completed, specific bottlenecks identified
**Current Issues**:
- `/redacted-search`: 12+ seconds (blocks UI due to external API rate limiting)
- `/artist` route: 279ms (needs optimization)
- **Primary bottleneck**: "get torrents table data" query - 171ms (61% of total time)
- **Secondary bottleneck**: "Get Settings" query - 95ms (34% of total time)
**Traditional Architecture Solutions**:
- [ ] **Database-first search**: Show cached results immediately (<50ms), background refresh for updates
- [ ] **Two-phase loading**: Fast database query + background job processing
- [ ] **Artist route optimization**:
- Cache settings query (95ms → <5ms)
- Optimize torrents table query (171ms → <50ms via indexes/query optimization)
- Target: Reduce 279ms to <100ms
- [ ] **Background job queue**: Celery/RQ-style for API calls (no WebSockets/PWA)
- [ ] **Database query result caching**: Redis/in-memory for common queries
- [ ] **HTTP cache headers**: For static content optimization
**Benefits**: Responsive user experience without blocking operations, traditional web patterns
**Priority**: High - Core user experience improvement
## Feature Enhancements
### Background Artist Updates
**Date**: 2025-01-08
**Feature Request**: Add background queue that automatically fetches up-to-date artist information daily
**Status**: 🔄 **PLANNED** - New feature implementation needed
**Current Issue**: Manual artist reload feature exists but is broken - deletes torrents belonging to that artist instead of updating artist data
**Root Cause**: Artist update logic incorrectly handles torrent relationships during refresh
**Requirements**:
- [ ] **CRITICAL**: Fix existing manual artist reload bug before implementing background queue
- [ ] Investigate current artist update implementation to understand torrent deletion issue
- [ ] Background job queue system (consider existing service architecture)
- [ ] Daily scheduled artist information updates
- [ ] Database integration for storing updated artist data
- [ ] Error handling and retry logic for failed API calls
- [ ] Logging/monitoring for background job status
**Benefits**: Keep artist information current without manual intervention
**Priority**: High - Manual reload bug affects data integrity
### Bandcamp Integration
**Date**: 2025-01-08
**Feature Request**: Add "buy on bandcamp" functionality for albums when artist is found on Bandcamp
**Status**: 🔄 **PLANNED** - New feature implementation needed
**Approach Strategy (prioritized)**:
1. **Primary**: Check existing release descriptions for Bandcamp links (parse existing data)
2. **Secondary**: MusicBrainz API lookup for release relationships/external URLs
3. **Fallback**: Direct Bandcamp search API for artist/album matches
**Requirements**:
- [ ] **Priority 1**: Parse existing release descriptions for Bandcamp URLs (regex/text extraction)
- [ ] **Priority 1**: URL validation and normalization for found Bandcamp links
- [ ] **Priority 2**: MusicBrainz API integration for release metadata lookup
- [ ] **Priority 2**: Parse Bandcamp URLs from MusicBrainz release relationships/external identifiers
- [ ] **Priority 3**: Bandcamp artist/album search API integration as final fallback
- [ ] UI integration - "Buy on Bandcamp" button/link on album pages
- [ ] Caching strategy for external API results (not needed for description parsing)
- [ ] Rate limiting for API calls (MusicBrainz has specific rate limit requirements)
**Technical Considerations**:
- **Description parsing**: Likely already available in database, no API calls needed
- **MusicBrainz approach**: More structured metadata, canonical release information
- **Direct Bandcamp**: Most complex, may have reliability issues
- URL pattern matching for Bandcamp links in descriptions
- Validation that extracted URLs are still valid/accessible
**Benefits**: Direct support for artists through purchase links, enhanced user experience
**Priority**: Medium - Nice to have feature for artist support
### Artist Recommendations
**Date**: 2025-01-08
**Feature Request**: Create artist recommendation system using Redacted's "related artists" API
**Status**: 🔄 **PLANNED** - New feature implementation needed
**Goal**: Suggest unchecked artists to users with their highest-weighted album recommendations
**Requirements**:
- [ ] Integrate with Redacted's "related artists" web/API functionality
- [ ] Track which artists user has already checked/viewed
- [ ] Algorithm to identify "unchecked" artists from related artists graph
- [ ] Weight calculation system for albums (by what metric? seeds/leechers/time/quality?)
- [ ] UI for displaying artist recommendations with top album suggestion
- [ ] User preference system (opt-in/out, recommendation frequency)
- [ ] Database schema for tracking user's artist interaction history
- [ ] Recommendation refresh logic (when to update suggestions)
**Technical Considerations**:
- Related artists API response format and rate limits
- User tracking methodology (sessions, accounts, local storage?)
- Album weighting algorithm design (popularity vs quality vs activity)
- Recommendation UI integration with existing interface
- Performance considerations for graph traversal and weight calculations
**Benefits**: Music discovery enhancement, increased user engagement, personalized experience
**Priority**: Medium - Enhanced user experience feature
### Auto-disable Freeleech on Token Exhaustion
**Date**: 2025-01-09
**Problem**: When freeleech is enabled but user has no tokens left, Redacted returns 400 error causing crashes instead of graceful fallback
**Status**: 🔄 **PLANNED** - Error handling enhancement needed
**Error**: `{"status":"failure","error":"You do not have any freeleech tokens left. Please use the regular DL link."}`
**Requirements**:
- [ ] **Extend Settings Data Structure**: Add `freelechLastAutoDisabled :: Maybe UTCTime` field to track auto-disable timestamp
- [ ] **Database Schema Update**: Extend `redacted.settings` table with new timestamp column
- [ ] **Enhanced Error Handling**: Detect specific "no freeleech tokens" error in `httpTorrent` function (400 responses)
- [ ] **Automatic Recovery**: On token exhaustion detection → auto-disable freeleech + record timestamp + retry without tokens
- [ ] **User Interface Warnings**: Show warning when enabling freeleech if auto-disabled within last 24 hours
- [ ] **Settings Management**: Update `settingButtons` and settings form to display auto-disable status
- [ ] **Database Migration**: Add migration for new settings field while preserving existing data
**Technical Implementation**:
- Modify `httpTorrent` in `Redacted.hs` to parse 400 error responses for token exhaustion
- Extend `Settings` data type in `WhatcdResolver.hs` with timestamp field
- Update `getSettings`/`writeSettings` functions for new field
- Add warning logic in `settingButtons` function
- Preserve existing error handling for other 400 errors
**Benefits**: Graceful error recovery, prevents crashes, better UX, provides user feedback about token status
**Priority**: High - Critical error handling improvement
|