BREAKING
fastdup Cleans Visual Datasets
Install, Run, Inspect, Delete
1
Install
↓
2
Analyze
↓
3
Inspect
↓
4
Remove
0
M
images / one CPU
0
default similarity
0
B+
images processed
ImageNet-21K Cleanup Signal
Dups (M)
1.2
Leaks (0.1M)
0.104
Praise Meets Caveats
Strengths
Users
●
CPU-local runs
●
Galleries before delete
●
Leak and label checks
Limits
Caveats
●
Tune thresholds
●
Dry-run deletes
●
CC BY-NC-ND 4.0
Clean Data Before You Train
AI NEWS BLITZ
Visual Layer fastdup is back in focus for analyzing image and video datasets and removing duplicates.