ADDeX — Advanced Data Deduplication Engine
I’m working on a new data deduplication and compression tool called ADDeX (Advanced Data Deduplication Engine).
The goal is to experiment with different levels of duplicate-data detection and compression, with a focus on achieving better compression ratios while allowing the user to control the trade-off between speed, memory usage, and compression efficiency.
Current compression modes
M1 — Standard: General-purpose compression with reasonable speed and memory usage.
M2 — Improved: More aggressive matching for better compression ratios.
M3 — Exhaustive: Performs much more intensive matching for maximum compression potential. This mode is significantly more RAM-intensive and can be considerably slower.
Available options
ADDeX.exe [OPTIONS] <command> <input> <output>
Commands:
-m1 <input> <output> Standard compression mode
-m2 <input> <output> Improved compression mode
-m3 <input> <output> Exhaustive compression mode
-d <input> <output> Decompress a file
-d - - Decompress stdin to stdout
-h Display help
Compression options:
-l<N> Minimum match length
Default: 512 | Min: 64 | Max: 1024
-a<N> Alignment
Default: 0 | Min: 0 | Max: 2
-b<N> Block size in MB
Default: 8 MB
Examples
ADDeX.exe -m1 input.dat output.addex
ADDeX.exe -m2 -l64 -a1 data.bin output.addex
ADDeX.exe -m3 -l512 -b16 data.bin output.addex
ADDeX.exe -d compressed.addex original.bin
type compressed.addex | ADDeX.exe -d - - > output.bin
- can be used as a filename to read from stdin or write to stdout, making ADDeX suitable for command-line pipelines.
Version: 0.1 Alpha
Author: Panker1992
This is an early alpha release, so the format, algorithms, command-line options, and compression behavior are still subject to change. Feedback, testing results, and suggestions are welcome.
I’m working on a new data deduplication and compression tool called ADDeX (Advanced Data Deduplication Engine).
The goal is to experiment with different levels of duplicate-data detection and compression, with a focus on achieving better compression ratios while allowing the user to control the trade-off between speed, memory usage, and compression efficiency.
Current compression modes
M1 — Standard: General-purpose compression with reasonable speed and memory usage.
M2 — Improved: More aggressive matching for better compression ratios.
M3 — Exhaustive: Performs much more intensive matching for maximum compression potential. This mode is significantly more RAM-intensive and can be considerably slower.
Available options
ADDeX.exe [OPTIONS] <command> <input> <output>
Commands:
-m1 <input> <output> Standard compression mode
-m2 <input> <output> Improved compression mode
-m3 <input> <output> Exhaustive compression mode
-d <input> <output> Decompress a file
-d - - Decompress stdin to stdout
-h Display help
Compression options:
-l<N> Minimum match length
Default: 512 | Min: 64 | Max: 1024
-a<N> Alignment
Default: 0 | Min: 0 | Max: 2
-b<N> Block size in MB
Default: 8 MB
Examples
ADDeX.exe -m1 input.dat output.addex
ADDeX.exe -m2 -l64 -a1 data.bin output.addex
ADDeX.exe -m3 -l512 -b16 data.bin output.addex
ADDeX.exe -d compressed.addex original.bin
type compressed.addex | ADDeX.exe -d - - > output.bin
- can be used as a filename to read from stdin or write to stdout, making ADDeX suitable for command-line pipelines.
Version: 0.1 Alpha
Author: Panker1992
This is an early alpha release, so the format, algorithms, command-line options, and compression behavior are still subject to change. Feedback, testing results, and suggestions are welcome.
