CLS cls-bitshuffle 26.02.05.rev.1

[Shegorat]

Shegorat добавил(а) новый ресурс:

cls-bitshuffle - Битовый препроцессор для структурированных данных

cls-bitshuffle - это один из моих старых проектов, по предложению товарища Skymmer. Он основан на базе проекта bitshuffle. Когда я его начинал несколько лет, то не нашёл подходящие данные, на которых алгоритм давал бы выгоду. И в итоге забросил проект. Сейчас я его закончил, может быть кто-то найдёт его полезным

Параметры:

  b=[N]  - Размер блока (default 4mb)
  es=[N] - Размер элемента [1, 2, 4, etc] (по умолчанию 4)

...

Узнать больше об этом ресурсе...

 

[Skymmer]

Неожиданно и очень приятно.
Спасибо большое!

 

[Razor12911]

Seems to work for very specific datasets

-mlzma

Compressed 1 file, 4,165,424 => 2,133,655 bytes. Ratio 51.22%
Compression time: cpu 0.58 sec/real 0.37 sec = 157%. Speed 11.29 mB/s

-mbitshuffle:es16+lzma

Compressed 1 file, 4,165,424 => 1,846,008 bytes. Ratio 44.32%
Compression time: cpu 0.27 sec/real 0.15 sec = 177%. Speed 27.79 mB/s

Also I'd like to point out that the cls has vcredist requirements, tried running it under VM where I test things and took me a while to figure out why it wasn't working.

 

[Skymmer]

Yes, DLL files are dynamic compiles with dependency to MSVC Redist 2015 (v14.0)

https://krinkels.org/redirect?to=aHR0cHM6Ly9sZWFybi5taWNyb3NvZnQuY29tL2VuLXVzL2NwcC93aW5kb3dzL2xhdGVzdC1zdXBwb3J0ZWQtdmMtcmVkaXN0P3ZpZXc9bXN2Yy0xNzA=

Your results look intersting. Could you please upload your test file somewhere?
Like send.cm or similar simple file share ?
Thanks!

 

[Shegorat]

Shegorat обновил(а) ресурс cls-bitshuffle новой записью:

cls-bitshuffle v.26.02.05.rev.1

Изменения:

• Пересобрал без зависимостей vcredist (thx Razor12911, Skymmer)

Узнать больше об этом обновлении...

 

[Razor12911]

Your results look intersting. Could you please upload your test file somewhere?
Like send.cm or similar simple file share ?
Thanks!

I wish I could share the sample but it was extracted from memory during a run of xtool. I couldn't find an actual file to test for this tool... at least from general game files until I did a bit of research on how bitshuffle worked and thought it would be a good idea to use on stored table of content information or headers. So I had the cls plugin process header information which contained compressed and decompressed size information of streams (basically the header structure of xtool) and this was the result. I'd reckon it can be used on databases or archives/game files table of content but you need to have prior knowledge on how the data was stored otherwise the results will be terrible.