[Feature Request] Compress Savestates

[hizzlekizzle]

Description

A number of cores have very large savestates that are mostly empty space, such as PPSSPP, and these can be reduced in size dramatically by compressing them:

Compressing them with fast compression, such as gzip, could potentially open such cores up to advanced state-based features such as runahead.

Expected behavior

Savestates should be as small as possible, without a bunch of filler.

Actual behavior

Some cores have states full of empty data

Steps to reproduce the bug

1. Fire up PPSSPP
2. Save a state
3. Look at filesize

Bisect Results

Always

Environment information

• OS: All
• Compiler: All

[ghost]

Use LZ4 with a basic file quantum for storing compressed/uncompressed file lengths. You could use a hardware accelerated CRC32 (using the CRC32 opcodes on SSE4, OR SSE2/NEON optimized CRC32 for folding. Look at Intel's paper on the subject.) on the data for checking integrity. Might be able to exploit HW SHA-1 too.

[RobLoach]

There are concerns around savestate compression and netplay. There was a fast flag added recently to help this. May be the core's responsibility now?

[andres-asm]

No, compression should only come into play when flushing states to disk and it should be done by the frontend

…

On Tue, Oct 30, 2018 at 2:04 AM Rob Loach ***@***.***> wrote: There are concerns around savestate compression and netplay. There was a fast flag added recently to help this. May be the core's responsibility now? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7501 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpC0MYy43quxXhxjS8KIZSis3URFu_lks5up_oDgaJpZM4X-IXP> .

[ghost]

Are savestates saved in worker threads? If thats the case you could use a stronger algo like LZMA even.

[andres-asm]

They are saved in a task, and they could surely be compressed, taking them and flushing them are clearly delimited operations.

…

On Thu, Nov 1, 2018 at 5:33 PM Brad Miller ***@***.***> wrote: Are savestates saved in workers? If thats the case you could use a stronger algo like LZMA even. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#7501 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpC0Iko61Fg5x5Pj5uVwaclO426XL-Xks5uq3a4gaJpZM4X-IXP> .

[ghost]

Ah, thanks.
In einweg, I could use the Windows 7 worker thread API for those.

[meepingsnesroms]

Someone has requested this for Mu too:
meepingsnesroms/Mu#87

The savestates contain the entire RAM(which includes the filesystem on Palm OS devices).

This is 16MB on Palm OS 4 and will be 64MB on Palm OS 5.

[hizzlekizzle]

This has been implemented by jdgleaver, so closing.