Compressing a file/array by saving as PNG

[Wampus]

Its possible to use the compression method used when saving PNG files to compress pretty much anything. Attached to this post is some code which does just that. Works nicely for large files. For smaller files the advantage diminishes since saving array data as a PNG file will add ~800 bytes of data.

The functions:-

LoadDataSprite( filename$ )
Loads a file into the array datasprite%[]

SaveDataSprite( filename$ )
Saves a file from the data contained in the array datasprite%[]

DeflateDataSprite (filename$ )
Compresses and saves the data in array datasprite%[]

InflateDataSprite ( filename$ )
Loads and uncompresses a file and puts the data into datasprite%[]

ClearDataSprite()
Clears the array datasprite%[] and size register

[attachment deleted by admin]

[Wampus]

n/p if I could find a way to get GLBasic to turn off saving superfluous palette info when saving PNG files then that reduces every file by 780 bytes. I know there is an option in pnglib to turn off palette saving for RGB files so maybe there is a way to override default GLBasic using that.

[Moru]

Wow, great idea, thanks!

[Kitty Hello]

Blimey, is this an awesome idea.
I'll try to cut the palette out. Do you know the parameter off your mind?

[Wampus]

Cool! Without the palette files would be real small.

I'm not sure how to cut the palette out completely but I have a couple of guesses. The official documentation for libpng says the palette (PLTE chunk) is optional for truecolor images with alpha but doesn't say what parameter to change when writing a file! In the source code I found a PNG_NO_READ_OPT_PLTE defined to prevent reading unnecessary optional PLTE data. I couldn't find the equivalent for turning off writing the optional palette.

In the source code it may be that color_type parameter being set to PNG_COLOR_TYPE_RGB_ALPHA is all that's needed. It is probably more complicated than that.

If the PLTE chunk can't be switched off in some way, maybe it can be reduced. Currently GLBASIC saves the full 256 optional palette. If the number of palette entries could be reduced to 0, 1 or 2 the PLTE chunk might still appear in file but only take up a few bytes instead of 780. There is a num_palette parameter I believe.

[Kitty Hello]

Code (glbasic) Select Expand


   png_set_IHDR(png_ptr, info_ptr, width, height, 8, PNG_COLOR_TYPE_RGB_ALPHA,
      PNG_INTERLACE_NONE, PNG_COMPRESSION_TYPE_DEFAULT, PNG_FILTER_TYPE_DEFAULT);



   /* set the palette if there is one.  REQUIRED for indexed-color images */
   palette = (png_colorp)png_malloc(png_ptr, PNG_MAX_PALETTE_LENGTH
             * sizeof (png_color));
   /* ... set palette colors ... */
   png_set_PLTE(png_ptr, info_ptr, palette, PNG_MAX_PALETTE_LENGTH);
   /* You must not free palette here, because png_set_PLTE only makes a link to
      the palette that you malloced.  Wait until you are about to destroy
      the png structure. */


just remove the last 2 calls?

[mentalthink]

HI I don´t understand very well this functions... but I think it´s a png... very compresed in a xml file?¿ sorry for my ignorance...

And a question some time ago... I comment about streaming reading great images... whit this functions can be read a bit package of info, and finaly have the complete info and draw the huge Image?¿... I read the info goes into an Array... can be possible...

[Kitty Hello]

@metalthink - He uses PNG "data" to save anything. Sound, Music, Level data, ...

[mentalthink]

ok, don´t understand too much, but I suppose someone in the forum put an example...   

[Ruidesco]

He is using the compression algorithm that PNG files use for any kind of file.

[mentalthink]

ok  , curious!!!, it´s the first time I look use a image compresion for any kind of file...  really interesting... I have to read a lot of more about programming   

[Slydog]

@mentalthink, don't worry, I wouldn't consider this traditional programming in any way.
Wampus is exploiting the png compression algorithm to store ANY type of data, not just image related.

Ha, I've been programming for 30 years, and have never done this, or have even thought about doing this.  Very ingenious idea!

You no longer need to program your own zip libraries, just use the device's built in png libraries for the same purpose.  (I assume it use's the device's native png library?).  And PNG files are lossless, so this works great (for example, JPGs wouldn't work as they loose some data in order to make the compression better, so when restoring, the data would have changed).

[spacefractal]

Actuelly a very cool abuse of the png compression.

The only I concern os about the Loading:

- If graphics is set to 16 or 15bit, often the texture would loaded as 16bit, and hence content would been changed. Howover since I seen its using memory and not graphics memory, I think its dont happens here?

[mentalthink]

@Slydog,  HI thanks for your consideration, but sometimes... well , a lot  , I look very impressive things in your codes... I have envious healthy 

[Wampus]

Slydog my code folder is full of failed experiments. This is a rare one that seems to have paid off...so far. I haven't tested it on all my devices yet. Tomorrow I will.

just remove the last 2 calls?

I'm fairly sure removing those two calls will work as intended. Ultimately the test will be to see if the generated PNG files can be opened in browsers/image editors, etc. It should work though, in theory, since the palette is only strictly necessary for images with indexed color. I've seen source code that leaves out those two lines. Example from a cross-platform PNG library - file writepng.c :-

Code (glbasic) Select Expand

/* Set the palette if there is one.
* REQUIRED for indexed-color images. */
if (img->depth <= 8)
{
int i;
palette = (png_colorp) png_malloc(png_ptr,
256 * sizeof (png_color));
/* Set palette colors. */
for (i=0; i < img->cmap_size; ++i)
{
palette[i].red   = img->cmap[i].red;
palette[i].green = img->cmap[i].green;
palette[i].blue  = img->cmap[i].blue;
/* palette[i].alpha = 256 - img->cmap[i].alpha; */
}
png_set_PLTE(png_ptr, info_ptr, palette, img->cmap_size);
}

So, should work.

ok, don´t understand too much, but I suppose someone in the forum put an example...   

I will do that soon. I need to improve how the files are saved. Currently the routine saves two files where only one should be needed. I'll update the code and post a better documented example...probably tomorrow.

- If graphics is set to 16 or 15bit, often the texture would loaded as 16bit, and hence content would been changed. Howover since I seen its using memory and not graphics memory, I think its dont happens here?

I've just tested this: GLBasic will load PNGs with 256 indexed color, RGBA color with 8bit channels and RGBA with 16bit channels just fine. All work (on the PC at the least). However, when you save a PNG file in GLBasic it will always save as RGBA with 8bit channels. Because of this you'll never compress a file with the PNGZ code that will be treated as 16bit and look odd. Even if that were possible there would be a way to work with it.