Heyas
Im having an issue with loading images, and my ram being wiped out.
Is there something about LOADSPRITE that im not aware of?
i have 8 gigs of ram installed.
The os uses 2 gig, the program uses about 4 gig, and the gpu is using 2 gig of onboard ram.
When i load about 38 images of around 1mb~2mb each, jpg format, my program ends.
This group of images total about 45 mb.
The windows7 os complaining about resources being depleted, and the program is forced to shutdown.
I was watching my task manager, and i noticed well over 4 gig of ram is being used up by the program, very odd.
The ram was disappearing slowly as the program loads each of the images....it takes about a minute to get to #38 image.
Thats about 1 gig of ram being used for roughly 8 images.
Could it be because GLB internally stores the images as an uncompressed format + alpha?
How big is one of those images?
Each images is less that 2 mb.
What are the image dimensions? for example a HD 1920x1080 works out about 8MB of memory usage, where as a 12MP digital camera image takes up 47MB.
Jpeg file sizes can be misleading, for example a 2MB jpeg image could have a low resolution & low compression or have a high resolution with high compression.
Perhaps the decompression of the jpg may be affecting things?
Lee.
Yep, that is what I meant too...what is the size of the image in pixels x pixels?
I think youre right, to a certain degree, the images are 3072x3072 96dpi 24bit
So one image is about 27mb as a BMP conversion.
This is still not really explaining the 4 gig usage...im calculating the size now...about 1 gig for the 38 images....not 4gb as windows taskmanager is showing.
30mb x 38 = 1140mb
hmmm...even if you add another 8bit for internal transparency, it would still no reach that, as 36mb per image.
I don´t know what to think, are your jpgs RGB or CMYK? Do they have extra stuff, like guides and paths into them?
edit: can you post an image sample zipped?
glbasic is a 32bit application, so its property crashout when its try to allcoate more than 2gb ram. This is the typical limit on a Windows application. Few applications is designed to uses 3gb ram in a 32bit app, but im dont thinks glbasic is designed for that.
howover is glbasic is compiled as 64bit application, its can uses much more ram.
LOADSPRITE with very big texture and pictures take a lots of ram, since its uses 4 bytes per pixel.
Well thought! That should be the reason me guesses. So it would probably be best Hemlos loads images in batches.
GLB being a 32bit program did cross my mind but I discounted it mainly because each image is around 37MB in raw RGBA for, multiply that by 38 images equals 1.4GB of space so under the 2GB win32 limit imposed on by most win32 apps.
However I done a simple test listed to below just to see memory usage out of curiosity & saw things I didn't really expect to see. The images loaded are the size the filename says, so 1024k is a 1024x1024 image etc & the images are just simply rendered clouds from photoshop to give some randomness to them.
The program starts using 12,916KB of ram before loading the 1st image, then jumps to 25,536 after image has loaded (1024k one). The next image uses 58k then drops to 38K but pagefile size increases, then 142K ram, 168K ram then finally 563K. After every LOADSPRITE the ram increases as stated earlier but as soon as the image is displayed on screen it drops down to 38K & pagefile usage increases. At one point (on the last image) the page file increases to over 1GB higher before running the program then drops down just over 320MB from initial.
All the images combined add up to around 376MB in raw RGBA form yet system ram reports only 38MB used, adding the page file increase gets me around the 376MB as expected.
What causes the big ram jump in the 1st place I have no idea other than possible jpg decompression, also why the pagefile usage is beyond me but expect nothing less from M$ & their never ending addiction to using the pagefile.
Lee
SETCURRENTDIR("Media") // go to media files
SETSCREEN 800,600,0
KEYWAIT
LOADSPRITE "1024k.jpg",1
DRAWSPRITE 1,0,0
SHOWSCREEN
KEYWAIT
LOADSPRITE "2048k.jpg",2
DRAWSPRITE 2,0,0
SHOWSCREEN
KEYWAIT
LOADSPRITE "3072k.jpg",3
DRAWSPRITE 3,0,0
SHOWSCREEN
KEYWAIT
LOADSPRITE "4096k.jpg",4
DRAWSPRITE 4,0,0
SHOWSCREEN
KEYWAIT
LOADSPRITE "8192k.jpg",5
DRAWSPRITE 5,0,0
SHOWSCREEN
KEYWAIT
I wonder if there is a problem with JPG decompression... Could you make the files available ?
Just done the same tests again but with BMP's & PNG's & witnessed exactly the same memory pattern as before.
I must mention that all tests where done with debug disabled so as to remove that from being a possible cause.
Pagefile usage in my case may be down to using 32bit windows 7 with 2GB of ram as my main machine is down at the moment. However it has 1.4GB free after booting windows & startup processes complete so more than enough free for the test program which shouldn't use more than 400MB inc the images.
Lee
What happens if the process is replaced with a simulated CREATESCREEN, which only 32 can be created? :|
Hmm, thanks for the help guys. And thanks for the testing.
ITs interesting, perhaps Gernot will drop by and give us some more input when he has a chance.
For now, i have resorted to images that are 10 times small as an alternative.
I will limit and use the hd images only when i need more info from them.
Heres the project i am working with right now so you guys can see what im doing....ps keep in mind, this is alpha stage, and it needs some optimizations...theres also a few more options that are not added yet...like frame count limiting how many frames will animate etc....
Any ideas to add to this are welcome, i will be publishing this program in the very near future.
-Neil
PS..the download is attached to this message.
Juuuust tested, looking pretty good!
Are the original images from nasa a circle or a rectangle? You could map that to 3d maybe.
Everything worked out fine.
I had a crash on first run on my opengl card, probably related to resolution change or full screen.
Second time it got windowed and worked fine.
The images are "as is" found on thier website. They are square jpgs.
I just netwebget "latest_ful.jpg" ...thats it.
I name them with timestamps.
And you must run it for a day or two to get a decent animation.
Note: it does not download an archive...its only a latest live version when you run it.
If it downloads an image of the same kb it deletes it and tries again at the next interval.
Edit;
note:
the images there are from the last 2 days since i got it running correctly
I must point out there was a post sometime back with regards to problems with some images acquired via netwebget. If I remember correctly they where greyscale JPG's upon inspection. I will have to see if the post is still about on the forum.
Lee
EDIT - Found it http://www.glbasic.com/forum/index.php?topic=9220.0 (http://www.glbasic.com/forum/index.php?topic=9220.0)
If it is any consolation, it does work fine on x64 Windows 8 - there is no memory change; it is all constant.
It does sound like a limitation of 32-bit Windows - it could be having to use the page file for it's work...
Quote from: MrTAToad on 2014-Aug-08
It does sound like a limitation of 32-bit Windows - it could be having to use the page file for it's work...
I compiled on an 64 windows 7
AMD a10 trinity w/8gb ram and 250gb ssd......
....with or without the paging file, i get the same exact issue....and the paging file doesnt seem to do anything as far as taking some of the memory load when the ram becomes critical
Sounds to me....a memory leak in LOADSPRITE
I wonder if there is still a loading problem with JPEGs...
Quote from: MrTAToad on 2014-Aug-09
I wonder if there is still a loading problem with JPEGs...
I just tested it by converting the jpegs to png....they went from 1 mb to 10 mb image file size.
And...The amount of memory being sucked up is the same.
Quote from: fuzzy70 on 2014-Aug-06
Just done the same tests again but with BMP's & PNG's & witnessed exactly the same memory pattern as before.
I said that earlier.
Probably a dumb question but could it be related to the size. For example gfx cards have differing texture size limits?, assuming GLB treats a loaded image as a texture that is.
Lee
Since you are loading images into glbasic, you need enough memory for them on the grafics card. I'm not sure how GLBasic handles this but I can imagine there will be trouble trying to load 1.5 GB of data onto the grafics card if it has shared memory with the computer. Are you doing an animation of these live pictures? I have done something similar, I downloaded them locally like you and used a program to combine them into a normal Xvid movie instead of even attempting to load them into GLBasic :-)
Quote from: Hemlos on 2014-Aug-15
Quote from: MrTAToad on 2014-Aug-09
I wonder if there is still a loading problem with JPEGs...
I just tested it by converting the jpegs to png....they went from 1 mb to 10 mb image file size.
And...The amount of memory being sucked up is the same.
Not surprising that the converting to PNG would produce large files :)
I presume that memory was still being continually consumed - does your program still crash after a while ?
One thing : How are you removing the graphics (assuming you do), or are you just reloading them over each other ? I was wondering if, when doing something like LOADSPRITE "",0 - that the memory being used by that sprite may not being totally freed, and thus a reload would consume more, and one after that would consume even more, and so on...
If you are loading without freeing, then its possible that LOADSPRITE isn't deleting memory beforehand.
Quote from: MrTAToad on 2014-Aug-15
If you are loading without freeing, then its possible that LOADSPRITE isn't deleting memory beforehand.
Aye
Well if the live image is a dupe....
LOADSPRITE "", self.INDEX
....and this index is reused later on when it isnt a dupe.
I made very sure that the memory doesnt have a leak when loading the images.
Im assuming saving as a BMP as opposed to a PNG isnt much of an alternative either.
Quote from: Moru on 2014-Aug-15
Since you are loading images into glbasic, you need enough memory for them on the grafics card. I'm not sure how GLBasic handles this but I can imagine there will be trouble trying to load 1.5 GB of data onto the grafics card if it has shared memory with the computer. Are you doing an animation of these live pictures? I have done something similar, I downloaded them locally like you and used a program to combine them into a normal Xvid movie instead of even attempting to load them into GLBasic :-)
My graphics card has 2 gig ram, thats why i dont understand why my binary is clogging up the main cpu ram when the images are about half that total size.
Yes, i am itterating through ZOOMSPRITE, incrementing the index every 128 milliseconds to make an animation which includes the latest live downloaded image.
Display adapter type AMD Radeon HD 6670
Total available graphics memory 5863 MB
Dedicated graphics memory 2048 MB
Dedicated system memory 0 MB
Shared system memory 3815 MB
Display adapter driver version 14.100.0.0
Primary monitor resolution 1920x1080
DirectX version DirectX 10
The available is higher, because this is an APU, which uses the gfx card memory first, after that is filled then it will spill into the motherboard ram.
When the sucks up about 6 gig of ram on the mb(during load time because i already have 40+ images ready to load)...this is when the app crashes.
The crash happens when it is loading image # 38+, not before.
This isnt a wierd native bug i created either, because a test case shows:
im using another version of the same routine, which is loading images that are only 100k in size.
...and so far i have not exceeded a gig of image space, yet....im at frame # 69 now.....that is 69 x 2 = 138 images being loaded. (2 satelites animating)
I think i figgured it out!
On Linux....works fine minimized, i can keep it minimized for days and it still renders....
On windows.....i think.....DIRECTX is dorking the rendering when the program is minimized?
You have stumbled on something there!
Tried it with a test program here, and memory usage is slowly increasing when minimised and stopped when the program is restored!
I have left mine running (minimised) whilst I'm out - the program starts at 30Mb, so it'll be interesting to see how much it is when I get back. It looks like 0.1Mb is consumed every second, so theoretically it should have increased by around 30Mb by the time I get back :)
Weird - its gone down to 9.1Mb :blink:
However, the program has crashed :S
Sprung a memory leak. :giveup:
I think we've found a bugette!
All in a days work.
Celebrate :booze:
I dont like this bug....hope we get an exterminator soon.
Yes, it is a bit of a naughty one!