GLBasic forum

Main forum => GLBasic - en => Topic started by: Zapp on 2012-Jun-08

Title: Virtual Screen & Stencil Shadows?
Post by: Zapp on 2012-Jun-08
UPDATE: 29/06/2012 - Reworded question to make it clearer for the community  :|

Hi Guys,

After leaving the problem for a while to think about it and then coming back to look at it again, after some research it seems that the correct terminology for what I wish to accomplish is stencil shadowing.

Wondering if the following is possible to do the following and if so how would I go about it? I've also attached an image from photoshop explaining what I'd like to accomplish, sadly Mentalthink's method will only assist with 1 lightsource, I need a method that is capable of multiple.

1. Create a Virtual Screen for calculating shadows.
2. Draw a Solid Black Rect for the shadow.
3. Draw over this rect with a gradient where light would be cast.
4. overlay this over my topdown 2d world.
5. Is this possible on IOS without massive slowdown? (I'm using polyvectors and need to maintain at least 30fps)
6. Is this possible with standard GLBASIC or will it require inline code?

Kind Regards,

-Zapp

(http://i.imgur.com/0D7Xf.png)


Hi Guys,

I'm usually pretty good with working out inventive ways to solve a problem on my own, but this one has me stumped.

So I have a small action rpg that I'm working on. The Room is top down, I'd like to be able to have some rooms with different levels of darkness to make the game a little harder.
I've managed to get to this stage just fine as a temporary measure I am able to cover the 2d environment with a solid black texture and vary the transparency using ALPHAMODE.

What I really would like to be able to do is somehow allow for lighting on the level, so for instance if the character has a torch, there is an area around him that isn't shadowed, likewise if there is a candle on the wall, then that area would also be without shadow.

Essentially I want to start with a Black fullscreen texture for shadows and then cut out what isn't in shadow by making it transparent. Is this possible and if so how would I go about doing it? Would it severly impact framerate by being too slow a process? Perhaps since my game is 2d Tile based, I may need to calculate shadows properly and use a shadow map, but I'd like to avoid the extra work of calculating all those poly's, if there is an easier solution that can be achieved by only drawing one poly.

Thanks in advance for the assistance.


-Zapp
Title: Re: Stumped - Cutting Transparency into a Black Image
Post by: okee on 2012-Jun-08
Hi Zapp

Is this topic of any use ?
http://www.glbasic.com/forum/index.php?topic=8124.msg68348#msg68348
Title: Re: Stumped - Cutting Transparency into a Black Image
Post by: Zapp on 2012-Jun-08
Hi Okee,

Thanks for the suggestion, away from development pc at the moment, will take a look at it in the morning.

Regards,

-Zapp
Title: Re: Stumped - Cutting Transparency into a Black Image
Post by: mentalthink on 2012-Jun-08
Hi Zaap I comment this , I think in the post Okee comment, but it´s very easy, only have to do a black great image whit a hole in the center, making whit settrasparency...

The better way it´s in photshop, make the edges in the black hole whit a blur effet this it´s more real else it´s too much cartoon style...

Title: Re: Stumped - Cutting Transparency into a Black Image
Post by: Zapp on 2012-Jun-09
Hi Mental,

Thanks for the reply, I think in this case this may not work, I need to be able to have multiple holes in the shadow. I may just have to go with a full screen poly an use co-ords to fill the screen with black transparency or a small gradient.
Title: Re: Stumped - Cutting Transparency into a Black Image
Post by: mentalthink on 2012-Jun-09
Hi zaap, well I was do another solution, but a little trouble was the constrast whit the colors...

You can try this, but perhaps don´t looks well...

You have your screen, your graphis and characters moving, ok... now, put over the screen a rectangle drak whit the sizes of the screen, and now you draw 2 circles whit white color...

I don´t rebember the Alphamodes values... for me works, but the colors don´t have too the perfect detail...
Title: Re: Stumped - Cutting Transparency into a Black Image
Post by: Zapp on 2012-Jun-10
Hi Again Mental,

Thanks for the suggestion, I've actually tried that thinking it wouldn't work but gave it a go anyway. You are correct it looks too washed out, the colours are messed up.
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Zapp on 2012-Jun-28
Hi guys,

I've updated my main post with a bit of a clearer description of what I am trying to do. I have a feeling the answer may require inline code with gl commands and may be beyond my current limitations, but I'd appreciate if anyone can answer my questions.

Kind Regards,

-Zapp
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Slydog on 2012-Jun-28
If you managed to create your middle picture (the one with the 4 white dots and a white arch) then I think it may be possible using alpha transparency.

Change your black / white values to all white, and instead convert that to various alpha values.
The black parts of your original image would be a colour such as (0,0,0,150) and your white would be (0,0,0,0).  And any grey would be interpolated between those.  Adjust the '150' until it looks right with enough dim.

Then just draw your game scene as usual and simply overlay the above picture over your scene.  The white / no-alpha portions will show your scene untouched, and the black / partial alpha will dim the parts it overlays.  And it shouldn't slow down your game much, just use polyvectors (four corner points) and cover the entire screen with the sprite.

[Edit] If you are doing this on the fly, you would need a way to specify the alpha values.  A new / updated colour command would be handy!  RGBA(r, g, b, a)

You may be able to figure that out on your own, I've never tried, but found this snippet to read the alpha value (never tested!):
Code (glbasic) Select
FUNCTION GetAlpha%: colour%
RETURN bAND(colour/0x1000000, 0xff)
ENDFUNCTION
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Slydog on 2012-Jun-28
Another cool benefit of overlaying your scene with the generated light sprite, as outlined in my post above, is that your light sprite doesn't need to be the same size (pixel wise) as your scene.  You could scale it by 1/2 or 1/4 and stretch it over your scene.  And this may save processing time while generating it.  The stretched sprite should still look very close and any blurring you get from stretching may help the light effect.  Heck, you could even try a 1/32 scale light sprite.
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Zapp on 2012-Jun-28
Hi Slydog,

Thanks so much for the response! I can make sense of what you are saying somewhat, still a little lost on the process.

I was originally planning on just using screen size (480x320) and scaling up as required for the higher resolutions of the other devices. However I think you are right, that 320x160 would work better and scaling it up might smooth the lighting a bit better. I'm a little lost with the conversion from white to transparency that you mention, can you give a little more assistance with this? I am assuming this would require a loop through of each pixel in the image to adjust its data? If so, a smaller sprite always be a benefit.

Forgive my ignorance, but which commands would i be using to adjust pixel data? The help menu suggests that getpixel/setpixel would be slow.
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Slydog on 2012-Jun-28
I'm in a bit of a rush, I may expand on this later!

What I meant was to NOT create the black / white sprite, but instead create an all black sprite with varying alpha values.  These alpha values will show more or less of the game scene, depending on where your lights are.  Where your lights are is where you put the '0' alpha values (which would just show the game scene at that location 'as is' since the alpha is 0/transparent), and where there is no light is where you put the '150' alpha values to partially darken the scene.

The problem is I don't know exactly how to specify the alpha when creating a pixel.  RGB() only allows setting the red, green, and blue, but no alpha.  I think a simple reverse of the 'bAND(colour/0x1000000, 0xff)' is the key.
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Zapp on 2012-Jun-28
Ok thanks! That makes a bit more sense. Would be greatful if you could expand later. In the meantime I'll try to work out the Band method.
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Slydog on 2012-Jun-28
Try the following (untested)!:
Code (glbasic) Select
FUNCTION RGBA%: r%, g%, b%, a%
    RETURN bOR(RGB(r, g, b), ASL(a, 24))
ENDFUNCTION


I based it from a post by @spacefractal:
http://www.glbasic.com/forum/index.php?topic=7826.msg65579#msg65579 (http://www.glbasic.com/forum/index.php?topic=7826.msg65579#msg65579)

[Edit] To use this for your project, you'd do something like this:
Code (glbasic) Select
cDark = RGBA(0, 0, 0, 150)  // Use this colour returned for your darker areas
cLight = RGBA(0, 0, 0, 0)  // Use this colour returned for your light areas


I think you can use SETPIXEL with these returned values.
And/or try a DRAWRECT first with the cDark colour to fill in the entire light sprite.
Then just draw circles using the cLight colour where your lights are.

I think you need an ALPHAMODE -1 before you plot the light sprite over top of your scene for the DRAWSPRITE / POLYVECTOR command to use your alpha values in your pixels.
Title: Re: Virtual Screen & Stencil Shadows?
Post by: erico on 2012-Jun-28
I see no reason why your stencils like the ones on the picture would not work.
For one thing, you could use half resolution for that stencil, and maybe use 2 screens to the light flicks.

Still, working all lights with a single stencil off screen seems to me the best option. Would have to think more about it.
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Hemlos on 2012-Jun-28
I dont know if this is exactly what youre looking for, but it is simple.
You can adjust the resolution of the spotlight too, to tweak it for more performance.


A parrallel project i made, giving a magnifier effect with the same sort of technique using poly is in discqus_magnifier.rar

More importantly, Disqus1_spotlight.rar is a test project i made giving a stencil shadow effect.

Hope this is useful, goodluck!

[attachment deleted by admin]
Title: Re: Virtual Screen & Stencil Shadows?
Post by: kanonet on 2012-Jun-29
Wow i really like Hemlos approach would have though about that way to do it, it needs a few adjustments to get it working the way you want it (and multiple lightsources, that even overlap each other), but its a really great idea.

My idea how to do it would be like Slydog suggested. Just a few thoughts about how i think it can be done:
-best way to change values is to use MEM2SPRITE, its faster than GETPIXEl etc.
-how to set alpha values is a bit hidden in the help, it should be under RGB(), but you need to look under SPRITE2MEM to find it:
rgba = bOR(RGB(r,g,b), ASL(a, 24) )
Title: Re: Virtual Screen & Stencil Shadows?
Post by: erico on 2012-Jun-29
maybe our RGB stuff could be implemented to read RGBA?
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Albert on 2012-Jun-29
I have this test app. You can test all the alphamode settings and different types of lights.

download: https://dl.dropbox.com/u/292449/glbasic/Shadow/Shadow.zip
(https://dl.dropbox.com/u/292449/glbasic/Shadow/shadow_zapp.png)
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Zapp on 2012-Jun-29
Quote from: Hemlos on 2012-Jun-28
I dont know if this is exactly what youre looking for, but it is simple.
You can adjust the resolution of the spotlight too, to tweak it for more performance.


A parrallel project i made, giving a magnifier effect with the same sort of technique using poly is in discqus_magnifier.rar

More importantly, Disqus1_spotlight.rar is a test project i made giving a stencil shadow effect.

Hope this is useful, goodluck!

Wow Hemlos, that might actually do it and so easily too! Thanks!

Edit: On closer examination the code falls over when you overlap spotlights, so I'll need to play with all the suggestions and see what works best.
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Hatonastick on 2012-Jun-30
The way I'd have done it (although it's probably not the most efficient way -- and might not even be possible as it's been ages since I coded anything) is this:

1) Create two work screens.
2) Colour work screen 1 black.
3) Draw all of my light sources on work screen 1. eg. white circles.  Use of various drawmodes might be required to blend light sources together if you are using gradients rather than just plain white.
4) Draw all my game graphics on work screen 2.
5) Alpha work screen 1 over work screen 2.
6) Display work screen 2.
7) Repeat.

This method, if it is possible in GLB, has some obvious drawbacks (and is a bit of a brute force approach rather than an intelligent one) but it would allow you to blend light sources.
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Zapp on 2012-Jul-02
That's essentially what I am already doing for every SHOWSCREEN

Screen 1 = World
Screen 2 = Black square with white circles for lights with gradient.

Draw Screen 1
Draw Screen 2 over Screen 1
SHOWSCREEN

The tricky part is:

1. Getting the white to not be white but rather a gradient from black to transparent
2. Maintaining 30fps on an Iphone (somehow I don't think this will be a problem)

I'm back to looking at number 1 right now after a weekend off, using all the wonderful suggestions given above.

Kind Regards,
-Zapp
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Hemlos on 2012-Jul-02
Quote
Edit: On closer examination the code falls over when you overlap spotlights, so I'll need to play with all the suggestions and see what works best.

Hmm, you tried alphamode with it(only the poly)?
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Zapp on 2012-Jul-12
Hi Hemlos,

Sorry for not replying sooner, I had to rebuild a large portion of code due to it not being flexible enough, I've just got back to the shadows and spent a few hours tinkering with your code. I must of completely misunderstood how it worked the last time, as I've played with it enough to get it to where I want it. Thanks again!
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Hemlos on 2012-Jul-13
Awesome, cant wait to see your game in action! :good:
btw, what plaform is the game for, in the end?
Title: Re: Virtual Screen & Stencil Shadows?
Post by: Zapp on 2012-Jul-14
IOS initially, if it's popular I'll probably look at releasing it on Steam.