Shadows popping in/out at edges of screen

[Zoomastigophora]

Currently, shadows and geometry have a visible pop in/out at the edges of the screen due to culling too early.

I'm not sure what the previous comment is referring to with shadow maps becoming too low-detail with a larger culling frustum, but one way to fix shadows popping in at the edge of the screen is to widen the culling frustum so that objects aren't culled as early. The downside is that objects will get rendered even though they appear off screen, which is somewhat inefficient. A better solution would be to decouple shadow map rendering from model rendering; as it is now, a shadow map is only rendered if its associated model is to be rendered (I think).

[Zoomastigophora]

Patch for culling frustum widening.

[Philip]

The shadow map detail is an issue because (as far as I can tell) the shadow-renderer picks a frustum that's large enough to fit in all non-culled objects (with the frustum axis matching the direction of the sun), and renders that (from the sun's location) to a fixed-size shadow map, and if fewer objects are culled then the frustum will have to be larger and each shadowed object will cover fewer pixels and be less detailed. (Enable the m_DisplayFrustum flag in Renderer.cpp then try a map with a small number of objects (maybe make one in the Atlas editor) and move the lighting around, to see what it's doing.)

Just expanding the culling frustum won't solve the problem properly, because far-away objects might still cast a shadow - the worst case is a tall building on a cliff with the sun at a low angle, so it casts across the entire map. It seems like it ought to be possible to handle this correctly - construct the shadow frustum based on all the objects and terrain patches that are visible on the screen (i.e. might receive shadows), then find all the objects and patches that intersect that frustum (i.e. might cast shadows) and render those onto the shadow map, and then do the normal rendering based on the normal tight visibility culling. I don't know how inefficient that'll be, but it seems better to focus on correctness now (to avoid ugliness with low sun angles) and optimisations later.

This patch is still an improvement over the previous code so I'll commit it now, but I'll leave the ticket open since it'd be good to fix this more robustly.

[philip]

(In [7562]) Expand the culling frustum to reduce shadow popping (see #504). Based on patch from Zoomastigophora.

[Zoomastigophora]

Weird, I didn't get an email notifying me of a reply to this ticket.

Anyway, I'm in agreement with you, hence my comment about the need to decouple shadow map rendering from object rendering. From what I could understand based on the code, Renderer maintains an internal list of models to be rendered at every frame and calls each models Render() when it's time to render everything. Each model then handles drawing itself and its shadow. This is where it gets confusing for me; from what I can tell, ShadowMap? is a collection of matrices and lighting values that a RenderModifier? uses if its render mode is set to MODELFLAG_CASTSHADOWS. Beyond that, I get lost due to the fragmentation of the code across so many files and classes (for instance, I still haven't figured out where the actual generation of the shadow texture is being done). Still, the main issue is that shadow map rendering is handled by models and what you're describing is generally how games do shadow rendering from my understanding i.e. a separate pass entirely for shadows, ignoring whether an object's model should be drawn or not. I don't mind trying to impelement this, but it'll take some time unless there is technical design documents and interaction diagrams I haven't seen.

[Philip]

Hmm, maybe I need to turn Trac's always_notify_reporter on so that you get this email...

There isn't any documentation other than what's in the code. As a historical note, the renderer started as a simpler design with no support for shaders or instancing and with most of the behaviour hard-coded or based on flags. It got largely redesigned later by a different programmer, adding those features and adding abstractions for passes and modifiers and abstract base classes and all that stuff, making it more powerful and more complex. So the current code is a combination of those two styles.

My current understanding is: I don't think it's accurate to say each model renders itself - the shadow map generation is driven by CRenderer::RenderShadowMap, which calls m->Model.Normal->Render(...) etc, where m->Model.Normal is a ModelRenderer. Each model renderer stores some representation of a list of models which have been Submit()ed to it via CRenderer::SubmitNonRecursive.

SubmitNonRecursive is called most commonly by CCmpVisualActor::RenderSubmit, which is part of the simulation code - the graphics code doesn't maintain a persistent list of renderable models, it just asks the simulation code what models ought to be rendered at this instant. (Many models are submitted by a single CCmpProjectileManager::RenderSubmit - there's not a one-to-one correspondence between models and simulation entities).

All those RenderSubmits are called by CGameView::EnumerateObjects, which is called by CRenderer::RenderScene.

If we want to do shadow-culling differently to visibility-culling, I think that means CRenderer::RenderScene should call CGameView::EnumerateObjects once with the camera frustum to collect all the visible objects (like it does now). Then it should compute the shadow-rendering frustum, and call CGameView::EnumerateObjects a second time with the new frustum and a new scene collector (which submits to a new set of ModelRenderers) in order to collect all the shadow-casting objects. Then CRenderer::RenderShadowMap can use the new set of ModelRenderers, and it's independent of the normal non-shadow models.

[Jayschwa]

#142 is a duplicate of this issue. I left this one open since it has more detail. As #142 pointed out, this issue also applies to objects which can be seen in reflections.

[anonymous]

Milestone Unclassified deleted

[asmartgoat]

With my map. "arabian nights, when the tops of the dunes are out of view the shadows click in and out of view. I'm running from SVN, compiled in late December."

[asmartgoat]

I can also find a few other maps that this is a problem with, maps with cliffs oe steep hills combined with sunset meet this fate.

[historic_bruno]

There's nothing reviewable here AFAIK. The patch was already reviewed and applied years ago, but it's not a good enough solution.

[k776]

Moving to backlog. Perhaps myconid's upcoming changes to the renderer will make generating shadows a lot less problematic. We'll wait and see. Refs #579