EGSR (for 2012) followed HPG and started Wednesday afternoon with free access to HPG attendees. One major difference between the conferences is that EGSR has a discussant who has first go at asking questions.
Please note that any errors or omissions are my own.
- Both keynotes were inspiring – the first about the use of the huge volume of publically available images in computer vision, the second about the CG film industry and the rendering techniques used therein.
- The Global Illumination and Geometry sections were the most relevant to game developers.
- Point-Based Global Illumination (PBGI) seems interesting and I really must read the SIGGRAPH 2010 course: Global Illumination Across Industries which includes a presentation about PBGI in games.
- The Progressive Virtual Beam Lights paper was actually an improvement to work to be presented in the future!
Keynote: Big Data and the Pursuit of Visual Realism
This keynote was presented by Alexei Efros from Carnegie Mellon. Early on he referred to the Unreasonable Effectiveness of Data paper from Google employees. This paper refers to simple techniques that use lots of reference data to generate their results. This lead to lots of examples where large numbers of images scraped from public sources such as flickr, Google Streetview, and Google images were used by different techniques. In one of the cases, Alexei highlighted the improvement when moving from 20K to 2M pictures as reference. Another interesting example was a paper based on shrinking images to compare them by Antonio Torralba and Rob Fergus which has not been selected for several conferences that it has been submitted to. As an example, once you can detect similar images, you can do things like colourisation. The other example I remember was “CG2Real: Improving the Realism of Computer Generated Images using a Large Collection of Photographs” which could be used to match computer generated images with real world photos. The final part of the talk was about the bias in the images you can gather from the internet. flickr is populated with personal images with a composition based on the personal preference of the photographer, and in the case of Streetview, the source of the images is even more constrained, although they are taken from the road which is unlikely with flickr imagery.
Polynomial Optics: A Toolkit for Efficient Ray-Tracing of Lens Systems
This paper discussed the need to perform sphere intersections for rays when tracing through lenses in order to correctly calculate the ray’s path. Since these calculations can be expensive for millions of rays and 10s of lenses, the paper aims to provide an approximation. The existing approximation is known as ABCD matrices based on ray transfer analysis and the presenter highlighted that there’s already lots of information available on Wikipedia. The paper indicated that source would be available (and actually now is) for a toolkit of code that can be used to calculate the ray’s path through the lenses.
One audience member did ask about a comparison with true lenses but the presenter indicated that it was near impossible to do due to the materials used to coat the lenses.
The presenter had also previously presented (patent pending) work on Lens Flare rendering at SIGGRAPH 2011.
Flexible Defocus Blur for Stochastic Rasterization
This paper demonstrated a mechanism for controlling the camera defocus blur applied to an object at the vertex level. This control is typically wanted for artistic or performance reasons. However, it seemed that the technique was only applied at an object level and the examples didn’t seem to take advantage of the flexibility. The technique itself fits well into a stochastic rendering pipeline but I think it would need to store extra information for a standard depth of field post process.
Pre-convolved Radiance Caching
The aim of this paper was to improve upon the quality of irradiance caching (which is fast but simple) and the performance of radiance caching (which is slower but provides higher quality results since it has a directional component). The cache was populated with different data for the diffuse and specular elements. The diffuse information was stored as a 4×4 set of normals. The specular information was stored as environment maps that you convolve in their mipmaps. When it comes to sampling the cache, the diffuse information is splatted as normal but the specular information is trilinearly mipmapped based on the gloss. The presentation also highlighted some of the fixups required for difference in location between cached sample and target points when sampling the cubemaps.
Quantized Point-Based Global Illumination
This paper extends PBGI to reduce the storage requirements of the spherical harmonic information from 9 float3s = 108 bytes per node (used for a point or cluster of points) since it’s possible for scenes to have millions or possibly even billions of nodes (as previously shown out-of-core at SIGGRAPH 2011 by Dreamworks). The point information is typically stored in a tree where points are clustered based on spatial locality. The aim of the paper is to convert the tree to a lookup table with an index per node (i.e. 64K entries and 2 bytes per node). To do this, they start from a set of sparse points to generate the initial lookup table values then perform the clustering based on the high resolution points. The lookup index is based on the SH values for the node and the presentation demonstrated a way of generating the lookup before quantising using a streaming mechanism to build the tree. At the end of the presentation, it was shown that there could be issues with too few clusters where some colours may be missed (although the missing color was quite prevalent in the scene). The net size improvement was in the range of 3x to 5x since not all of the data in the tree is compressed.
Progressive Virtual Beam Lights
This paper seems to continue from work yet to be presented at SIGGRAPH 2012 (Virtual Ray Lights for Rendering Scenes with Participating Media) and I’m assuming the ordering is due to deadlines. The SIGGRAPH 2012 work based on integrating lights as lines which leaves laser-like highlights in the scene as singularities. The new version uses beams – i.e. swept sphere lines although the choice of sphere radius wasn’t clarified during the talk. The presentation highlighted the different cases to handle, but the most complex was the samples from the beam lights in the media and how they affect the sampled ray in the media – media-to-media scattering.
(Some of this talk related seemed like it could be related to the importance sampling work done in the Arnold renderer that was presented on Friday).
Image Analysis and Editing
Intrinsic Images by Clustering
The aim of this work is to separate images into their reflectance and shading components. This allows retargeting of images by replacing the reflectance and a given example was a baby image where the baby was redressed and the scene was retextured. Another goal of the work was to achieve the separation without user input which can be quite intensive in some of the example images.
The first step was to extract shading based on luminance. The luminance changes with reflectance so they need to identify areas of similar reflectance which was done using K means in the CIELAB colour space based on chromaticity i.e. independant of luminance. This clustering had the added advantage of simplifying the matrix part of the operation by reducing the number of inputs down to 10s or 100s rather than thousands.
One thing I noticed with the clustering which also came up in the question section was that it was not spatially aware. The audience member suggested adding a spatial component to the colours used in the clustering to handle this.
Artistic Illumination Transfer for Portraits
This presentation discussed how to extract the illumination from an image of a face and apply it to another image. Initially they generated a database of illuminations from existing images and applied those to an average face as a basis. This calculates the required warp to transfer the illumination to/from the average face. They also support user provided images and use a similarity mechanism that picks the 10 most similar illuminations from the database and then rates them to pick the closest. They can also apply the effect to sketch images and paper cuts which gave more impressive results in my opinion.
My notes on this talk are a little sketchy since the presenter was a little difficult to understand.
Matting and Compositing for Fresnel Reflection on Wavy Surface
This paper shows how to take an image and composite it into a scene that contains a reflective surface that may also be perturbed due to waves. This involves breaking the scene into three components:
1) Transmission – the colour of the water, assumed to be uniform for this technique. This is calculated based on the user identifying a feature in the reflected and unreflected parts of the image (automatic matching can fail due to the waves) and then the difference between the colours is averaged.
2) Alpha matte – Initially assumed to be smooth based on the camera and derived from fresnel. This is later updated after calculating the reflection component.
3) Reflection – This requires a better filter than bilateral which they demonstrated and compared.
Finally they raytrace the image to be composited into the scene as a billboard.
Geometry Presorting for Implicit Object Space Partitioning
The aim of this paper is to avoid storing an acceleration structure in order to save memory. The presentation started by highlighting the usefulness of the divide and conquer ray tracing algorithm. The bounding hierarchy is built by defining the node bounds based on boundary triangles and then implicitly dividing the nodes based on preset ordering of split planes. The whole tree can be described by ordering the triangles in a specific way. The paper also presents how to build this hierarchy in parallel.
The results showed that this ordering technique has a performance impact on the ray tracing, but this can be minimised based on allocating some memory for an optimised BVH that branches into the presorted data. I think this was one of the major values of the technique since it allowed you to be selective about how much memory to allocate to the highly optimised part of the hierarchy.
(This was the first presentation of the conference rendered from an iPad.)
This paper shows how generating mipmaps based on how the texture is applied to the surface can generate much better results. Existing techniques for generating mipmaps are unaware of the parameterisation and can therefore filter texels from regions outside of the painted texels. This technique transforms the triangles to texture space and samples the colours from the texture into the grid for the texels of the mipmap.
The paper also discusses how to improve trilinear results but due to ringing artifacts, constrained quadratic optimisation is required. However, the shown timing results didn’t include this. 1 minute for highest quality with a 1024×1024 seems a little high, but I guess that it doesn’t need updating that often, but the process will require the underlying geometry. Overall though, the visual result was much more pleasing than standard mipmapping with an additional sharpness noticeable in the images.
Tessellation-Independent Smooth Shadow Boundaries
This technique extends shadow volumes based on gathering the 1D silhouette and tesselating it to a higher detail level. A more accurate silhouette is based a a previous paper which I believe is “Illustrating smooth surfaces” (http://mrl.nyu.edu/~dzorin/papers/hertzmann2000iss.pdf) which uses normals to interpolate the silhouette position along the silhouette edges. Once the silhouette is calculated, the vertices are displaced based on the normals using Hermite splines and then adaptively subdivided.
Although the shadow results look good (video), the result was slightly jarring when combined with a low resolution mesh, so they suggest that you apply mesh silhouette smoothing techniques although they have to be carefully combined to fit with the shadow volume, possibly including adding some skirts to hide cracks. I was worried about the overhead of rendering heavily subdivided shadow volume geometry (due to concerns about low ROP efficiency and high vertex count) but the timings given for the tesselated render appeared to result in a minimal increase of rendering time.
A Stylized Approach for Pencil Drawing from Photographs
This paper introduces a new type of convolution filter – the swing bilateral LIC (SBL) ﬁlter – to generate strokes that give the appearance of pencil marks. This new filter extends a previous form of convolution filter. The authors identified 3 types of stroke used by artists; feature, object and background. The underlying technique applies quantisation to straighten the lines and generates and applies noise to create the final result. The paper also demonstrates colouring the image with a fixed set of colours, based on either a set of pencils or a user selected set of colours. The results looked good and there were even some comparisons to images created in pencil by actual artists, where the main difference was that the artists had applied additional processes outside the scope of the work, such as smudging.
Surface Relief Analysis for Illustrative Shading
This paper discussed extracting relief detail from surfaces and multiple detail levels. The first step is to smooth the mesh to calculate relief detail. They then intersect the surface with a plane at multiple angles with each intersection generating a line which is matched to a cubic curve. The cubic curve can then be used to extract values like curvature, torsion, ambient occlusion, and then average these values against the multiple planes in the neighbourhood. A video of the results is available here.
A Statistical Method for SVBRDF Approximation from Video Sequence in General Lighting Conditions
This paper is based on the assumption that you cannot move the object so the inputs are moving imagery from a video camera and a 3D model with texture parameterisation which is expected to be collected from a scanner. The first step is to calculate the environment map which allows you to extract a diffuse map from the surface. You can then calculate the specular per texel (which the spatially varying part of the BRDF). The mesh is automatically segmented during the process but can also take advantage of user assistance to direct the segmentation.
Analytic Tangent Irradiance Environment Maps for Anisotropic Surfaces
The aim of this paper is to calculate spherical harmonic tangent irradiance maps to be used to approximate anisotropic effects. The whole presentation was very equation heavy but the net result appears to be a dot product. Apparently this is a simple modification to an existing shader (a sample fragment shader is provided here) so it would be interesting to see how it’s done and what is actually required to fit it into a game pipeline.
Accurate fitting of measured reflectances using a Shifted Gamma micro-facet distribution
This paper is looking for a better approximation for the probability distribution (used for calculating the normal distribution) of macro facets than the commonly used Beckmann distribution when used with the Cook-Torrance BRDF. The paper demonstrates a Shifted Gamma Distribution with two input parameters. The presenters have calculated the required parameter values have been calculated for the MERL BRDF database. The paper also provided an alternative for the Schlick approximation to Fresnel to better match the results in the MERL database. An audience member asked if there was a good representation for the two parameters for an artist, but that’s currently under future work.
Importance Sampling Techniques for Path Tracing in Participating Media
This presentation is based on the Arnold renderer from SolidAngle and was previously a SIGGRAPH 2011 sketch (Importance Sampling of Area Lights in Participating Media). For point or sphere lights, the paper suggests using equiangular sampling along the eye ray to get the best results from fewer samples. To extend the technique to heterogeneous media, the paper introduces decoupled ray marching whcih precalculates volume properties at sample points to avoid the expense of linearly accumulating properties such as transmittance as in previous methods.
Fast Generation of Approximate Blue Noise Point Sets
This paper presented a mechanism for generating sample points based on a poisson disk distribution in which there is a rule that each point is separated by at least a minimum distance from all other points. Previous works have used a dart throwing algorithm (more correctly known as stochastic sampling, for example) which has subsequently required optimisation due to the slowness of the technique. The presentation showed how to generate a larger patch of points by applying a small patch of points to a jittered grid with a rotation and subsequent crop. The splatting of the points into the output breaks the rule of minimum separation distance, but the error can be bounded based on the number of points in the initial sample.
ISHair: Importance Sampling for Hair Scattering
This paper was from Dreamworks who had been looking at hair rendering techniques based on an internal request for the upcoming film: The Croods since the alternative is for the artists to manually light the hair with CG lights. The paper adds importance sampling to the existing work of Iman Sadeghi et al “An artist friendly hair shading system“. The focus was on the gaussian part of the equation since that adds the most variance and the implementers used a Cauchy distribution to sample the gaussian.
The result images are here and the presentation also had impressive images based on the assets from the film itself.
Example-Based Fractured Appearance
This technique extracts statistics from an example of a fractured surface to create sets of fractures in a similar surface. They used a user study to decide the most important statistics for generating matching fractures. The underlying system uses the finite element method to propagate the cracks. The actual statistics for the generated cracks are randomly generated based on approximate Bayesian computation. The presentation also demonstrated an interactive editor which allows them to crack things based on a set of examples.
Gamut Mapping Spatially Varying Reflectance with an Improved BRDF Similarity Metric
This presentation was about creating metallic output on a printer with metallic inks that matched an example image. To test similarity between BRDFs they developed a special environment with a specific inverse squared falloff, which was slightly odd since it implied that the light was immediately behind the viewer. The implementers also added a clustering mechanism to maintain detail where there is contrast between the regions. They also tested their results against the MERL dataset.
The closing keynote was presented by Eric Tabellion from Dreamworks, who has contributed to all of their works from Shrek to date. He started with an overview of Dreamworks as a company, where they’re based and how many people work for them. He then went further into some of the rendering techniques that they use. One of the more interesting ones were Deep Frame Buffers as presented by Jonathan Ragan-Kelley at SIGGRAPH 2007, (paper here). In terms of shadows they started with an adaptive PCF technique that sounded a lot like PCSS. They then moved to dual depth layers by Weiskopf et al presented at Eurographics in 2003 (pdf here)
He also mentioned their work on Point Based Global Illumination (PBGI) starting from Michael Bunnell’s 2005 work “Dynamic Ambient Occlusion and Indirect Lighting” in GPU Gems 2. PBGI was further discussed in the SIGGRAPH 2010 course: Global Illumination Across Industries (which includes a presentation about PBGI in games). Eric will also be presenting at SIGGRAPH 2012 about PBGI shading optimisations.
In the question section of the talk, an audience member asked about their use of threading due to their huge datasets. Eric said that Dreamworks currently aim for multiprocessor rather than multithread since it’s easier to manage in light of lots of legacy code, but it does suffer from greater IO impact.
When asked what he’d do if he could redo it all from scratch, Eric reiterated that in future he would like to move to something like a Monte Carlo raytracer.
The next EGSR is expected to be hosted in Zaragoza which is 1h20 from Madrid and Barcelona by high speed train. The date is not currently set, beyond approximately June.
There was no wifi at the venue! At the start of HPG, the proceedings were shared on an adhoc network but those have issues with connectivity with some phones and tablets. Other than that, you only had what you could sync at the hotel before leaving – I discovered pocket!
At both conferences, there was no coffee and no pastries on the morning after the socials – possibly due to expecting low attendance, but surely those attending will need them!?
I’ve also found some other coverage on Jeroen Baert’s blog here: