Realistic Materials in OpenGL ES

My brother has shown me a sculpting application with ability to assign materials from images and render them in real-time. Since they looked so great, just like raytraced(after briefly blaming the developers of the application of witchcraft), I started researching the subject and ended up writing my own implementation.

View demo

Matcap

Matcap(Material-Capture), is a technique of combining material properties and lighting information in an image. Basically image is interpreted as a hemi-sphere, each point on sphere is matched against a normal. When object is rendered with matcap material, normals are calculated for each fragment, and that information used to map matcap image and get fragment color.

Some matcap materials

Reflection and specular

Without actually making any lighting or reflection calculation in the shader, we are using a precalculated table, so very sophisticated looking rendering with reflection, subsurface scattering, specularity, becomes possible.

Lights are embedded

Limitations

However, since matcap is visual information of an object, seen in a certain camera angle and certain ligthing conditions, the material itself is actually not lit, meaning no actual lighting is done on the shader, the light is fixed in related to camera. When your camera moves, light also moves with it.

Another limitation of matcap is not being able to use any kind of texture on MatCap image. Since matcap mapping is in relation to camera any texture you apply on to matcap material will look like environment mapping.

Lights are embedded

Shaders

In vertex shader, we first get the vertex normal attribute and move it into model(world) space and assign it to an interpolated vector.

In fragment shader, the interpolated value of the normal is transformed into camera space and normalize. Each component of normal now in [-1,1] inverval, we move them into texture space [0,1] and ignore Z value.

Vertex shader:

vNormal = normalize(vec3(world * vec4(normal, 0.0))); // Normal in model space

Fragment shader:

// Move normal to view space
highp vec2 muv = vec2(view * vec4(normalize(vNormal), 0))*0.5+vec2(0.5,0.5);
// read texture inverting Y value
gl_FragColor = texture2D(matcapTexture, vec2(muv.x, 1.0-muv.y));

Source code & Demo

You try matcap materials in demo I wrote here. You can also obtain source code from my Github repo, or just view source of the page.