TouchDesigner - GLSL Image Displacement

Requirements

TouchDesigner 099 2017.15400+
今回の完成ソースはここ。

Description

Open GLSL_Displacement.toe

Image Displacement can be easily setup in TouchDesigner using TOPs, SOPs and CHOPs.
But the SOPs and CHOPs calculate on the CPU, performance will slow down as more points are added to geometry.
Example /base_top2sop_displacement
Try increasing Grid SOP Rows/Columns = 250.
TOP -> CHOP - can be slow
CHOP -> SOP displacement - can be slow

GLSL Materials can be used displace geometry on the GPU for better performance so you can use many more points.
Example /container_glsl_displacement
TOP -> GLSL displacement - fast!

Prepare Geometry

Create Grid SOP -> Geometry COMP.
Grid SOP settings
Rows = 250
Columns = 250

Add Attribute Create SOP to calculate tangents for GLSL shader later.
Attribute Create SOP settings
Compute Tangents = On

Basic geometry setup

Create GLSL Material

Easy way to start, output shader from Phong MAT

Click Output Shader in Phong MAT parameters

Click OK on Output GLSL Code... dialog

Result is GLSL MAT with all the same settings as Phong MAT

Assign phong1GLSL to Geometry COMP

Add Image Sampler to GLSL Shader

Create Noise TOP -> Null TOP
On phong1GLSL Samplers page, assign Null TOP and give it a sampler name dispTexture

Declare sampler2D dispTexture in vertex shader

Displace Point Positions Using Sampler

1. Use texture() function to sample red channel of dispTexture
2. Create newPos which adds offset to original position
3. Use newPos in TDDeform()

Displacement!
But bad lighting...

Calculating Normals For Lighting

For correct lighting, we must re-calculate the normals in vertex shader and pass the data to pixel shader for final lighting.

Get normal information for input image using Normal Map TOP
Normal Map TOP settings
Heightmap in Alpha Channel = On

Now sample alpha channel in vertex shader to get height map

Update Vertex Data Structure

From vertex shader, pass tbn matrix and texture coordinates to pixel shader
Vertex Shader

Pixel Shader

Vertex Shader - Calculate tbn and texCoord0

In the vertex shader
Declare tangents T

Calculate tbn matrix (tangent, bitangent, normal matrix) and set texCoord0 texture coordinates

TDCreateTBNMatrix() is new in TouchDesigner (Build 2017.15400+)

Pixel Shader - Normal Recalculation

In the pixel shader
Declare sampler2D dispTexture

Do normal recalculation and comment old normal code

GLSL Displacement should be working!
Try more points in Grid SOP (Rows/Columns = 500 or 1000 depending on your computer)

Extras in GLSL_Displacement.toe

Here are some extras to make it more interesting...

1. In TOP to input any image or movie for displacement
2. Fit TOP so that TOP pixels = Grid SOP points
3. dispScale uniform in shader controls amount of displacement
4. Particle render mode from Convert SOP
5. Custom parameters on component for easier control
6. Use /container_glsl_displacement viewer to move camera (arcBall camera component from Palette)
7. Pretty lighting

Summary

Using this method you can displace many more points and higher density geometry than with CHOPs and SOPs.

I taught this example in our TouchDesigner 202 Intro to GLSL workshop this fall, but the math is easier now because we added TDCreateTBNMatrix() to TouchDesigner.

Original inspiration for this displacement example came from Matthew Ragan and his Make_Some_Noise workshop earlier this year. Thank you again Matthew!

ごめん、日本語のカレンダーなのに、日本語で教えられない！はは！