How to transfer Shaders from Shadertoy to ThreeJS (deutsche Version).
Spil a part of our research at the Publicis Pixelpark Innovationlab wij studied how wij could use lower level languages for web technologies. The visible choice seemed to be asm.js or WebAssembly.
But you could also use WebGL Shaders to solve machine-oriented problems. Shaders are programmed ter a language that is similar to C/C++, and while they are not mainly meant for solving generic problems, they can be used for more things than just rendering photos and 3D scenes.A 2nd motivation stems from the asthetics that can be achieved with shaders. Te 2002, a group of students from the University of Wisconsin-Madison published NPRQuake (“Non-PhotoRealistic Rendering Quake”), a variation of the well-known spel Quake by injecting code into the rendering pipeline.
The aesthetic quality of this variation wasgoed stunning, wij instantaneously understood that thesis kinds of effects could be a game-changer ter projects. While ter 2002 this variation wasgoed only possible by writing drivers for OpenGL, can now ter 2018 be achieved by shaders — even te web browsers.
So when wij recently were involved ter an kunst project, wij determined to give shaders a go.
Availability of shader code
If you are not indeed used to program shaders the visible choice is to search for loosely available examples and use thesis (with a careful look at the licences involved). One library that stands out te this regard is Shadertoy with ShaderFrog spil another example.
Since wij successfully worked with ThreeJS before wij determined to publish our findings with postprocessing shaders from Shadertoy te ThreeJS.
Shader te ThreeJS
ThreeJS can be used to utilize postprocessing shaders which alter the entire rendered photo and also material shaders, that can alter the material of 3D objects. Both types need a vertex and a fragment shader part, a vertex shader can switch the position of vertices te 3D while a fragment shader usually substitutes the color of a rendered pic.
This photo shows the four possible variations.
Types of shaders
Ter the upper left a postprocessing shader adds a color gradient to the rendered pic. To the right of it, a vertex shader reduces the render area. The two bottom photos demonstrate material shaders, the left one only alters the color while the right one switches the position of the vertices. Since shaders are always composed of both vertex and fragment parts, the last example also switches the color.
Trivial example from Shadertoy
Wij studied how to transfer a shader from Shadertoy to ThreeJS back ter 2014 with very first results published on StackOverflow. Wij found the following pattern useful:
By following this patters you can transfer a elementary shader to ThreeJS.
Trivial ShaderNon-trivial example from Shadertoy
With a more ingewikkeld shader you need to do much more, spil wij will outline now. For a non-trivial example wij chose Noise Omlijning by candycat since you run into a duo of problems with it. You can find it here: https://www.shadertoy.com/view/MscSzf
This example also creates a entire toneel ter shader language. But ter ThreeJS you usually want control overheen the 3D objects, so wij determined to create the toneel ter ThreeJS while still utilizing the shaders to alter it.
Understanding the structure of shaders
Wij embark by attempting to get a capture of the structure of the shader, this can be achieved with Shadertoy’s editor. Since edits to the code can be seen ter realtime wij can make puny switches to understand how it works.
Below the actual code wij see that this code is based on a channel called iChannel0, with the B indicating a buffer.
To see this buffer ter activity wij comment out line 37 and add this:
The result should be:
This ordinary switch results ter showcasing the color of the previous buffer and not the result of this buffer.
By examining the previous buffer — Buf B — wij see that this one also uses iChannel0, so wij are still not looking at the original toneel creating code.
Utilizing the same trick spil before, wij comment out line 29 and add a line that calculates uv and the actual color like so:
This should leave us with:
This looks very much more like a regular toneel. Also, Buf A doesn’t use another buffer, so wij are looking at the original toneel creating code.
Reconstruction te ThreeJS
Utter disclaimer here: what goes after is by no chance ‘optimal’ code, but just one way to solve the problem ter the most straight forward way.
Creating the toneel
Wij commence by creating a somewhat simpler toneel with only a sphere and a plane. Additionally wij want to use the MeshNormalMaterial from ThreeJS.
A possible result is shown here:
Shader ter ThreeJS step 0
The code is contained te a HTML-file called index.html:
Wij need to take care of the dependencies to the ThreeJS library and wij also add our own code te index.js:
This concludes step zero of porting the toneel from Shadertoy.
Shader ter ThreeJS step 0Recreating the very first shader pass
Te the next step wij attempt to recreate the very first shader render step ter the buffer, this is basically copying the shader code to ThreeJS.
This should be the result:
Shadertoy withou the last pass
To achieve this wij used the EffectComposer for ThreeJS, which provides a ordinary way to use postprocessing shaders.
This creates an EffectComposer Example which adds a normal rendering pass and an extra shader pass. Wij copy the shader code te the variables VERTEX and FRAGMENT. The shader definition also defines a Uniform called tDiffuse used by the EffectComposer. It contains the pic from the previous rendering pass that will be altered ter the current pass.
With this fresh render step te activity, wij vertoning this pass rather than the original toneel. Thus wij need to add some code for resizing purposes, hence wij add:
Now wij need to define the constants VERTEX und FRAGMENT. Wij can’t use Shadertoy’s vertex shader, so wij need to define our own:
Wij do use the fragment shader from Shadertoy, however, and add it to FRAGMENT:
This basically creates the shader but wij still need to address the following problems:
- the vertex shader coordinate isn’t used te the fragment shader yet- the fragment shader uses texture which is unknown ter the current WebGL setting
- mainImage voorwaarde be renamed to main
- iResolution isn’t set yet.
So the shader isn’t working yet.
Addressing the very first problem results te this definition:
Now wij can use the vector vUv instead of fragCoord / iResolution.xy. This results ter:
Now wij simply substitute every occurence of texture with texture2D.
Additionally wij alter the mainImage to main without parameters:
main should also terugwedstrijd gl_FragColor instead of fragColor which defines the color ter the shader.
Lastly wij need to set iResolution by adding it to the uniforms. Wij do this by defining a ThreeJS vector storing width and height:
Now wij can add the resolution to the uniforms:
Wij need to enhance our resize function:
It is significant that wij use the uniforms of the actual render pass. The original one has bot deeply cloned by the EffectComposer, switching the variable resolution would have no effect.
Since wij did define two uniforms, wij need to introduce them to our fragment shaders so wij define them:
This concludes this shader pass and if all went well wij see this:
Shader Pass 1 without shadows
From the blue lines wij see that it generally works but the pink part is still missing. Let’s switch that.
Solve the problem with shadows
The pink part is missing since the shader ter Shadertoy secretly renders shadows to an alpha channel that wasn’t visible te the very first place spil wij can see ter the next picture:
Shadows ter Shadertoy
There are several ways to solve this — wij used the straight forward one by adding a material that holds the shadows. Thesis voorwaarde be treated te an extra render pass.
So let’s create shadows ter ThreeJS:
Shadows need light, ter this case a directional one:
A MeshPhongMaterial can hold shadows.
While a fresh render target saves them.
And again, a resize function is needed:
Now wij can transfer the shadows to the fresh render target and prepare it for the shader:
Thesis lines set the material, render the toneel, set the shadow to a uniform and switch the material back to MeshNormalMaterial.
Now the shader needs to know about the shadows to be able to process them, so wij switch the uniforms:
Same for the fragment shader:
Then wij substitute the former line with our shadow.
The result should look like the the 2nd step on Shadertoy.
Now wij only miss the 2nd shader pass to accomplish this.
The final shader pass
For the final shader pass wij add another EffectComposer example.
Let’s define another shader:
Wij deactivate renderToScreen for the previous render pass:
Again, more variables are introduced, iTime to switch variables overheen time and iChannel1 to add noise.
Shadertoy Noise and iTime
Wij use a ThreeJS clock for iTime.
With every switch wij also update iTime:
Wij add iTime und noise to the uniforms:
The noise is simply a noisy texture (for example the one from Shadertoy) that wij flow with ThreeJS into tNoise.
Now wij need to adapt the fragment shader to our fresh variables, so wij apply the following measures:
After thesis switches the shader still won’t compile, because this shader needs a specific WebGL extension. Gratefully, this is effortless to add te ThreeJS:
This gives us the following result:
Which is very close to the original Shadertoy:
Wij successfully transferred a ingewikkeld Shadertoy shader to ThreeJS by following thesis steps:
- understand the structure of the specific shader
- implement shader passes
- address possible GLSL incompatibilities
- create optional shader passes and/or materials
- activate optional extensions
Wij expect that thesis challenges will be mitigated with the upcoming WebGL2 support ter ThreeJS since possible GLSL incompatibilities should vanish.
The total source code is here.
The final result
Helpful linksaf and resources
- Shadertoy Shader: https://www.shadertoy.com/view/MscSzf
- WebGL2Fundamentals: https://webgl2fundamentals.org
- ThreeJS: https://threejs.org/](https://threejs.org/
- ThreeJS Examples: https://github.com/mrdoob/three.js/tree/dev/examples
- WebGL-Extensions: https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/getExtension and https://github.com/mrdoob/three.js/issues/7379
- EffectComposer: https://github.com/hughsk/three-effectcomposer
- Another EffectComposer treatment: https://github.com/spite/Wagner
- WebGL Quick Reference Card: https://www.khronos.org/files/webgl/webgl-reference-card-1_0.pdf
Converting Shaders from Shadertoy to ThreeJS wasgoed originally published te Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.