Custom shaders

A custom shader is a surface that looks the way you write it: cel shading, a hologram, glowing lava, stylised water. You write it in HLSL, in one .hlsl file, and materials use it as they use the engine’s own shaders. It comes in two kinds:

• A surface shader says what the surface is: its colour, shininess, bumps and glow, and if you like, how its vertices move. The engine does the rest: every light, shadow and reflection probe, shadows cast by the moved vertices, picking in the editor. This is the kind to start with.
• A low-level shader is the whole program, vertex and pixel, for a shading of your own (a cel shader that bands its light, say). You keep the engine’s pass contract yourself, with its library’s help.

Making one

Create > Materials has three: Lit Surface Shader (glowing lava cracks, lit by the scene), Unlit Surface Shader (a colour and a texture, unaffected by light) and Low-Level Shader. Each makes a shader in the Shaders folder, with its .hlsl file in the project’s Sources/, and opens it. You can also drop an .hlsl file into the Assets panel; the shader is named after the file.

The shader page

On the left, the code. A moment after you stop typing, the shader is checked: errors are marked on their lines, listed under the code, and counted in the status line. On the right, a sphere drawn with the shader as it stands, at its parameters’ defaults. Below it:

• Name: what materials find the shader by. It cannot be the name of one of the engine’s own shaders (forward, unlit, and the rest).
• Vertex Entry Point and Fragment Entry Point: a low-level shader’s functions each stage starts in, VSMain and PSMain at first. A surface shader’s are the engine’s.

The status line says which kind the shader is. The check as you type compiles it plainly; Save compiles every version of it (skinned, instanced, masked, a surface shader’s shadow and pick passes) and reports what fails. A shader that fails keeps its last good version in your scenes until it compiles again.

Parameters: the properties block

A shader declares the parameters its materials set in a properties block, one per line: a type, a name, an optional default and, for a number, an optional [min, max] range, which the material page shows as a slider.

properties
{
    Color    BaseColor = (1, 0.4, 0.1, 1);
    ColorHdr Glow      = (1, 0.3, 0, 4);     // a colour with an intensity
    float    Speed     = 0.2 [0, 2];
    int      Steps     = 3   [1, 8];
    float2   Tiling    = (1, 1);
    Texture  Cracks;
    Sampler  MainSampler;
}

Type
float, float2, float3, float4, int	Numbers.
Color	A colour, picked as you see it; the shader reads it linear. White unless given.
ColorHdr	A colour and an intensity (the fourth value), for glowing things. Black unless given.
Texture	A 2D texture.
TextureCube	A cube map.
Sampler	How the textures are sampled. Declare one to sample with.

The engine turns the block into the HLSL declarations, so the shader uses each parameter by its name, and a material’s settings are exactly these. Lines after the block keep their numbers, so errors point at the line you wrote.

Surface shaders

A surface shader starts with a line saying how it is lit, then its parameters, then Surface():

// surface: lit
properties
{
    Color   CrackColor = (1.0, 0.35, 0.05, 1.0);
    float   Speed      = 0.2 [0, 2];
    float   Glow       = 4.0 [0, 20];
    Texture Cracks;
    Sampler MainSampler;
}

void Surface(SurfaceInput input, inout SurfaceOutput surface)
{
    float2 uv = input.uv + float2(0.0, Time() * Speed);
    float crack = Cracks.Sample(MainSampler, uv).r;
    surface.albedo = lerp(float3(0.08, 0.05, 0.04), CrackColor.rgb, crack);
    surface.roughness = 0.9;
    surface.emissive = CrackColor.rgb * crack * Glow;
}

// surface: lit is shaded with every light, shadow and reflection probe; // surface: unlit shows its colour as it is.

What Surface() is given (SurfaceInput), in world space: worldPos, normal, tangent, uv, color (the vertex colour times the mesh’s tint), viewDir (toward the camera) and screenPos (in pixels).

What it decides (SurfaceOutput), starting as a plain white surface:

Field		Starts at
albedo	its colour	white
alpha	opacity; a Masked material cuts out below its cutoff	1
normal	a bump, in tangent space (a normal map’s value)	(0, 0, 1), flat
metallic	0 for most things, 1 for bare metal	0
roughness	0 is a mirror, 1 matte	0.5
occlusion	darkening in creases, 1 = none	1
emissive	light it gives off, unlit; above 1 it glows	black

Moving the vertices. Add Vertex() to move them, in world space, before they are drawn:

void Vertex(inout VertexInput v)
{
    v.position.y += sin(v.position.x * 2.0 + Time()) * 0.1;   // waves
}

VertexInput has position, normal and tangent, which you may change, and localPosition (the mesh’s own, before it is placed), uv and color. The shadow, the depth and the editor’s picking all move with it.

Time() is the scene’s clock in seconds, in both functions.

Cut-outs. On a Masked material, the surface’s alpha cuts it out, its shadow too: below the material’s AlphaCutoff when the shader declares one, else below a half.

What a surface shader cannot use. It reads the camera and the clock through its input and Time(), not the frame’s own constants, since each of its passes has different ones. A parameter cannot take a name the engine’s shaders already use (Tint, for one): the shader page says so at the parameter’s line.

Low-level shaders

A low-level shader draws in the engine’s forward pass, so it keeps its contract: it takes the mesh’s vertex in, and its pixel stage writes what the pass expects. The engine’s shader library does most of that for you, and the Low-Level Shader starter shows how to use it:

// variants(vertex): SKINNED INSTANCED
// variants(fragment): ALPHA_TEST GBUFFER DITHER

#define FORWARD_VS_ENTRY VSMain
#include "forward_vs.hlsli"     // the whole vertex stage, as VSMain
#include "lighting.hlsli"       // the frame's lights, shadows and image-based light
#include "forward_ps.hlsli"     // what the pixel stage writes

• The vertex stage. forward_vs.hlsli is the engine’s own: skinning, instancing and wind, writing a VSOutput (world position, normal, UV, vertex colour) the pixel stage reads.

• Variants. The variants lines say which versions of each stage to compile. A skinned mesh uses SKINNED, an instanced one INSTANCED; ALPHA_TEST is a masked material, DITHER a fading mesh, and GBUFFER the opaque pass, which writes the extra targets the post-processing reads. Test them with #ifdef. A flag the shader tests but does not declare is an error.

• Lighting the engine’s way. DirectLight adds up every light reaching a pixel, shaded and shadowed; AmbientLight adds the sky’s light, reflection probes and the scene’s ambient.

• Lighting your own way. Walk the lights yourself, as a toon shader does:

  LightList lights = LightsAt(input.clip.xy, viewDepth);
  for (uint i = 0; i < lights.count; ++i)
  {
      GpuLight L = LightAt(lights, i);
      float atten;
      float3 toLight = LightDirection(L, input.worldPos, atten);
      float shadow = ShadowOf(L, input.worldPos, N, viewDepth);
      // ... your response to it
  }

• What the pixel stage writes. Under GBUFFER, a GBufferOutput: the colour, the normal (OctEncode), the motion vector (MotionVector), roughness and metallic, and the albedo; otherwise just the colour. Under DITHER, call ScreenDoorFade first.

#include reaches the engine’s shader library; your project’s own include files are not supported yet.

Materials on it

On a material’s page, the Shader row picks the shader: drag one onto it or pick it. The material’s settings become the shader’s parameters, keeping every value whose name and type still match. Clear the row to go back to a built-in, PBR or Unlit. When you change the shader’s parameters and save it, its materials follow.

In an exported game

Exporting compiles every shader the game uses for its platform, so an exported game, the web included, needs no shader compiler. The .hlsl source is left out; the export preset’s Include shader source keeps it.

Learning by example

The Shader Gallery sample (Data/SampleProjects/ShaderGallery) is a course in custom shaders: a hall of plinths, one shader on each, with a list that flies the camera to each lesson. Every .hlsl is a short lesson that builds on the one before and comments only what is new in it, so read them in order, or open one alone and copy it. Open the project in the editor to edit a lesson on its shader page, its preview beside it, or to drag its material’s sliders.

#	Lesson	Kind	What it teaches
1	Unlit Colour	unlit surface	The smallest shader: a properties block and Surface()
2	Textured	lit surface	Sampling a texture, the uv and the vertex colour, lit by the scene
3	Lava	lit surface	Time() to make it move, and emission to make it glow
4	Hologram	unlit surface	viewDir for a fresnel rim, screenPos for scanlines, alpha on a blended material
5	Triplanar Rock	lit surface	Mapping by worldPos with no uv, and a normal map turned into tangent space
6	Dissolve	lit surface	Alpha on a masked material cutting holes, its shadow cut too, with a burning edge
7	Waves	lit surface	Vertex() moving the vertices, with the normal from the slope
8	Wind	lit surface	Vertex() by localPosition: a flag still at the pole, flapping at the end
9	Built-in PBR	lit surface	The engine’s own material written as a surface shader: the one to start from
10	Toon	low-level	Lighting of its own: light in bands, a shadow colour and a rim
11	Custom Lighting	low-level	Walking the lights with LightsAt, LightAt and ShadowOf: half-Lambert and Blinn-Phong