[SA]Basic guide to the ENBSeries effect.txt shader

[Marty McFly]

Ok, so I guess I'll start straight away. I'll try to be as precise as possible. It's my personal understanding, hopefully they are all correct or at least not misleading. Also I'm a bad explanator, excuse this, I'm no native english speaker also, I'll try to make as less mistakes as possible.

 

 

What I'll explain here is the structure of the code, develop an own short exposure shader (with basic explanation of the language HLSL). Also I'll upload the finished file AND a basic file where main structure is there but the pixel shader does nothing by default, good to get started.

 

 

Introductioneffect.txt is a shader plugin which is loaded by ENBSeries. Nothing special has to be done to make ENB loading it, placing the file into same folder like d3d9.dll and enbseries.ini etc will do the job. This little (or big file) is really powerful. Think of it as a customizeable slot that Boris Vorontsov left for us to develop a plugin. It is the almost last stage where post processing happens, after all normal ENB effects like shadows, water, bumpmapping etc. The file is written in HLSL, that's the shader language of DirectX, very similiar to C.

 

There has been a mess about multipass/singlepass, I'll explain later what that is.

 

 

 

StructureFirst of all, there has to be input given. How? Through so called "textures", effect.txt for SA has access to those:

 

 

texture2D texColor; (=color input)texture2D texDepth;(=depth texture, used to get distance from camera n such for effects like DOF or SSAO)texture2D texNoise; (=some kind of random texture for image grain used)

 

 

 

These have to be sampled with so called "samplers", like here:

 

 

sampler2D InputSampler = sampler_state{   Texture = (texColor); (here you see the texture)   MinFilter = POINT;   MagFilter = POINT;   MipFilter = POINT;   AddressU   = Clamp;AddressV   = Clamp;SRGBTexture=FALSE;MaxMipLevel=0;MipMapLodBias=0;};

 

 

For now on, when referring to color, the "InputSampler" is used. You can name it whatever you want, "Iamasampler123" would also work. But better let it be like it is.

 

 

Now comes the struct, keep your fingers away from it, it has to be like this:

 

 

struct VS_OUTPUT_POST {float4 vpos  : POSITION;float2 txcoord : TEXCOORD0;};struct VS_INPUT_POST {float3 pos  : POSITION;float2 txcoord : TEXCOORD0;};

 

 

 

A following important thing is the Vertex Shader. It can manipulate image with some geometry info, rather than treat the image as a flat paint. I am not good at vertex shader, besides it's rarely used in this stage of post processing. I'll just skip it.

 

 

 

 

VS_OUTPUT_POST VS_PostProcess(VS_INPUT_POST IN){VS_OUTPUT_POST OUT;float4 pos=float4(IN.pos.x,IN.pos.y,IN.pos.z,1.0);OUT.vpos=pos;OUT.txcoord.xy=IN.txcoord.xy;return OUT;}

 

 

 

Then comes the pixel shader, important! Here is everything where 99% of the magic happens and what I modify to create new effects. Later more, so here no explanation code.

 

 

At the very end, the shader(s) are wrapped in a technique, it is called "technique" or "compiler" but it's no actual one, it links vertex with pixel shader and some interpretations the ENB binary (the d3d9.dll) needs.

 

 

technique PostProcess{   pass P0   {VertexShader = compile vs_3_0 VS_PostProcess(); (=vertex shader title, here "VS_PP")PixelShader  = compile ps_3_0 main(); (pixel shader title, here "main")ZEnable=FALSE;CullMode=NONE;ALPHATESTENABLE=FALSE;SEPARATEALPHABLENDENABLE=FALSE;AlphaBlendEnable=FALSE;FogEnable=FALSE;SRGBWRITEENABLE=FALSE;}}

 

 

 

 

The code itself, basic explanationsTo create own shaders, I'll explain some data types, operations, syntax n stuff. I can't cover everything, that'd be too much.

 

DATA TYPES

• float

  This is the data type which is used most. Its can be every rational number, also with commas such as 3.0, 255.43453, -1000.4 or such.

   

• int

  Mostly used for stuff like passes, it can only be full numbers like 2, 4, 100, -34464 or 99999 or such. Used for counts, if something does the same thing all over again and again, you can tell the shader "do this X times". Half times or 1/3 or whatever won't work.

   

• bool

  Also often used, can only be 1 or 0, true or false. You already know this thing! EnableShadows or EnableWater, AlternativeDepth or UseMRTRendering from enbseries.ini are booleans. Can only be enabled or true (1) or disabled/false (0).

HLSL has some specific stuff like float2, float3 and float4. It basically means that a variable which is a float2 has not one number but 2. To get X and Y coordinates I can create 2 new variables, one for X and one for Y coordinate OR I can create only one variable from type float2 whose one component is X coordinate and second is Y coordinate. In effect.txt, for example the stuff around COLOR is a float4 whose 4 components represent Red, Green and Blue channels and the fourth is alpha channel which means transparency. To access the first value of a float4 named asdf, we use this: asdf.x, for the others y z and w.

 

To set up a new variable, there are several ways. I'll create a variable named "gtaforums" from data type float.

 

float gtaforums = 5.0;

 

 

as you can see, the line ends with ";". Most shader lines end with that, it determines that a new line begins. What this above does is creating the new variable, say it is a float and give the value, here 5.0. I can also write:

 

 

float gtaforums; (which firstly doesn't give it any value)gtaforums = 5.0; (which now uses the given variable and sets it to 5)

 

 

What doesn't work is:

 

 

gtaforums = 5.0;float gtaforums;  (wrong order since shaders are read from top to bottom)orfloat gtaforums; float gtaforums = 5.0; (gtaforums is already set as variable, doing this again isn't valid.

 

 

Now we modify that value! Let's multiplicate it with 2.5:

 

 

 

gtaforums = gtaforums *2.5; (now gtaforums doesn't have the value 5 but 12.5)orgtaforums *= 2.5; (combining = and math operator works for +-/*)OR we insert a new variable:float multiplicator = 2.5; (since we have 2.5, bool or int don't work)gtaforums *= multiplicator;

 

 

 

ADD COMMENTS

 

 

To exclude a line, just write "//" in front of it. Anything after taht will be ignored by the shader. To exclude a block,, use "/*" and "*/" like here:

 

 

enabledenabled//disableddisableddisabledand/*disableddisableddisableddisableddisableddisableddisableddisableddisableddisableddisableddisabled*/

 

 

 

-----------------------

 

I think this will do, the rest does google

 

 

 

Pixel ShaderSooo finally, we begin with actual shader writing. Go grab this code:

 

http://pastebin.com/aVzWqRcJ

 

It's a raw effect.txt with every struct, vertex shader, technique n all and a pixel shader but this pixel shader doesn't modify anything by default.

Look after "START HERE WITH YOUR OWN MAGIC" there you can begin.

 

 

As you can see, I gave you a variable named color, since we want to modify the color of the screen. It's linked the sampled texColor (InputSampler) and the pixel coordinates (IN.txcoord) so it represents in RGBA the color of every pixel. Let's multiply it:

 

 

float4 color = tex2D(InputSampler, IN.txcoord); color *= 2;color.w=1; //sets transparency to offreturn color; //returns the values to the screen. if you return 0 for example, you have blackscreen.

 

 

Result is 2 times brighter screen! What you or we did is a simple exposure control shader. Let's add a control value, we call it "Exposure":

 

 

 

float4 color = tex2D(InputSampler, IN.txcoord); float Exposure = 2.0;color *= Exposure;color.w=1;return color;

 

 

Now we don't want to make ALL channels brighter, only RED! Muahaha. Red is represented by the first value of the float4:

 

 

 

float4 color = tex2D(InputSampler, IN.txcoord); float Exposure = 1.0;float RedExposure = 2.0; //control for redcolor *= Exposure;color.x *= RedExposure; //multiplies x value of color, here red.color.w=1;return color;

 

 

Results in red tinted screen. We want to create a full control for all channels:

 

 

float4 color = tex2D(InputSampler, IN.txcoord); float Exposure = 1.0;float RedExposure = 2.0; //control for redfloat GreenExposure = 2.0; //control for greenfloat BlueExposure = 2.0;//control for bluecolor *= Exposure;color.x *= RedExposure; //multiplies x value of color, here red.color.y *= RedExposure; //multiplies y value of color, here green.color.z *= RedExposure; //multiplies z value of color, here blue.color.w=1;return color;

 

 

Now we have a shader that controls general exposure/brightness and for every channel. With own control values! So we can create a blue tint or a violet tint or whatever!

 

 

 

 

This is the basic for a short shader. With this knowledge, next time I'll take a finished shader, tear it apart and explain what each thing does.

Edited July 30, 2013 by Marty McFly

[Marty McFly]

Preserved

[Fantomax]

whoo.. good TUT. going to learn 'bout this

[PacketOverload_x64bit]

Cool Marty, will give it a good read when I get home from work. Also, I heard if you decompile the d3d9 the ENB d3d9.dll can actually read another enbseries.ini file. Do you know how to decompile it? Mr LaGlace used that process to analyze the min and max of the d3d9.dll.

 

-P

[Marty McFly]

Cool Marty, will give it a good read when I get home from work. Also, I heard if you decompile the d3d9 the ENB d3d9.dll can actually read another enbseries.ini file. Do you know how to decompile it? Mr LaGlace used that process to analyze the min and max of the d3d9.dll.

 

-P

Nah, this was something different, we used this structure because we had no multicast on old enb versions:

 

D3d9.dll loads enbseries.ini, effect.txt and via proxy d3d2.dll

 

D3d2 was same enb dll hex edited to load other file names so:

 

D3d2.dll loads enbseriesA.ini, effect2.txt and via proxy d3d3.dll

D3d3.dll loads enbseriesB.ini, effect3.txt and via proxy d3d4.dll

 

Result was multiple times loaded enb's (always 0.075) so we could iron out the lack of multipass support with multi effect.txts with one pass each.

 

Since 0.076 is out, we don't need this any more because all shaders can be combined.

 

Also, the newer enb versions are encrypted so hex editing doesn't work anymore. I was able to find the strings for the file names with IDA Pro but I saw no sense modifying them. But I think icelaglace used the same program to find the min max values, although his dlls aren't hex edited to load iceconfig instead of enbseries.ini because also the enbseries logo is missing at startup of gta iv which can't be achieved with hex editing.

[Michael_Xiong]

I had translated your post into Chinese.So,can I post it on www.gtabbs.com？

For sure I will mention your name and indicate “Copyright : Marty McFly”

[Michael_Xiong]

Oh...I have a question.

"*" means "Multiply"，but in this line，you said it means "plus"?

 

 

gtaforums = gtaforums *2.5; (now gtaforums doesn't have the value 5 but 7.5)

 

 

5*2.5=7.5

Is it a mistake？

[Marty McFly]

 

Oh...I have a question.

"*" means "Multiply"，but in this line，you said it means "plus"?

 

 

gtaforums = gtaforums *2.5; (now gtaforums doesn't have the value 5 but 7.5)

 

 

5*2.5=7.5

Is it a mistake？

Yes, this is a mistake xD, fixed now. And sure, you can post it, I'd like to see the finished post since I visit gtabbs frequently.

Edited July 30, 2013 by Marty McFly

[Michael_Xiong]

Oh...I have a question.

"*" means "Multiply"，but in this line，you said it means "plus"?

 

 

gtaforums = gtaforums *2.5; (now gtaforums doesn't have the value 5 but 7.5)

 

 

5*2.5=7.5

Is it a mistake？

Yes, this is a mistake xD, fixed now. And sure, you can post it, I'd like to see the finished post since I visit gtabbs frequently.

Thanks！

http://www.gtabbs.com/read-gta-tid-2903189.html

↑↑↑link of my post.

[Michael_Xiong]

And when will you release the next part of this tutorial？

I can not wait

[Marty McFly]

And when will you release the next part of this tutorial？

I can not wait

I'm working on it, I'm learning new stuff atm and relearn old stuff to be as precise as possible.

[FMHn]

Hello marty! Sorry for a Out Off Topic reply, but possibly to create shader (i mean HLSL) on rendermonkey?

but, how to import that shader to GTA sa? I don't know how that shader work, especialy SSAO shader,

realy interesting to learn how create shader!

 

i realy like if you update this thread and more significant, btw thanks for guide!

 

realy goodwork man keep it up!

(sorry didn't reply your message now, back to collage)

[J16D]

nice! looking for a simple tutorial of hlsl shaders finally I found this one

could you be kind enough to explain, how to create shaders for this plugin;

http://gtaforums.com/topic/703763-relcleo-shader-api/

and I'm wondering, it is possible create shader for water in this way?

[RitongaA1991]

OKay Thanks Marty

Edited July 21, 2018 by RitongaA1991

[Dmitrysass]

Great tutorial on end.

[flippaaa111]

I love you marty your a god! Hlsl is amazing and without enb we would be lost! God bless you your the best!