texelFetch sampling function does not work correctly.

[crashGG]

A simple example:
#version 450

layout(std140, set = 0, binding = 0) uniform UBO
{
mat4 MVP;
vec4 SourceSize;
} global;

#pragma stage vertex
layout(location = 0) in vec4 Position;
layout(location = 1) in vec2 TexCoord;
layout(location = 0) out vec2 vTexCoord;

void main()
{
gl_Position = global.MVP * Position;
vTexCoord = TexCoord;
}

#pragma stage fragment
layout(location = 0) in vec2 vTexCoord;
layout(location = 0) out vec4 FragColor;
layout(set = 0, binding = 1) uniform usampler2D Source;

void main()
{
// 1. Calculate integer texture coordinates (to avoid floating-point precision errors)
ivec2 texCoord = ivec2(vTexCoord.xy * global.SourceSize.xy);

// 2. Sample the four independent RGBA channels 
uvec4 rgbaUint = texelFetch(Source, texCoord, 0).rgba;

// 3. Convert integers in the range 0~255 to normalized floating-point numbers in the range 0.0~1.0  
vec4 rgba = rgbaUint / 255.0;

FragColor = rgba;

}
Expected result: The image is output as is. However, the brightness of each RGB channel is abnormally high.

[crashGG]

Follow-up: I tried

ivec2 texCoord = ivec2(vTexCoord.xy * global.SourceSize.xy);

uvec4 rawData = texelFetch(Source, texCoord, 0);

vec4 rgba = uintBitsToFloat(rawData);

FragColor = rgba;

and the image returned to normal. It seems that the shader incorrectly uses the usampler2D function and still uses the 32-bit floating point format. uintBitsToFloat() successfully reinterprets the floating point bit pattern into the correct color value. My guess: The application did not create a VK_FORMAT_R8G8B8A8_UINT format texture and upload integer data.

[hizzlekizzle]

Have you tried any other video drivers to confirm that it's across the board and not driver-specific?

[crashGG]

tried glcore,vulkan,dx11,dx12,the issue is the same.

[hizzlekizzle]

What does your first pass look like? I'm getting different but inconsistent behavior between d3d11 and glcore with 2 basic shaders.

Pass 1:

#version 450

#pragma format R8G8B8A8_UINT

layout(push_constant) uniform Push
{
	vec4 SourceSize;
	vec4 OriginalSize;
	vec4 OutputSize;
	uint FrameCount;
} params;

layout(std140, set = 0, binding = 0) uniform UBO
{
	mat4 MVP;
} global;

#pragma stage vertex
layout(location = 0) in vec4 Position;
layout(location = 1) in vec2 TexCoord;
layout(location = 0) out vec2 vTexCoord;

void main()
{
   gl_Position = global.MVP * Position;
   vTexCoord = TexCoord;
}

#pragma stage fragment
layout(location = 0) in vec2 vTexCoord;
layout(location = 0) out uvec4 FragColor;
layout(set = 0, binding = 2) uniform sampler2D Source;

void main()
{
   FragColor = uvec4(texture(Source, vTexCoord) * vec4(255.));
}

and Pass 2:

#version 450

layout(push_constant) uniform Push
{
	vec4 SourceSize;
	vec4 OriginalSize;
	vec4 OutputSize;
	uint FrameCount;
} params;

layout(std140, set = 0, binding = 0) uniform UBO
{
	mat4 MVP;
} global;

#pragma stage vertex
layout(location = 0) in vec4 Position;
layout(location = 1) in vec2 TexCoord;
layout(location = 0) out vec2 vTexCoord;

void main()
{
   gl_Position = global.MVP * Position;
   vTexCoord = TexCoord;
}

#pragma stage fragment
layout(location = 0) in vec2 vTexCoord;
layout(location = 0) out vec4 FragColor;
layout(set = 0, binding = 2) uniform usampler2D Source;

void main()
{
   uvec4 temp = texelFetch(Source, ivec2(vTexCoord * params.SourceSize.xy), 0);
   FragColor = vec4(temp / 255);
}

On glcore, I get a dark-but-visible image. On d3d11, it's pitch-black.

[crashGG]

I think ,in your 1st pass ,#pragma format R8G8B8A8_UINT is not a standard HLSL instruction, so the DX11 compiler selectively skips it.

[hizzlekizzle]

That all gets translated into native code by SPIR-V Cross. If I use texture() in the second pass with the d3d11 driver, the compiler complains about not being able to sample non-float textures in HLSL, so I know it's getting applied.

[crashGG]

After trying your code,the performance of dx11 is also different from other drivers.its pitch black.I figured there was a problem on the program side that couldn't handle the R8G8B8A8_UINT format correctly, so I also filed a bug on the program side.

[hizzlekizzle]

Looks like with a little clamping, everything works as expected with glcore, but d3d11 returns a black screen and d3d12 crashes:

pass1:

#version 450

#pragma format R8G8B8A8_UINT

layout(push_constant) uniform Push
{
	vec4 SourceSize;
	vec4 OriginalSize;
	vec4 OutputSize;
	uint FrameCount;
} params;

layout(std140, set = 0, binding = 0) uniform UBO
{
	mat4 MVP;
} global;

#pragma stage vertex
layout(location = 0) in vec4 Position;
layout(location = 1) in vec2 TexCoord;
layout(location = 0) out vec2 vTexCoord;

void main()
{
   gl_Position = global.MVP * Position;
   vTexCoord = TexCoord;
}

#pragma stage fragment
layout(location = 0) in vec2 vTexCoord;
layout(location = 0) out uvec4 FragColor;
layout(set = 0, binding = 2) uniform sampler2D Source;

#define conv(f) int(clamp(f * 255.0 + 0.5, 0.0, 255.0))

void main()
{
	vec4 img = texture(Source, vTexCoord);
	FragColor = uvec4(conv(img.r), conv(img.g), conv(img.b), conv(img.a));
}

pass 2:

#version 450

layout(push_constant) uniform Push
{
	vec4 SourceSize;
	vec4 OriginalSize;
	vec4 OutputSize;
	uint FrameCount;
} params;

layout(std140, set = 0, binding = 0) uniform UBO
{
	mat4 MVP;
} global;

#pragma stage vertex
layout(location = 0) in vec4 Position;
layout(location = 1) in vec2 TexCoord;
layout(location = 0) out vec2 vTexCoord;

void main()
{
   gl_Position = global.MVP * Position;
   vTexCoord = TexCoord;
}

#pragma stage fragment
layout(location = 0) in vec2 vTexCoord;
layout(location = 0) out vec4 FragColor;
layout(set = 0, binding = 2) uniform usampler2D Source;

#define uconv(f) (float(f) / 255.0)

void main()
{
   uvec4 temp = texelFetch(Source, ivec2(vTexCoord * params.SourceSize.xy), 0);
   FragColor = vec4(uconv(temp.r), uconv(temp.g), uconv(temp.b), uconv(temp.a));
}

With this, everything is the correct brightness.

[crashGG]

The problem lies here. Even though you've added the R8G8B8A8_UINT instruction, the resulting floating-point value is still normalized to 1. The reason we use usampler2D instead of sampler2D is to get a lossless integer value. This allows the shader to compare the RGBA values ​​of two pixels directly.

[crashGG]

The current retroarch cannot use many shader codes based on lossless integers. Instead, it has to use floating point numbers after a lossy conversion. This has to bear the risk of unexpected loss of precision.

[hizzlekizzle]

No, the "conv" macro is converting them to integers #define conv(f) int(clamp(f * 255.0 + 0.5, 0.0, 255.0))

and then the next pass is using the usampler2D to reference it.

[crashGG]

// Ideal case: absolute 0.0
vec4 img = vec4(0.0, 0.0, 0.0, 0.0);

// conv conversion:
0.0 * 255 + 0.5 = 0.5 → int(0.5) = 0 ✓
// Result: uvec4(0, 0, 0, 0) ✓

// Due to floating-point precision, it might be very small positive numbers
vec4 img = vec4(0.000000001, 0.000000002, 0.000000003, 0.000000004);

// conv conversion:
0.000000001 * 255 + 0.5 ≈ 0.500000255 → int(0.500000255) = 0 ✓
0.000000002 * 255 + 0.5 ≈ 0.500000510 → int(0.500000510) = 0 ✓
0.000000003 * 255 + 0.5 ≈ 0.500000765 → int(0.500000765) = 0 ✓
0.000000004 * 255 + 0.5 ≈ 0.500001020 → int(0.500001020) = 0 ✓

// Ideal case: absolute 1.0
vec4 img = vec4(1.0, 1.0, 1.0, 1.0);

// conv conversion:
1.0 * 255 + 0.5 = 255.5 → int(255.5) = 255 ✓
// Result: uvec4(255, 255, 255, 255) ✓

// Due to precision loss, it might be values slightly less than 1.0
vec4 img = vec4(0.999999996, 0.999999997, 0.999999998, 0.999999999);

// conv conversion:
0.999999996 * 255 + 0.5 ≈ 255.4999990 → int(255.4999990) = 255 ✓
0.999999997 * 255 + 0.5 ≈ 255.4999994 → int(255.4999994) = 255 ✓
0.999999998 * 255 + 0.5 ≈ 255.4999999 → int(255.4999999) = 255 ✓
0.999999999 * 255 + 0.5 ≈ 255.5000000 → int(255.5000000) = 255 ✓

Yes, I have verified the precision of the conv function, even in extreme cases at the 0 and 1 boundaries. However, if texelFetch could directly retrieve the original integer data, these algorithmic hardware overheads should not have occurred.

[hizzlekizzle]

I don't think that's possible, since RetroArch/libretro cores doesn't output int textures on its own. We have to convert the float values that it outputs to int, set the framebuffer format to int, then we can sample that int texture later in the chain.