Skip to content

Commit 3ba0e59

Browse files
spencer-lunargspencer-lunarg
committed
docs: Use SDK naming for sync docs
1 parent 736f4ed commit 3ba0e59

9 files changed

+1688
-1685
lines changed

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ See [documentation](./docs/khronos_validation_layer.md#configuring-the-validatio
2929
### Internals
3030

3131
The Validation Layers are broken up as many smaller internal "Validation Objects" each tasked with their own job.
32-
[Core Checks](./docs/core_checks.md) is what is enabled by default, but there are other validation objects that can be additionally enabled, such as [Synchronization Validation](./docs/syncval_design.md), [GPU Assisted Validation](./docs/gpu_validation.md), and [more](./docs/README.md).
32+
[Core Checks](./docs/core_checks.md) is what is enabled by default, but there are other validation objects that can be additionally enabled, such as [Synchronization Validation](./docs/synchronization_design.md), [GPU Assisted Validation](./docs/gpu_validation.md), and [more](./docs/README.md).
3333
There are also a few [limitations](./docs/limitations.md) stemming from the Validation layers being a Vulkan layer.
3434

3535
## Community Assistance

‎docs/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,9 @@ This directory contains more detailed information about how each part of the Val
1717
- [VK_KHR_portability_subset](./portability_validation.md)
1818
- [Shader Validation](./shader_validation.md)
1919
- [Stateless Parameter Validation](./stateless_validation.md)
20-
- [Synchronization Validation Design](./syncval_design.md)
21-
- [Synchronization Validation Usage](./syncval_usage.md)
22-
- [Synchronization Validation Development](./syncval_development.md)
20+
- [Synchronization Validation Design](./synchronization_design.md)
21+
- [Synchronization Validation Usage](./synchronization_usage.md)
22+
- [Synchronization Validation Development](./synchronization_development.md)
2323
- [Thread Safety Validation](./thread_safety.md)
2424
- [Limitations](./limitations.md)
2525
- [Creating Tests](./creating_tests.md)

‎docs/error_messages.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -181,4 +181,4 @@ adb setprop debug.vulkan.khronos_validation.log_filename=/data/local/tmp/vvl_err
181181

182182
# Additional information
183183

184-
Synchronization validation detects memory hazards and has custom error conventions. Additional information about SyncVal errors can be found in the [Synchronization Validation Messages](./syncval_usage.md#synchronization-validation-messages) document.
184+
Synchronization validation detects memory hazards and has custom error conventions. Additional information about SyncVal errors can be found in the [Synchronization Validation Messages](./synchronization_usage.md#synchronization-validation-messages) document.

‎docs/synchronization_design.md‎

Lines changed: 1429 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/synchronization_development.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Synchronization Validation Development
2+
3+
This document contains miscellaneous information related to development of synchronization validation.
4+
5+
## Error messages
6+
7+
### How to diff-compare error messages before and after the change
8+
9+
The syncval error reporting code uses helper routines to generate the parts of the error messages. One potential issues with this shared code is when modifying a specific error message it's possible to accidentally break another one. The easiest way to validate the scope of the changes is to compare the output of all negative tests before and after the change.
10+
11+
The following components of the error messages vary between the runs:
12+
* gtest timing
13+
* dispatchable handles
14+
15+
`--gtest_print_time=0` removes the timing information
16+
17+
Use [this patch](https://github.com/user-attachments/files/18318045/remove-dispatchable-handles.patch) to replace all dispatchable handles with zero values. It should be applied only locally to `layers/error_messages/logging.cpp`.
18+
19+
Run the negative tests before and after the change:
20+
```
21+
--gtest_filter=NegativeSyncVal*.* --print-vu --gtest_print_time=0 >syncval_{before/after}.txt
22+
```
23+
24+
Diff-compare output files.

‎docs/synchronization_usage.md‎

Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
<!-- markdownlint-disable MD041 -->
2+
<!-- Copyright 2015-2025 LunarG, Inc. -->
3+
[![Khronos Vulkan][1]][2]
4+
5+
[1]: https://vulkan.lunarg.com/img/Vulkan_100px_Dec16.png "https://www.khronos.org/vulkan/"
6+
[2]: https://www.khronos.org/vulkan/
7+
8+
# Synchronization Validation
9+
10+
Synchronization Validation is implemented in the `VK_LAYER_KHRONOS_validation layer`. When enabled, the Synchronization Object is intended to identify resource access conflicts due to missing or incorrect synchronization operations between actions (Draw, Copy, Dispatch, Blit) reading or writing the same regions of memory.
11+
12+
Synchronization will ideally be run periodically after resolving any outstanding validation checks from all other validation objects, so that issues may be addressed in early stages of development.
13+
14+
The specific areas covered by this layer are currently tracked in the
15+
[Synchronization Validation Project](https://github.com/KhronosGroup/Vulkan-ValidationLayers/projects/5).
16+
Requests for additional checks can be requested by creating a [Github issue](https://github.com/KhronosGroup/Vulkan-ValidationLayers/issues).
17+
18+
## Enabling Synchronization Validation
19+
20+
Synchronization Validation is just a [normal layer setting](https://github.com/KhronosGroup/Vulkan-ValidationLayers/blob/main/docs/settings.md) that can be turned on.
21+
22+
The main 3 ways to turn on Sync
23+
24+
1. We **highly** suggest people to use [VkConfig](https://www.lunarg.com/introducing-the-new-vulkan-configurator-vkconfig/) and use the Synchronization Preset.
25+
26+
> **NOTE** - This will turn off other non-sync validation for you makeing Synchronization Validation run faster.
27+
28+
2. Use `VK_EXT_layer_settings`
29+
30+
```c++
31+
// Will turn on as an additional setting with core validation
32+
const VkBool32 verbose_value = true;
33+
const VkLayerSettingEXT layer_setting = {"VK_LAYER_KHRONOS_validation", "validate_sync", VK_LAYER_SETTING_TYPE_BOOL32_EXT, 1, &verbose_value};
34+
VkLayerSettingsCreateInfoEXT layer_settings_create_info = {VK_STRUCTURE_TYPE_LAYER_SETTINGS_CREATE_INFO_EXT, nullptr, 1, &layer_setting};
35+
36+
VkInstanceCreateInfo instance_ci = GetYourCreateInfo();
37+
instance_ci.pNext = &layer_settings_create_info;
38+
```
39+
40+
3. Set as an environment variable (will turn on as an additional setting with core validation)
41+
42+
```bash
43+
# Windows
44+
set VK_LAYER_VALIDATE_SYNC=1
45+
46+
# Linux
47+
export VK_LAYER_VALIDATE_SYNC=1
48+
49+
# Android
50+
adb setprop debug.vulkan.khronos_validation.validate_sync=1
51+
```
52+
53+
[Additional configuration settings](https://vulkan.lunarg.com/doc/sdk/latest/windows/khronos_validation_layer.html) will all share the `VK_LAYER_SYNCVAL_`/`khronos_validation.syncval_*` prefix namespace.
54+
## Synchronization Validation Functionality
55+
56+
### Overview
57+
58+
The pipelined and multi-threaded nature of Vulkan makes it particularly important for applications to correctly insert needed synchronization primitives, and for validation to diagnose unprotected memory access hazards. Synchronization reports the presence of access hazards including information to identify the Vulkan operations which are in conflict. The reported hazards are:
59+
60+
61+
<table>
62+
<tr>
63+
<td>RAW
64+
</td>
65+
<td>Read-after-write
66+
</td>
67+
<td>Occurs when a subsequent operation uses the result of a previous operation without waiting for the result to be completed.
68+
</td>
69+
</tr>
70+
<tr>
71+
<td>WAR
72+
</td>
73+
<td>Write-after-read
74+
</td>
75+
<td>Occurs when a subsequent operation overwrites a memory location read by a previous operation before that operation is complete (requires only execution dependency).
76+
</td>
77+
</tr>
78+
<tr>
79+
<td>WAW
80+
</td>
81+
<td>Write-after-write
82+
</td>
83+
<td>Occurs when a subsequent operation writes to the same set of memory locations (in whole or in part) being written by a previous operation.
84+
</td>
85+
</tr>
86+
<tr>
87+
<td>WRW
88+
</td>
89+
<td>Write-racing-write
90+
</td>
91+
<td>Occurs when unsynchronized subpasses/queues perform writes to the same set of memory locations.
92+
</td>
93+
</tr>
94+
<tr>
95+
<td>RRW
96+
</td>
97+
<td>Read-racing-write
98+
</td>
99+
<td>Occurs when unsynchronized subpasses/queues perform read and write operations on the same set of memory locations.
100+
</td>
101+
</tr>
102+
</table>
103+
104+
105+
106+
### Current Feature set
107+
108+
- Hazard detection for memory usage for commands within the *same* command buffer.
109+
- Synchronization operations .
110+
- vkCmdPipelineBarrier.
111+
- vkCmdSetEvent/vkCmdWaitEvents/vkCmdResetEvent.
112+
- renderpass/subpass barriers.
113+
- The `VK_KHR_synchronization2` extension
114+
- vkCmdPipelineBarrier2KHR
115+
- vkCmdSetEvent2KHR/vkCmdWaitEvents2KHR/vkCmdResetEvent2KHR.
116+
- Image layout transition hazard and access tracking.
117+
- Load/Store/Resolve operations within Subpasses.
118+
- ExecuteCommands detection of hazard from or with secondary command buffers
119+
120+
- QueueSubmit/QueueSubmit2 time hazard detection
121+
- Semaphore and Fence synchronization operations/effects
122+
- Device and Queue WaitIdle support
123+
- Dynamic Rendering support
124+
125+
### Known Limitations
126+
- Does not support precise tracking of descriptors accessed by the shader (requires integration with GPU-AV)
127+
- Hazards related to memory aliasing are not detected properly
128+
- Indirectly accessed (indirect/indexed) buffers validated at *binding* granularity. (Every valid location assumed to be accessed.)
129+
- Queue family ownership transfer not supported
130+
- Host set event not supported.
131+
- No dedicated support for sparse resources. Need to investigate which kind of support is needed.
132+
- Host memory accesses are not tracked. Corresponding race conditions are not reported.
133+
- Does not include implementation of multi-view renderpass support.
134+
- Memory access checks not suppressed for VK_CULL_MODE_FRONT_AND_BACK.
135+
- Does not include component granularity access tracking, or correctly support swizzling.
136+
137+
## Typical Synchronization Validation Usage
138+
139+
### Debugging Synchronization Validation Issues
140+
141+
To debug synchronization validation issues (all platforms):
142+
143+
- Create a debug callback with `vkCreateDebugUtilsMessengerEXT` with the `VK_DEBUG_REPORT_ERROR_BIT_EXT` set.
144+
- Enable synchronization as noted above. On Linux and Windows this can be simplified by enabling Synchronization Validation using [Vulkan Configurator (vkconfig)](https://vulkan.lunarg.com/doc/sdk/latest/windows/vkconfig.html).
145+
- Set a breakpoint in the debug callback and run your application in the debugger.
146+
- The callback will be called when a `vkCmd`... command with a hazard is recorded.
147+
148+
On Windows, Synchronization Validation can be run using just vkconfig and the debugger without defining a callback:
149+
150+
* In vkconfig.
151+
* Enable Synchronization Validation.
152+
* Select 'Debug Actions' 'Break' and 'Debug Output'.
153+
* Debug application in Visual Studio.
154+
* Hazard messages will appear in the debugger output window and the debugger will break (in the validation layer code) when a `vkCmd`... command with a hazard is recorded.
155+
156+
157+
### Synchronization Validation Messages
158+
159+
A synchronization validation error message describes a race condition by identifying two memory accesses that caused the hazard and the state of applied synchronization.
160+
161+
Error message example:
162+
163+
> vkCmdExecuteCommands(): WRITE_AFTER_READ hazard detected. vkCmdCopyImage (from the secondary VkCommandBuffer 0x1fb2f224d40) writes to VkImage 0xf56c9b0000000004, which was previously read by another vkCmdCopyImage command (from the primary VkCommandBuffer 0x1fb245f4200).
164+
>
165+
> No sufficient synchronization is present to ensure that a write (VK_ACCESS_2_TRANSFER_WRITE_BIT) at VK_PIPELINE_STAGE_2_COPY_BIT does not conflict with a prior read (VK_ACCESS_2_TRANSFER_READ_BIT) at the same stage.
166+
>
167+
> Vulkan insight: an execution dependency is sufficient to prevent this hazard.
168+
169+
The error message usually includes the following:
170+
* The Vulkan API function that triggered the synchronization error
171+
* A brief description of the type of race condition (e.g., WRITE_AFTER_WRITE)
172+
* The commands that performed memory accesses and the resource involved (e.g., vkCmdCopyBuffer and vkCmdDispatch accessing the same VkImage)
173+
* Synchronization details: pipeline stages, access types, and applied synchronization
174+
* In some cases, a "Vulkan insight" section at the end of the error message may provide additional information related to the current error
175+
176+
Unlike core validation error messages, where each message is identified by a VUID, synchronization validation primarily detects a single type of error: a race condition between two memory accesses. There are limitless ways to produce a race condition (combinations of command pairs and different synchronization methods), which is why race condition scenarios are not identified by VUIDs.
177+
178+
To suppress or filter synchronization validation error messages, one can use the optional `Extra properties` section. Extra properties contain key-value pairs that help identify the error message and are presented in a more structured format compared to the main error message, making parsing easier.
179+
180+
One of the benefits of parsing `Extra properties` rather than the main error message is that the former is more stable and changes less frequently. This creates a nice separation: on one side, we can take every opportunity to improve the error message wording while keeping the Extra properties values unchanged in most cases, so suppression and filtering logic does not need to be updated.
181+
182+
Example of error message with extra properties enabled:
183+
> vkQueueSubmit(): WRITE_AFTER_WRITE hazard detected. vkCmdEndRenderPass (from VkCommandBuffer submitted on the current VkQueue ) writes to resource, which was previously written by vkCmdClearColorImage (from VkCommandBuffer submitted on VkQueue ).
184+
>
185+
>The current synchronization allows VK_ACCESS_2_COLOR_ATTACHMENT_READ_BIT accesses at VK_PIPELINE_STAGE_2_COLOR_ATTACHMENT_OUTPUT_BIT, but to prevent this hazard, it must allow VK_ACCESS_2_COLOR_ATTACHMENT_WRITE_BIT accesses at VK_PIPELINE_STAGE_2_COLOR_ATTACHMENT_OUTPUT_BIT.
186+
```
187+
[Extra properties]
188+
message_type = SubmitTimeError
189+
hazard_type = WRITE_AFTER_WRITE
190+
prior_access = VK_PIPELINE_STAGE_2_CLEAR_BIT(VK_ACCESS_2_TRANSFER_WRITE_BIT)
191+
write_barriers = VK_PIPELINE_STAGE_2_COLOR_ATTACHMENT_OUTPUT_BIT(VK_ACCESS_2_COLOR_ATTACHMENT_READ_BIT)
192+
command = vkCmdEndRenderPass
193+
prior_command = vkCmdClearColorImage
194+
command_buffer_index = 1
195+
submit_index = 2
196+
batch_index = 0
197+
batch_tag = 5
198+
```
199+
200+
Extra properties can be enabled in Vulkan Configurator or by using `khronos_validation.syncval_message_extra_properties` validation layer setting.
201+
202+
### Frequently Found Issues
203+
204+
* Assuming Pipeline stages are logically extended with respect to memory access barriers. Specifying the vertex shader stage in a barrier will **not** apply to all subsequent shader stages read/write access.
205+
* Invalid stage/access pairs (specifying a pipeline stage for which a given access is not valid) that yield no barrier.
206+
* Relying on implicit subpass dependencies with `VK_SUBPASS_EXTERNAL` when memory barriers are needed.
207+
* Missing memory dependencies with Image Layout Transitions from pipeline barrier or renderpass Begin/Next/End operations.
208+
* Missing stage/access scopes for load operations, noting that color and depth/stencil are done by different stage/access.
209+
210+
211+
### Debugging Tips
212+
213+
* Read and write barriers in the error message can help identify the synchronization operation (either subpass dependency or pipeline barrier) with insufficient or incorrect destination stage/access masks (second scope).
214+
* `Access info read_barrier` and `write_barrier` values of 0, reflect the absence of any barrier, and can indicate an insufficient or incorrect source mask (first scope).
215+
* Insert additional barriers with stage/access `VK_PIPELINE_STAGE_ALL_COMMANDS_BIT`, `VK_ACCESS_MEMORY_READ_BIT`|`VK_ACCESS_MEMORY_WRITE_BIT` for both` src*Mask` and `dst*Mask` fields to locate missing barriers. If the inserted barrier _resolves_ a hazard, the conflicting access _happens-before_ the inserted barrier. (Be sure to delete later.)
216+
217+
218+
## Synchronization blogs/articles
219+
220+
Synchronization Examples[ https://github.com/KhronosGroup/Vulkan-Docs/wiki/Synchronization-Examples](https://github.com/KhronosGroup/Vulkan-Docs/wiki/Synchronization-Examples)
221+
222+
Keeping your GPU fed without getting bitten [ https://www.youtube.com/watch?v=oF7vOTTaAh4](https://www.youtube.com/watch?v=oF7vOTTaAh4)
223+
224+
Yet another blog explaining Vulkan synchronization[ http://themaister.net/blog/2019/08/14/yet-another-blog-explaining-vulkan-synchronization/](http://themaister.net/blog/2019/08/14/yet-another-blog-explaining-vulkan-synchronization/)
225+
226+
A Guide to Vulkan Synchronization Validation https://www.khronos.org/news/permalink/blog-a-guide-to-vulkan-synchronization-validation
227+

0 commit comments

Comments
 (0)