Question about the verification of old and new data

[MorningWeicheng]

I'd like to confirm: Can fio's verify-related parameters (such as verify_state_save) check the "newness" or "oldness" of data? For example, after writing data to the same LBA, can it verify whether the data read back is the last written data? Additionally, after writing data and performing a power cycle (powering off and then on again), can it verify whether the data read back is the last written data before the power cycle?

[sitsofe]

Hello @MorningWeicheng:

Can fio's verify-related parameters (such as verify_state_save) check the "newness" or "oldness" of data?

No, split verification (where one job writes the data and a separate job verifies it) does not verify the "generation" or seed a block was written in:

./fio --name=sequential --rw=write --verify=crc32c --size=16k --do_verify=0 --filename=/tmp/fio-sequential.tmp
sequential: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.40-28-g89c6d-dirty
Starting 1 process
sequential: Laying out IO file (1 file / 0MiB)

sequential: (groupid=0, jobs=1): err= 0: pid=1678: Wed Aug 20 17:29:42 2025
[...]
./fio --name=random --rw=randwrite --verify=crc32c --size=16k --do_verify=0 --filename=/tmp/fio-random.tmp
random: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.40-28-g89c6d-dirty
Starting 1 process
random: Laying out IO file (1 file / 0MiB)

random: (groupid=0, jobs=1): err= 0: pid=1813: Wed Aug 20 17:30:11 2025
[...]
# Verify fio-random.tmp using the random state file
./fio --name=random --rw=randread --verify=crc32c --size=16k --filename=/tmp/fio-random.tmp --verify_state_load=1
random: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.40-28-g89c6d-dirty
Starting 1 process

random: (groupid=0, jobs=1): err= 0: pid=1942: Wed Aug 20 17:30:40 2025
[...]
# Verify fio-sequential.tmp using the random state file by keeping the job name the same as above but changing the file
./fio --name=random --rw=randread --verify=crc32c --size=16k --filename=/tmp/fio-sequential.tmp --verify_state_load=1
random: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.40-28-g89c6d-dirty
Starting 1 process

random: (groupid=0, jobs=1): err= 0: pid=2003: Wed Aug 20 17:30:54 2025
[...]

One might expect the data that was written sequentially not to verify with the state file from the random run but it does because the block sizes are the same and all the file was written - fio is reading each block from the disk, looking at the header it just read and then using that to verify the rest of the block is correct. Techniques to cope with this involve writing a different fixed pattern on each "pass" and then checking the exact pattern at verification time or changing the block size (e.g. from 4k to 8k) between passes.

Additionally, after writing data and performing a power cycle (powering off and then on again), can it verify whether the data read back is the last written data before the power cycle?

If you are somehow telling (and letting) fio to write its state file before cutting the power then yes. See Verification and triggers in the fio documentation (I think @vincentkfu might have been going to write about it in a blog post on https://github.com/vincentkfu/fio-blog/wiki/Verify-overview#state-files but it says "See this blog post for more details about the verify state file" without a link...).

Does this answer your question?

[MorningWeicheng]

Thank you for your answer, and it was quiet helpful to me. But I still got a question: excluding the "verify_state_load" parameter, when writing twice to the same address, does fio have a way to verify whether the data read back is the one written in the second operation?

[sitsofe]

@MorningWeicheng:

excluding the "verify_state_load" parameter, when writing twice to the same address, does fio have a way to verify whether the data read back is the one written in the second operation?

I think this was answered in my original reply:

Can fio's verify-related parameters (such as verify_state_save) check the "newness" or "oldness" of data?

No, split verification (where one job writes the data and a separate job verifies it) does not verify the "generation" or seed a block was written in

But for the sake of clarity: today (August 2025) with split verification, it doesn't matter whether verify_state_load is used or not - fio does not verify the "generation" that blocks come from (i.e. whether the data is from the first write or a later overwrite). If you want a previous generation's blocks to fail verification you will have to do something that makes them different between overwrites (e.g. using a different fixed data pattern, a different block size etc).

If this answers your question can you close this discussion? Thanks!

[MorningWeicheng]

Now I've got everything I want to comfirm, thanks!

[MorningWeicheng]

Close this discusstion