Network-Block-Device Copy-on-Write fixes
Berend De Schouwer
Fixing two copy-on-write bugs in the NBD server.
What is What?
NBD is a network block device protocol. It has some overlap with iSCSI, and a little with NFS. The protocol is much simpler than either, and has one extra feature: copy-on-write.
Copy-on-write allows for sharing the same file to multiple machines, and only writing changes back to disk. This can save a lot of storage space in certain situations.
There are two bugs, and two one-line fixes presented here.
The Bugs
Sequential Read after Write
For the first case, it’s enough to read and write more than one sequential block. The second and subsequent blocks read will read into the wrong offset of the buffer, and copy invalid data to the client.
I use a 4096 block size in this example, but I’ve used others. I did that to match the filesystem, but for the test I don’t even need a filesystem.
export OFFSET=0
export COUNT=3 # anything >= 1
dd if=/dev/urandom of=testdata bs=4096 count=$COUNT # random data
dd if=testdata of=/dev/nbd0 bs=4096 seek=$OFFSET count=$COUNT
dd if=/dev/nbd0 of=compdata bs=4096 skip=$OFFSET count=$COUNT
sum testdata compdata
The data, testdata and compdata, will be different.
If the kernel does a partition check when /dev/nbd0 is mounted, this test will fail with COUNT=1 as well.
Sparse Write at the Wrong Offset
For the second case, with sparse_cow=true, we need to repeat the test with an offset > 0. expwrite() calls write() instead of pwrite().
export OFFSET=100
export COUNT=3 # anything >= 1
dd if=/dev/urandom of=testdata bs=4096 count=$COUNT # random data
dd if=testdata of=/dev/nbd0 bs=4096 seek=$OFFSET count=$COUNT
dd if=/dev/nbd0 of=compdata bs=4096 skip=$OFFSET count=$COUNT
sum testdata compdata
The first time it’s run, it will result in an Input/Output error.
The second time it’s run, it will work.
The Patches
diff --git a/nbd-orig/nbd-server.c b/nbd-patched/nbd-server.c
index 92fd141..18e5ddd 100644
--- a/nbd-orig/nbd-server.c
+++ b/nbd-patched/nbd-server.c
@@ -1582,6 +1582,7 @@ int expread(READ_CTX *ctx, CLIENT *client) {
if (pread(client->difffile, buf, rdlen, client->difmap[mapcnt]*DIFFPAGESIZE+offset) != rdlen) {
goto fail;
}
+ ctx->current_offset += rdlen;
confirm_read(client, ctx, rdlen);
} else { /* the block is not there */
if ((client->server->flags & F_WAIT) &&
(client->export == NULL)){
diff --git a/nbd-orig/nbd-server.c b/nbd-patched/nbd-server.c
index 92fd141..9a57ad5 100644
--- a/nbd-orig/nbd-server.c
+++ b/nbd-patched/nbd-server.c
@@ -1669,7 +1669,7 @@ int expwrite(off_t a, char *buf, size_t len,
CLIENT *client, int fua) {
if(ret < 0 ) goto fail;
}
memcpy(pagebuf+offset,buf,wrlen) ;
- if (write(client->difffile, pagebuf, DIFFPAGESIZE) != DIFFPAGESIZE)
+ if (pwrite(client->difffile, pagebuf, DIFFPAGESIZE, client->difmap[mapcnt]*DIFFPAGESIZE) != DIFFPAGESIZE)
goto fail;
}
if (!(client->server->flags & F_COPYONWRITE))