bcache, partitions, and DKMS

bcache is a fantastic way to speed up your system. The idea is simple: You have a fast-but-small SSD and a slow-but-large HDD and you want to cache the HDD with the SSD for better performance. This is what bcache was designed for and it does it’s job very, very well.

It has been in the kernel since 3.10 and according to the author, Kent Overstreet, it has been stable since 2012. I have personally been using it since it was announced as “stable” and haven’t ever had any data corruption issues with it. I even wrote a how-to on it a few years back. I highly recommend it and I am actually a bit shocked it hasn’t gotten more attention from the community at large. My guess is that the reason for that is it requires you to reformat your disk (or use a tool to convert the disk by shuffling metadata blocks) and this turns some people off of the idea.

One of the pain points at the time of this writing I just ran into is bcache devices are not setup to allow partitions. This is due to only being allocated 1 minor number when created in the kernel. Normally this would be ok, you could use tools like kpartx to generate a target for that partition. But in this case I needed a true partition that was predictable and autogenerated by the kernel for use with some existing tooling that expected to be able to take a raw block device and create a partition table and then address the partition on it. That tool was ceph-disk while playing around with the new bluestore OSD backend in Jewel.

The fix for this only-allocates-one-minor number is really quite simple, two lines of code. The issue is compiling the new module and using it. I long ago stopped rolling my own kernel and now-a-days I only do so when I am testing a new feature (like bcachefs). That leaves me with a situation where even if I did patch this partition “issue” upstream, I wouldn’t be able to consume that on a stable kernel for a few years.

DKMS saves the day! You have likely had some experience with DKMS in the past, perhaps when compiling zfs.ko or some proprietary graphics module. It is normally a smooth procedure and you don’t even notice you are actually compiling anything. Let’s start by installing dkms and the headers for your target kernel.

# apt-get install dkms linux-headers

Thanks to DKMS we can build a new, updated bcache module with the small changes needed to support partitions and use it with your preferred packages and stable kernel. The process was pleasantly simple. Since bcache is an in-tree linux kernel module we need to start by grabbing the source for your running kernel. On debian based systems this can be done with the following command:

# apt-get install linux-source

Now you will see the source tarball for your particular kernel in /usr/src/. We need to extract that. After extraction we need to copy the bcache source files to a new directory for dkms to use. The version number here is made up, in this case you should be able to use anything. I chose the kernel version I was working with.

# cd /usr/src
# tar xvf linux-source-3.16.tar.xz
# mkdir bcache-3.16
# cp -av linux-source-3.16/drivers/md/bcache/* bcache-3.16/

At this point we have copied all we need out of the kernel source tree for this particular module. If you are adapting these instructions for a different module then you may need additional files from other locations. Now that we have the source can go ahead and make our changes to the code. Here is a patch of the two lines I have changed. Like I said, very simple code change here.

# diff -up a/super.c b/super.c
--- a/super.c   2016-03-31 21:03:25.189901913 +0000
+++ b/super.c   2016-03-31 21:03:08.205513288 +0000
@@ -780,9 +780,10 @@ static int bcache_device_init(struct bca
        minor = ida_simple_get(&bcache_minor, 0, MINORMASK + 16, GFP_KERNEL);
        if (minor < 0)
                return minor;
+        minor = minor * 16;
        if (!(d->bio_split = bioset_create(4, offsetof(struct bbio, bio))) ||
-           !(d->disk = alloc_disk(1))) {
+           !(d->disk = alloc_disk(16))) {
                ida_simple_remove(&bcache_minor, minor);
                return -ENOMEM;

Now we need to create a dkms.conf file and use dkms to build and install our new module. Finally, we update our initramfs to pull in the new module on boot.

# cat << EOF > bcache-3.16/dkms.conf
# dkms add -m bcache -v 3.16
# dkms build -m bcache -v 3.16
# dkms install -m bcache -v 3.16
# update-initrafs -u -k all

And there you have it. We are now using our version of bcache with slight twist without a custom kernel! Here are the fruits of our labor, bcache0p1 :

# ls -lh /dev/bcache*
brw-rw---- 1 root disk 254,  0 Mar 31 20:17 /dev/bcache0
brw-rw---- 1 root disk 254,  1 Mar 31 20:17 /dev/bcache0p1
brw-rw---- 1 root disk 254, 16 Mar 31 20:17 /dev/bcache16
brw-rw---- 1 root disk 254, 17 Mar 31 20:17 /dev/bcache16p1
brw-rw---- 1 root disk 254, 32 Mar 31 20:17 /dev/bcache32
brw-rw---- 1 root disk 254, 33 Mar 31 20:17 /dev/bcache32p1

4 thoughts on “bcache, partitions, and DKMS

    • Thank you for appreciating it. I have not gotten this upstream yet, though once _a_ solution makes it upstream, the end result should be exactly the same as here.

  1. I can’t wait for a definitive upstream solution to this. It’s a pity we currently can’t easily and realiably take advantage of bcache for Ceph OSDs.

    Thank you for showing this quick workaround !

    • This was merged into the 4.10 kernel fyi. So starting with ubuntu 17.04 it will be available from the Ubuntu kernel without any DKMS.

      Unfortuntely Debian stretch is using 4.9. So it will be 2-3 years from this point until its available in Debian.

Leave a Reply

Your email address will not be published. Required fields are marked *