With thanks once again to Ubuntu Community Forums, I managed to get my Caps Lock key working. Not one of the things you’d think would be tricky, but…

sudo apt-get --purge remove mouseemu

Then reboot.