Opened 2 years ago

Closed 20 months ago

#226 closed enhancement (implemented)

Safe USB block devices/DVD attach to VMs

Reported by: marmarek Owned by: marmarek
Priority: major Milestone: Release 1 Beta 3
Component: kernel Keywords:
Cc:

Description

For now, xen-4 + linux-2.6.34.xenlinux.qubes doesn't work with PV USB.
xm usb-attach gives OOPS in domU (something about creating duplicate entry in sysfs).

For future investigation, maybe upgrade kernel in domU will solve the problem.

Change History (9)

comment:1 Changed 2 years ago by joanna

  • Type changed from defect to enhancement

comment:2 Changed 2 years ago by marmarek

Kernel message from domU on usb-attach (2.6.34...):

[  852.056870] ------------[ cut here ]------------
[  852.056894] WARNING: at /home/joanna/qubes/kernel-dom0/kernel-2.6.34.1/linux-2.6.34.1/fs/sysfs/dir.c:451 sysfs_add_one+0xc8/0x130()
[  852.056913] sysfs: cannot create duplicate filename '/devices/xen/vusb-0/usb1/power/power'
[  852.056931] Modules linked in: xen_hcd fuse ipt_REJECT xt_state xt_tcpudp iptable_filter ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables x_tables binfmt_misc dm_mirror dm_region_hash dm_log u2mfn evtchn uinput xennet usbcore ext4 jbd2 crc16 dm_snapshot xenblk cdrom [last unloaded: scsi_wait_scan]
[  852.057083] Pid: 222, comm: khubd Not tainted 2.6.34.1-13.xenlinux.qubes.x86_64 #1
[  852.057098] Call Trace:
[  852.058139] Inexact backtrace:
[  852.058143] 
[  852.058170]  [<ffffffff80042ec3>] ? warn_slowpath_common+0x73/0xb0
[  852.058186]  [<ffffffff80042f60>] ? warn_slowpath_fmt+0x40/0x50
[  852.058202]  [<ffffffff8017ed88>] ? sysfs_add_one+0xc8/0x130
[  852.058217]  [<ffffffff8017ee50>] ? create_dir+0x60/0xb0
[  852.058239]  [<ffffffff80180352>] ? internal_create_group+0x52/0x1b0
[  852.058256]  [<ffffffff803cf807>] ? klist_add_tail+0x27/0x80
[  852.058280]  [<ffffffff8029465e>] ? device_add+0x1fe/0x4f0
[  852.058315]  [<ffffffffa00e2bdd>] ? usb_new_device+0x9d/0x1a0 [usbcore]
[  852.058354]  [<ffffffffa00e5b48>] ? hub_port_connect_change+0x4e8/0xb00 [usbcore]
[  852.058394]  [<ffffffffa00ec384>] ? usb_control_msg+0x114/0x1a0 [usbcore]
[  852.058427]  [<ffffffffa00e6524>] ? hub_events+0x3c4/0x930 [usbcore]
[  852.058463]  [<ffffffffa00e6ad5>] ? hub_thread+0x45/0x1b0 [usbcore]
[  852.058484]  [<ffffffff80063350>] ? autoremove_wake_function+0x0/0x30
[  852.058518]  [<ffffffffa00e6a90>] ? hub_thread+0x0/0x1b0 [usbcore]
[  852.058535]  [<ffffffff80062d6e>] ? kthread+0x8e/0xa0
[  852.058553]  [<ffffffff80007d64>] ? kernel_thread_helper+0x4/0x10
[  852.058572]  [<ffffffff80062ce0>] ? kthread+0x0/0xa0
[  852.058590]  [<ffffffff80007d60>] ? kernel_thread_helper+0x0/0x10
[  852.058606] ---[ end trace 2edf83825a029fdc ]---
[  852.058745] usb 1-1: can't device_add, error -17
[  852.058787] BUG: unable to handle kernel paging request at ffff89a40764a380
[  852.058807] IP: [<ffffffff801059ba>] kfree+0x7a/0x290
[  852.058830] PGD 0 
[  852.058843] Oops: 0000 [#1] SMP 
[  852.058857] last sysfs file: /sys/devices/xen/vusb-0/usb1/1-0:1.0/bInterfaceProtocol
[  852.058873] CPU 0 
[  852.058880] Modules linked in: xen_hcd fuse ipt_REJECT xt_state xt_tcpudp iptable_filter ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables x_tables binfmt_misc dm_mirror dm_region_hash dm_log u2mfn evtchn uinput xennet usbcore ext4 jbd2 crc16 dm_snapshot xenblk cdrom [last unloaded: scsi_wait_scan]
[  852.058999] 
[  852.059009] Pid: 222, comm: khubd Tainted: G        W  2.6.34.1-13.xenlinux.qubes.x86_64 #1 /
[  852.059026] RIP: e030:[<ffffffff801059ba>]  [<ffffffff801059ba>] kfree+0x7a/0x290
[  852.059047] RSP: e02b:ffff88000302bc00  EFLAGS: 00010002
[  852.059059] RAX: ffff89a40764a380 RBX: ffff880003e4a000 RCX: 0000000000000009
[  852.059073] RDX: 0000003c00800080 RSI: ffffffff801ef180 RDI: 0000000100010123
[  852.059086] RBP: 0000000100010123 R08: 0000000000000000 R09: 0000000000000000
[  852.059099] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880016de1e00
[  852.059112] R13: ffff880018f8f000 R14: 0000000000000000 R15: 00000000ffffffef
[  852.059135] FS:  00007fbddd681720(0000) GS:ffff88000357f000(0000) knlGS:0000000000000000
[  852.059151] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  852.059163] CR2: ffff89a40764a380 CR3: 0000000017824000 CR4: 0000000000002660
[  852.059177] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  852.059192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  852.059206] Process khubd (pid: 222, threadinfo ffff88000302a000, task ffff8800028182c0)
[  852.059220] Stack:
[  852.059227]  0000000000000000 0000000000000001 0000000000000023 005a005a00000001
[  852.059247] <0> 0000000000000001 0000000000000001 ffff88000a190000 ffff880016de1e60
[  852.059271] <0> ffff880018f8f000 0000000000000000 00000000ffffffef ffffffffa00f04ee
[  852.059299] Call Trace:
[  852.060219] Inexact backtrace:
[  852.060223] 
[  852.060260]  [<ffffffffa00f04ee>] ? usb_destroy_configuration+0x4e/0x120 [usbcore]
[  852.060290]  [<ffffffffa00df201>] ? usb_release_dev+0x21/0x70 [usbcore]
[  852.060307]  [<ffffffff802935aa>] ? device_release+0x1a/0x70
[  852.060323]  [<ffffffff801eefc0>] ? kobject_cleanup+0x80/0x240
[  852.060337]  [<ffffffff801ef180>] ? kobject_release+0x0/0x20
[  852.060351]  [<ffffffff801f0663>] ? kref_put+0x33/0x70
[  852.060382]  [<ffffffffa00e59d5>] ? hub_port_connect_change+0x375/0xb00 [usbcore]
[  852.060416]  [<ffffffffa00ec384>] ? usb_control_msg+0x114/0x1a0 [usbcore]
[  852.060447]  [<ffffffffa00e6524>] ? hub_events+0x3c4/0x930 [usbcore]
[  852.060477]  [<ffffffffa00e6ad5>] ? hub_thread+0x45/0x1b0 [usbcore]
[  852.060495]  [<ffffffff80063350>] ? autoremove_wake_function+0x0/0x30
[  852.060523]  [<ffffffffa00e6a90>] ? hub_thread+0x0/0x1b0 [usbcore]
[  852.060538]  [<ffffffff80062d6e>] ? kthread+0x8e/0xa0
[  852.060553]  [<ffffffff80007d64>] ? kernel_thread_helper+0x4/0x10
[  852.060568]  [<ffffffff80062ce0>] ? kthread+0x0/0xa0
[  852.060581]  [<ffffffff80007d60>] ? kernel_thread_helper+0x0/0x10
[  852.060593] Code: c1 67 00 00 01 48 89 ef 48 8b 1d 92 f4 77 00 e8 ed ef f1 ff 48 c1 e8 0c 48 8d 14 c5 00 00 00 00 48 c1 e0 06 48 29 d0 48 8d 04 03 <48> 8b 10 f7 c2 00 00 01 00 0f 85 76 01 00 00 84 d2 0f 89 6a 01 
[  852.060600] RIP  [<ffffffff801059ba>] kfree+0x7a/0x290
[  852.060600]  RSP <ffff88000302bc00>
[  852.060600] CR2: ffff89a40764a380
[  852.060600] ---[ end trace 2edf83825a029fdd ]---
[  892.037281] ------------[ cut here ]------------
[  892.037300] kernel BUG at /home/joanna/qubes/kernel-dom0/kernel-2.6.34.1/linux-2.6.34.1/mm/slab.c:3086!
[  892.037307] invalid opcode: 0000 [#2] SMP 
[  892.037314] last sysfs file: /sys/devices/xen/vusb-0/usb1/1-0:1.0/bInterfaceProtocol
[  892.037319] CPU 0 
[  892.037322] Modules linked in: xen_hcd fuse ipt_REJECT xt_state xt_tcpudp iptable_filter ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables x_tables binfmt_misc dm_mirror dm_region_hash dm_log u2mfn evtchn uinput xennet usbcore ext4 jbd2 crc16 dm_snapshot xenblk cdrom [last unloaded: scsi_wait_scan]
[  892.037368] 
[  892.037373] Pid: 12, comm: xenbus Tainted: G      D W  2.6.34.1-13.xenlinux.qubes.x86_64 #1 /
[  892.037380] RIP: e030:[<ffffffff80104caf>]  [<ffffffff80104caf>] cache_alloc_refill+0x23f/0x290
[  892.037394] RSP: e02b:ffff880015907e10  EFLAGS: 00010046
[  892.037399] RAX: 0000000000000018 RBX: ffff880018c65800 RCX: ffff880018efe000
[  892.037405] RDX: ffff880016de1000 RSI: 0000000000000070 RDI: 0000000000000018
[  892.037410] RBP: ffff880018c00040 R08: ffff880018c44450 R09: ffff880018c44460
[  892.037414] R10: ffff880016de1030 R11: dead000000200200 R12: ffff880018c44440
[  892.037419] R13: dead000000100100 R14: ffff880018c67000 R15: ffff880018c44480
[  892.037429] FS:  00007fbddd681720(0000) GS:ffff88000357f000(0000) knlGS:0000000000000000
[  892.037435] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  892.037439] CR2: 00000000004fb5a0 CR3: 0000000017824000 CR4: 0000000000002660
[  892.037445] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  892.037450] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  892.037456] Process xenbus (pid: 12, threadinfo ffff880015906000, task ffff880015904340)
[  892.037461] Stack:
[  892.037464]  ffff88000000003c 0000003000000000 0000000000000000 0000000000000020
[  892.037472] <0> 0000000000000010 0000000000000030 ffffffff802af933 ffff880018c00040
[  892.037481] <0> 0000000000000000 ffffffff80105220 0000000000000001 0000000000000030
[  892.037491] Call Trace:
[  892.038274] Inexact backtrace:
[  892.038276] 
[  892.038288]  [<ffffffff802af933>] ? process_msg+0xb3/0x2d0
[  892.038294]  [<ffffffff80105220>] ? __kmalloc+0x1e0/0x210
[  892.038300]  [<ffffffff802afb50>] ? xenbus_thread+0x0/0x50
[  892.038305]  [<ffffffff802af933>] ? process_msg+0xb3/0x2d0
[  892.038310]  [<ffffffff802afb50>] ? xenbus_thread+0x0/0x50
[  892.038315]  [<ffffffff802afb6d>] ? xenbus_thread+0x1d/0x50
[  892.038322]  [<ffffffff80062d6e>] ? kthread+0x8e/0xa0
[  892.038330]  [<ffffffff80007d64>] ? kernel_thread_helper+0x4/0x10
[  892.038335]  [<ffffffff80062ce0>] ? kthread+0x0/0xa0
[  892.038341]  [<ffffffff80007d60>] ? kernel_thread_helper+0x0/0x10
[  892.038345] Code: 48 8d 7c d3 18 48 8d 14 c5 00 00 00 00 49 8d 74 ce 18 e8 25 31 0f 00 45 29 2e 44 01 2b 49 8b 44 24 48 80 48 0c 01 e9 65 ff ff ff <0f> 0b eb fe 0f 0b eb fe 8b 74 24 0c 48 89 ef e8 fd f8 ff ff 65 
[  892.038402] RIP  [<ffffffff80104caf>] cache_alloc_refill+0x23f/0x290
[  892.038409]  RSP <ffff880015907e10>
[  892.038414] ---[ end trace 2edf83825a029fde ]---

comment:3 Changed 2 years ago by joanna

  • Summary changed from PV USB Attach to Safe USB block devices/DVD attach to VMs

We decided to implement USB/DVD attach via xm block-attach. This requires, however, that we modify Dom0 kernel to not parse the block device's partition table (for security reasons).

Changes the name of the ticket accordingly.

comment:4 Changed 23 months ago by joanna

As I wrote in an email some time ago:

After I successfully used my USB disk yesterday via my NetVM, I started
thinking that we simple might not need pvusb at all.

First, there are a few types of USB devices that are most commonly
plugged into the USB ports:

1) USB storage
2) USB modems (e.g. 3G)
3) USB keyboards and mouses
4) USB authentication tokens/smart cards

Regarding #3 and #4 -- for obvious reasons we would like to keep them in
the most trusted domain, the Dom0.

So, we're left only with 3G modems and storage devices. Now, adding to
the fact that a security-conscious user might (should) always use
LUKS/trucrypt/etc to securely store data on an external USB disk, it
should be perfectly acceptable to share one domain that has access to
both 3G modems as well as all the pluggable USB disks. Most likely this
should be a different domain than the default NetVM (which itself has
large attack vector through exposition to untrusted WiFis? in
hotels/airports via their WiFi? drivers and dhcp client-like tools),
because of this threat:

http://www.pcpro.co.uk/news/368383/40gb-of-data-that-costs-the-same-as-a-house

So, the bottom line is that if the user is able to delegate a USB
controller (that servs the USB external ports), then all the user might
need to take full advantage of this setup is that:
1) our UsbVM be also a type of netvm (to take advantage of 3G modems)
2) should use blkbk to expose (untrusted) block devices to other domains
(those can later use cryptsetup or truecrypt to turn them into
ultimately trusted storage).

As a bonus (for those systems where the USB controllers can be nicely
divided into 1) those that serve externa USB ports, and 2) those that
serve only internal USB devices -- like it is on my Core i5 laptop), we
get resistance to all sorts of USB attacks via malicious devices that
turn out to be e.g. malicious keyboards. In order to protect against
malicious partitions one would need to delegate single volumes (e.g.
phy:/dev/sda1) instead of the whole devices (e.g. phy:/dev/sda) -- would
that be all that is needed?

The only problem would be with systems that do not strictly divide USB
controllers into those that serv external USB ports and those that serve
only the internal ones (such as: camera, fingerprint reader, and
(optionally) internal keyboard and mouse). Still, the use of pvusb would
not change much in this case, would it?

So, the bottom line is: all we need in the UsbVM is a blkbk, not a pvusb
back. Which is cool, because this is a well tested code.

Regarding the USB-based auth tokens/smart cards -- one might argue that
they would be unusable in this setup. But there are two counter-arguments:
1) I'm pretty sure pvusb, at least at current stage, would not support
many such tokens anyway,
2) We are preparing a nice alternative for a crypto card
using... qvm-run -- stay tuned!

comment:5 Changed 23 months ago by joanna

  • Milestone changed from Release 1 Beta 2 to Release 1 Beta 3

So, I don't think we should do anything more in this areas as of Beta 2. In Beta 3 we might think how to provide a nice, user-friendly UI (e.g. via Qubes Manager) that would hide the details of doing xl attach (so, would also know which domain is providing blkback for a given device).

So, moving this to Beta 3.

comment:6 Changed 20 months ago by joanna

So, the idea is that every domain that has assigned some device, such as a USB controller, which provides a block device, creates a key in xen store, informing Dom0 about it.

Then we will have a tool in Dom0, e.g. qvm-block, that would allow to do things such as:

  • qvm-block -l, which would list all available block devices in the system (including their friendly names)
  • qvm-block -a <block_dev> <appvm>, which would attach the device (possibly owned by some other domain) to the domain <appvm>
  • qvm-block -r <block_dev> <-- which would detach the device.

A special case might be handling of CDROM devices.

comment:7 Changed 20 months ago by marmarek

  • Status changed from new to accepted

comment:8 Changed 20 months ago by marmarek

http://git.qubes-os.org/gitweb/?p=marmarek/core.git;a=commit;h=e3993ca5f9c2e4f1fa5c55da7dd33502b686428d
TODO:

  • fix xl tool to not verify device name (fails when backend!=dom0)
  • auto detach device when underlaying dev disconnected (eg. stick removed)
Note: See TracTickets for help on using tickets.