I got your LUNs right here, pal

2006 December 21
by Dick

I spotted iSCSI in the autumn and it seemed to tick a lot of boxes
(think SANs without the hype, HBAs, fibre channel, vendor lock-in or horrendous cost).

b54 SXCR has really nice ZFS iscsi integration :

zfs create tank/iscsi
zfs set mountpoint=legacy tank/iscsi
zfs set shareiscsi=on tank/iscsi

Then start the iSCSI target daemon1. Nothing is shared yet.

sol# svcadm enable iscsitgt
sol# iscsitadm list target
sol#

Thanks to ZFS inheritance, any zvols (not filesystems, I checked) created under tank/iscsi will automatically become LUNs:

zfs create -V 4G tank/iscsi/lefty
zfs create -V 4G tank/iscsi/righty

And that’s it.

iscsitadm list target -v
Target: tank/iscsi/lefty
    iSCSI Name: iqn.1986-03.com.sun:02:5ee25ac8-4930-ea8e-bc86-9d475793e277
    Alias: tank/iscsi/lefty
    Connections: 0
    ACL list:
    TPGT list:
    LUN information:
        LUN: 0
            GUID: 0x0
            VID: SUN
            PID: SOLARIS
            Type: disk
            Size: 4.0G
            Backing store: /dev/zvol/rdsk/tank/iscsi/lefty
            Status: online
Target: tank/iscsi/righty
    iSCSI Name: iqn.1986-03.com.sun:02:9cdcfa4d-a1dc-40f2-d70d-f294d5d9151f
    Alias: tank/iscsi/righty
    Connections: 0
    ACL list:
    TPGT list:
    LUN information:
        LUN: 0
            GUID: 0x0
            VID: SUN
            PID: SOLARIS
            Type: disk
            Size: 4.0G
            Backing store: /dev/zvol/rdsk/tank/iscsi/righty
            Status: online

Renaming the ZVOLs causes the iSCSI target/alias to update (the IQN is preserved so clients should be OK) :

zfs rename tank/iscsi/lefty tank/iscsi/pinky
zfs rename tank/iscsi/righty tank/iscsi/perky
iscsitadm list target
Target: tank/iscsi/pinky
   iSCSI Name: iqn.1986-03.com.sun:02:5ee25ac8-4930-ea8e-bc86-9d475793e277
   Connections: 0
Target: tank/iscsi/perky
   iSCSI Name: iqn.1986-03.com.sun:02:9cdcfa4d-a1dc-40f2-d70d-f294d5d9151f
   Connections: 0

When you do the usual zfs magic, iscsitadm seems to keeps up:

zfs snapshot tank/iscsi/lefty@snap
zfs clone tank/iscsi/lefty@snap tank/iscsi/sinister
iscsitadm list target
Target: tank/iscsi/righty
    iSCSI Name: iqn.1986-03.com.sun:02:4cf907ba-db89-4403-c1bb-ed5db6fcb76b
    Connections: 0
Target: tank/iscsi/lefty
    iSCSI Name: iqn.1986-03.com.sun:02:25fb9c0e-c58f-edcc-e8bc-d9686c0f0e88
    Connections: 0
Target: tank/iscsi/sinister
    iSCSI Name: iqn.1986-03.com.sun:02:c734e4a8-7d65-e427-80c7-dda523e814b7
    Connections: 0

Note the new IQN for the clone.
Adding

zfs set readonly=on  tank/iscsi/sinister

gets you the benefit of the .zfs/snapshot directory NFS clients would see if you’d set sharenfs=on (OK, you’re sharing a snapshot of the entire LUN rather than the filesystem, but clients can still get their old files without bugging you, so it does the job).

Course, you can share them out read/write too.
Combining that with ‘zfs promote’ should be more than enough rope for anyones needs :)

Having all the config in ZFS itself means that
when I upgrade the server, I just need to zpool import -f tank and I get my iscsi LUNs along with everything else.

There are a few things lacking – there’s no way to configure TPGT groups, it’s 1 LUN per target – but nothing I need badly. Discovery works fine.

(UPDATE: if you need TPGT support, you can use iscsitadm directly until it’s integrated into SXCR)

1 Without the last step, you’ll get an error (when inheriting the shareiscsi property) when you create the first zvol:

cannot share 'tank/iscsi/lefty': iscsitgtd failed request to share
filesystem successfully created, but not shared

which is bollocks, fortunately. It autostarts iscsitgtd for you.

4 Responses leave one →
  1. 2006 December 21

    So now they’re not zero-filled? Cool.

  2. 2006 December 21

    Can’t be sure, but I don’t see how sparse volumes could work otherwise.

  3. 2008 December 19

    Dick,

    Can you explain what you mean by:

    “Adding
    zfs set readonly=on tank/iscsi/sinister
    gets you the benefit of the .zfs/snapshot directory NFS clients would see if you’d set sharenfs=on (OK, you’re sharing a snapshot of the entire LUN rather than the filesystem, but clients can still get their old files without bugging you, so it does the job).”

    How does this work to get clients to see their stuff without bugging you? sinister is a clone of a snapshot of lefty. If this were straight zfs (including filesystem), then any client could just go to .zfs/snapshot under their volume and get the data. For iSCSI, they are mounting lefty as a single iSCSI target. How do you get them to see the other snapshots?

    Thanks.

  4. 2009 January 17

    Wow, that was a while ago :)

    But I think my plan was to name the clones in a standard way, so a iSCSI client would know that they could attach ‘your-lun-monday’ and get mondays volume.

    Like I said, it’s much clankier than in-filesystem snapshots, but better than nothing.

Leave a Reply

Note: You can use basic XHTML in your comments. Your email address will never be published.

Subscribe to this comment feed via RSS