V-Series on Netapp, Lun removal notes
Posted: Mon Nov 24, 2014 2:56 pm
We had a V-series Netapp (v3140) in production using 3PAR luns for its own storage. In fact, it even used 3PAR for aggr0 and lun0. The time has come to decommission this v-Series and here are some commands, notes, tips, I would like to share.
Once you assign a 3PAR LUN to a Netapp aggregate, it is committed until you destroy the entire aggregate.
Netapp does not support growing LUNs already assigned to Netapp.
Netapp's are not thin provision friendly to the 3PAR. Disk scrubs/checksumming/other will fully commit your LUN within a short amount of time. Save your TP licensing and use full provisioned volumes from the CPG.
Best practice on the 3PAR side is to dedicate host ports to the netapp ports. This is because the netapps are very chatty, even when idle... all part of the disk scub/check summing/other that they do in the back ground.. when idle our netapps would churn a constant 2000 iops.
In order to remove/delete a LUN assigned to the Netapp without a reboot, the LUN has to be unowned by either node of the netapp. In order to remove ownership, the LUN can not belong to a aggregate. If you want to "shrink" an aggregate, you basically will need to create a new smaller one, and migrate all your volumes to the new one, and delete the old larger aggregate.
There is some secret sauce to gracefully removing ownership of LUNs so that you can unexport them from the 3PAR with out a reboot. Not doing this correctly will cause the Netapp to mark the disk as failed and retain information about it even after its been removed/deleted on the back end.
To show all disks, owned by aggregates and raid sets:
sysconfig -r
To show LUNs assigned to the filers but not in use (aka spares), these need to be un-owned first:
vol status -s
To show all the disks owned by node:
disk show -o <node name>
To Check for "unassigned" disks ready to be unexported:
disk show -n
Use the above utils/cmds to build a list of LUNs you wish to remove from the system. To remove them use the following commands:
(change to a privileged command mode)
priv set advanced
disk remove_ownership <LUN1 NAME>
disk remove_ownership <LUN2 NAME>
disk remove_ownership <LUN..etc NAME>
priv set
(returns to normal command mode)
At this point you can remove the export from the 3PAR side.
You will get console and log messages on the Netapp while this happens, here are some examples:
Mon Nov 24 11:02:35 CST [ESFWV3140-1A:mlm.array.lun.OnePath:warning]: Array LUN 'ESFW9509-A2:8-11.138L28' is connected using only one path. To avoid the potential for data loss caused by the remaining path's failure, all array LUNs should have at least two paths.
Mon Nov 24 11:03:35 CST [ESFWV3140-1A:mlm.array.lun.removed:warning]: Array LUN 'ESFW9509-B2:7-11.138L4' (50002AC0020107DA0000000000000000) is no longer being presented to this node.
Once you assign a 3PAR LUN to a Netapp aggregate, it is committed until you destroy the entire aggregate.
Netapp does not support growing LUNs already assigned to Netapp.
Netapp's are not thin provision friendly to the 3PAR. Disk scrubs/checksumming/other will fully commit your LUN within a short amount of time. Save your TP licensing and use full provisioned volumes from the CPG.
Best practice on the 3PAR side is to dedicate host ports to the netapp ports. This is because the netapps are very chatty, even when idle... all part of the disk scub/check summing/other that they do in the back ground.. when idle our netapps would churn a constant 2000 iops.
In order to remove/delete a LUN assigned to the Netapp without a reboot, the LUN has to be unowned by either node of the netapp. In order to remove ownership, the LUN can not belong to a aggregate. If you want to "shrink" an aggregate, you basically will need to create a new smaller one, and migrate all your volumes to the new one, and delete the old larger aggregate.
There is some secret sauce to gracefully removing ownership of LUNs so that you can unexport them from the 3PAR with out a reboot. Not doing this correctly will cause the Netapp to mark the disk as failed and retain information about it even after its been removed/deleted on the back end.
To show all disks, owned by aggregates and raid sets:
sysconfig -r
To show LUNs assigned to the filers but not in use (aka spares), these need to be un-owned first:
vol status -s
To show all the disks owned by node:
disk show -o <node name>
To Check for "unassigned" disks ready to be unexported:
disk show -n
Use the above utils/cmds to build a list of LUNs you wish to remove from the system. To remove them use the following commands:
(change to a privileged command mode)
priv set advanced
disk remove_ownership <LUN1 NAME>
disk remove_ownership <LUN2 NAME>
disk remove_ownership <LUN..etc NAME>
priv set
(returns to normal command mode)
At this point you can remove the export from the 3PAR side.
You will get console and log messages on the Netapp while this happens, here are some examples:
Mon Nov 24 11:02:35 CST [ESFWV3140-1A:mlm.array.lun.OnePath:warning]: Array LUN 'ESFW9509-A2:8-11.138L28' is connected using only one path. To avoid the potential for data loss caused by the remaining path's failure, all array LUNs should have at least two paths.
Mon Nov 24 11:03:35 CST [ESFWV3140-1A:mlm.array.lun.removed:warning]: Array LUN 'ESFW9509-B2:7-11.138L4' (50002AC0020107DA0000000000000000) is no longer being presented to this node.