Proper Software iSCSI Initiator Setup for VMware vSphere 4

As with anything in VMware there is the right way to do something and the wrong way to do it... or rather, there is the I slapped this together and it works way (the wrong way) or the best practice wont give you any trouble way (the correct way). So I am going to focus on right way to setup the Software iSCSI initiator setup in vSphere 4 with this post; yes it is true that you can simply right click the SWiSCSI device in vCenter (or the VIC) and click enable and give it a target to discover, and that may work for a while... but in reality there is a more complex and correct way to set the SWiSCSI up.

First things first... there are many, many ways to setup SWiSCSI for vSphere 4, you can use redundant switches, but then you cannot use Port Channeling unless you have lots of money and infrastructure to buy a switch that can do Port Channeling over different physical switches. You can use redundant VMkernel ports on the same subnet, but unless your iSCSI target responds on the same interface as the traffic came in on you end up using just 1 interface for outbound traffic. You can have multiple vSwitches with multiple VMkernels but then you add the overhead and complexity of that configuration. You can have a single VMkernel with 'standby' links, but then you only have 1 active link at a time. The list can go on and on... So for this post I will focus on what I believe is the most stable and redundant method there is (unless of course you have money to burn and very expensive switches).

The situation that I recommend is one of simple configuration and ease of management, this configuration looks like so... There is 1 vSwitch with multiple VMkernel Port Groups associated with this vSwitch all on different subnets, along with multiple pNICs (Physical NICs), each VMkernel would have 1 active pNIC and the rest of the pNICs would be 'Unused' and not Stand-By or Active. This will allow us to use Round Robin with the mulitpathing for the SWiSCSI, and a level of redundancy with the SWiSCSI traffic. What this does not give us though is the higher bandwidth that comes with Port Channeling but that higher bandwidth would not come in to play unless there is a very high end iSCSI array servicing the SWiSCSI and the request for that much traffic comes in a single SCSI command (both of which are unlikely as of today).

On to the actual configuration of the SWiSCSI adapter and VMkernel ports...

1. Open vCenter or the VIC
2. Select the desired ESX host to be configured
3. In the upper tab listing select the Configuration tab
4. In the Hardware selection box, select Networking
5. In the upper right corner select 'Add Networking' (unless you already have the vSwitch and vmk's configured)
6. Select VMkernel and click next
7. Configure the VMkernel with and IP and subnet (do not specify a Gateway unless you have a real need to)
8. NOTE: You will need to repeat this step for every pNIC that you are going to assign to this vSwitch. Also note that each vmk should be on its own subnet
9. If this is the first vmk then you will need to make a new vSwitch and add the pNICs that are desired to that vSwitch; if this is not the first vmk then select to add this vmk to the existing vSwitch with the other vmk's
10. After you have completed the configuration of the vmk's open the properties of the vSwitch in the VIC
11. Select the first vmk and select edit
12. Switch over to the NIC Teaming tab
13. Place a check mark next to 'Override vSwitch Failover Order'
14. Move all adapters except for 1 down to the Unused Adapters field
15. NOTE: Each pNIC should only have 1 vmk assigned to it; i.e vmnic0 goes with vmk0, 1 with 1 and so on (if your vmk and pNICs line up as such)
16. Open PuTTY (or a console to the ESX host)
17. In that new terminal or console session on the server do the following for each vmk that was created in step 7 and 8:


esxcli swiscsi nic add –n -d

To get the Virtual HBA ID, you can get this from the Storage Adapters page in the VIC or vCenter Client

18. To verify that they are configured right you type in:


esxcli swiscsi nic list -d


19. Go back to the VIC
20. Select the Storage Adapters item from the Hardware Box
21. Select the Software iSCSI Adapter
22. Select Properties option in the Details box
23. Select Configure and check the 'Enabled' box, Click Ok
24. If you have a password or CHAP authentication on the iSCSI Target select that button next
25. Switch to the Dynamic Discovery Tab
26. Click the Add button at the bottom of the window
27. Enter the destination or the iSCSI Target in the IP field
28. NOTE: Do steps 24 and 25 for each iSCSI Target
29. Click Close; vCenter at this time should ask if you want to rescan the HBA for new devices, and of course you will want too.
30. Once the devices are discovered you can then set the Multipathing to be Round Robin by right clicking on each device and selecting 'Manage Paths'
31. Format the RAW LUN if not already done so and begin to deploy VM's

That is it... now you have a balanced SWiSCSI traffic source that is easy to add more adapters too in teh future if you need too.