Projects/DellRedHatHALinuxCluster/Documentation/Storage/MD3000

From DellLinuxWiki

Jump to: navigation, search

Dell|Red Hat HA Linux > Storage > MD3000

Contents

Introduction

The following sections will walk-through the hardware, software, and configuration of the Dell PowerVault MD3000 Storage Array and nodes for the Dell|Red Hat HA Linux Cluster.

NOTE: Additional documentation you may find useful:

  • The Dell PowerVault MD3000 Hardware Owners Manual and the Dell PowerEdge System Hardware Owners Manual provide information about the host servers and storage array.
  • The Dell PowerVault MD3000 Systems Installation Guide provides an overview of setting up and cabling your storage array.
  • The PowerVault Modular Disk Storage Manager CLI Guide provides information about using the command line interface (CLI).
  • The Dell PowerVault MD3000 Resource media provides documentation for configuration and management tools, as well as the full documentation set included in this section.
  • The Dell PowerVault Modular Disk Storage Manager User’s Guide provides instructions for using the array management software to configure the storage array.
  • The Dell PowerVault MD Systems Support Matrix provides information on supported software and hardware for PowerVault MD systems. This document can be located on the Dell Support website at support.dell.com.

Hardware Setup

The MD3000 physically connects to each node by SAS for I/O, and by Ethernet for management

Cabling the Power Supplies

To ensure that the specific power requirements are satisfied, see the documentation for each component in your cluster solution. It is recommended that you adhere to the following guidelines to protect your cluster solution from power-related failures:

  • Plug each power supply into a separate AC circuit.
  • Plug each power supply into separate optional network power switches.
  • Use uninterruptible power supplies (UPS).
  • For some environments, consider having backup generators and power from separate electrical substations.

Cabling the Management Ports

Storage management is performed through the out-of-band Ethernet management connections on the controllers. Cable the Ethernet management ports to the private network on the same subnet as the cluster private network.

Cabling the SAS I/O Ports

Each cluster node attaches to the storage system using two SAS 5/E HBA and two SAS cables. In this configuration, there are redundant storage paths from the cluster nodes to the storage system. If a component fails in the storage path such as the port, the cable, or the storage controller, the Multi-Path Proxy driver automatically re-routes the I/O requests to the alternate path so that the storage array continues to operate.

To cable the cluster:

  1. Connect cluster node 1 to the storage system.
    • Install a SAS cable from the cluster node 1 HBA 1 port 0 to the RAID controller module 0 port In-0.
    • Install a SAS cable from the cluster node 1 HBA 2 port 1 to the RAID controller module 1 port In-0.
  2. Connect cluster node 2 to the storage system.
    • Install a SAS cable from the cluster node 2 HBA 1 port 0 to the RAID controller module 0 port In-1.
    • Install a SAS cable from the cluster node 2 HBA 2 port 1 to the RAID controller module 1 port In-1.

Figure 2-5 shows a redundant cluster configuration with dual HBAs

Figure 2-5. Redundant Cluster Configuration With Dual HBAs

Se600l-figure-2-5.jpg

Configuring the SAS HBAs

1. Turn on each node
2. During the BIOS post, use the option to invoke the serial-attached SCSI (SAS) 5/E configuration (which is typically <Ctrl><C> key combination). The following screen appears. 

Dell SAS 5 Host Bus Adapter BIOS
MPTBIOS-6.12.05.00 (2007.09.29)
Copyright 2000-2007 LSI Corporation
  
Adapter(s) disabled by user
Press Ctrl-C to run SAS Configuration Utility...

The following message appears: 

Dell SAS 5 Host Bus Adapter BIOS
MPTBIOS-6.12.05.00 (2007.09.29)
Copyright 2000-2007 LSI Corporation
  
Adapter(s) disabled by user
Please wait, invoking SAS Configuration Utility...


3. In the SAS configuration utility menu, select an adapter from the list as shown in the figure below: 

┌──────────────────────────────────────────────────────────────────────────────┐
│ LSI Corporation Config Utility  For Dell SAS 5  v6.12.05.00 (2007.09.29)     │
│ Adapter List  Global Properties                                              │
│ Adapter     PCI  PCI  PCI  FW Revision          Status    Boot               │
│             Bus  Dev  Fnc                                 Order              │
│ SAS1068     0B   08   00   0.10.51.00-IT        Disabled  0                  │
│ SAS1068     0F   08   00   0.10.51.00-IT        Disabled                     │
│                                                                              │
│                                                                              │
│ Esc = Exit Menu       F1/Shift+1 = Help                                      │
│ Alt+N = Global Properties -/+ = Alter Boot Order  Ins/Del = Alter Boot List  │
└──────────────────────────────────────────────────────────────────────────────┘

After you select an adapter, the adapter properties are displayed as shown below: 

┌──────────────────────────────────────────────────────────────────────────────┐
│ LSI Corporation Config Utility  For Dell SAS 5  v6.12.04.00 (2007.07.19)     │
│ Adapter Properties -- SAS1068                                                │
│                                                                              │
│           Adapter                        SAS1068                             │
│           PCI Address(Bus/Dev/Func)      0B:08:00                            │
│           MPT Firmware Revision          0.10.51.00-IT                       │
│           SAS Address                    500188B1:4D924100                   │
│           NVDATA Version                 25.10                               │
│           Status                         Disabled                            │
│           Boot Order                     0                                   │
│           Boot Support                   [Disabled]                          │
│                                                                              │


4. Record the SAS address. The SAS address is used to assign virtual disks later.
5. Toggle the option for Boot Support and select Disabled. This field enables you to boot from the PowerVault MD3000 storage array and is not supported with the Dell|Red Hat HA Linux Cluster.
6. Save the configuration and exit.
7. Repeat the procedure on all other SAS 5/E adapters.
8. Exit the configuration utility and reboot the system.

Software Installation

The software required for your cluster is discussed here. Some software is available via the Dell Software Repository, and all is available on the Dell PowerVault MD3000 Resource CD. For information on configuring access to the repository, see the section Installing Dell Community Repositories in the System section.

Dell PowerVault MD3000 Resource CD

All necessary software is available on the Dell PowerVault MD3000 Resource CD. If you do not have the Resource CD or access to the Dell Community Repositories, download the latest ISO image from the Dell Support website at support.dell.com.

Resource CD Pre-requisites

Before running the Dell PowerVault MD3000 Resource CD installer, ensure that the packages libXp, libXtst, and kernel development are installed on each node. These are needed for the installer.

1. Run the following commands to ensure that the packages are installed: 

[root]# yum install libXp libXtst

2. The Dell PowerVault MD3000 Resource CD requires development tools in order to install the Dell HBA drivers and Multi-Path Proxy driver. Before running the installer, ensure your node has these packages by entering the following commands: 

[root]# yum install kernel-devel gcc

3. To use the PowerVault Modular Disk Storage Manager ensure that X is running. If X is not running, execute the following commands: 

[root]# yum groupinstall base-x

If X does not start up properly, run the following configuration utility: 

[root]# system-config-display

Enter runlevel 5 with the following command: 

[root]# init 5

Or start X with the following command: 

[root]# startx

4. To view the various documentation located on the Dell PowerVault MD3000 Resource CD install a web browser on your system: 

[root]# yum install firefox

Launching the Resource CD Main Menu

To install the required software:

1. Access the Dell PowerVault MD3000 Resource CD in one of two ways:

  • Insert the media into a cluster node. If X is running, the media mounts automatically under the /media mount point and launches the main menu. If the media does not mount automatically, execute the following command: 
[root]# mount /dev/cdrom {/path/to/mount/point}

where {/path/to/mount/point} is a directory that is used to access the contents.

  • If you have downloaded the ISO image, it is not necessary to create physical media. Mount the ISO image and access the contents with the following command: 
[root]# mount -o loop {ISO filename} {/path/to/mount_point}

where {/path/to/mount_point} is a directory that will be used to access the contents.

2. The installer launches automatically. If it did not, run the Dell PowerVault MD3000 Resource CD Linux installation script manually: 

[root]# ./autorun

If the main menu does not appear, change the directory using the following command: 

[root]# cd {/path/to/mount/point/linux/}

Run the installer: 

[root]# ./install.sh

The Dell PowerVault MD3000 Resource CD menu is displayed. 

#################################################
Dell PowerVault MD3000 Resource CD
#################################################
1. View MD3000 Readme
2. Install SAS 5/E Adapter Driver
3. Install MD3000 Storage Manager
4. Install Multi-Pathing driver
5. Install MD3000 Documentation
6. View MD3000 Documentation
7. Dell Support
8. View End User License Agreement
Enter the number to select a component from the
above list.
Enter q to quit.
Enter:

Installing the PowerVault Modular Disk Storage Manager and Host Utilities

NOTE: Perform the following steps on any management or cluster nodes.

The PowerVault Modular Disk Storage Manager contains the necessary management software and host agent utilities. To launch the installer:

1. Complete the steps in Dell PowerVault MD3000 Resource CD.
2. Select Option 3. Install MD3000 Storage Manager.
3. The Java-based installer appears.
NOTE: If you encounter any errors or do not see the Java™ installer, ensure that you have completed the steps in Dell PowerVault MD3000 Resource CD. For more information, go to the main menu and view documentation.
4. Follow the instructions on the screen. To continue with installation, accept the license terms.
5. Select an installation location. The default value is /opt/dell/mdstoragemanager.
6. The Select an Installation Type window appears and displays the following options. Choose the option relevant to your node's function:

Install Option Node Type
Typical (Full) Select this for a node that will be both a management and cluster node
Management Station Select this option for a dedicated management node
Host Select this option for a cluster node


NOTE: If you choose Typical (Full) or Host, the Multi-Path Proxy driver warning is displayed. It is safe to ignore this warning.

RDAC Multi-Path Proxy Driver

Each cluster node has multiple paths to the same virtual disk via the SAS cables. This may cause issues as the cluster nodes are able to access the same data through different paths that could lead to data corruption. Multi-Path Proxy driver consolidates all paths to the virtual disk into a pseudo device. This allows the nodes to access the pseudo device instead of a path directly. If a path failure occurs, the Multi-Path Proxy driver automatically switches paths, and the node continues to access the same data through the same pseudo device.

Red Hat Enterprise Linux 5.1 Advanced Platform includes a multipath driver as part of the base operating system with the package device-mapper-multipath. However, Dell provides Multi-Path Proxy driver that is specific to the PowerVault MD3000 storage array. It is recommended that you use the Dell-specific RDAC Multi-Path Proxy (MPP) driver with the Dell|Red Hat HA Linux Cluster.


NOTE: For more information on device-mapper-multipath, see Using Device-Mapper Multipath located on the Red Hat website at www.redhat.com/docs/manuals/enterprise/.

Installing the Multi-Path Proxy Driver via MD3000 Resource CD

To install the RDAC MPP driver:

  1. Complete the steps in Dell PowerVault MD3000 Resource CD.
  2. Select 4. Install Multi-pathing Driver
  3. Follow the steps on the screen to complete installation. After completion, The follow message appears: 
DKMS: install Completed
You must restart your computer for the new settings to take effect.

For additional information about installing RDAC MPP driver, see the RDACreadme.txt on the Dell PowerVault MD3000 Resource media.

Installing the Multi-Path Proxy Driver via Dell Repository

Ensure you have configured your systems to access the Dell Software Repository. For information on configuring access to the repository, see the section Installing Dell Community Repositories in the System section.

Install the Multi-Path Proxy driver: 

[root]# yum install linuxrdac

Installing the SAS 5/E HBA Drivers

NOTE: The SAS 5/E driver is provided in the form of a Dynamic Kernel Module Support (DKMS) package. For more information on DKMS, visit the project page at: www.linux.dell.com/dkms
NOTE: Perform the following steps on your cluster nodes only. Do not install the SAS HBA driver on any dedicated management node.

Installing the SAS 5/E HBA Drivers via MD3000 Resource CD

  1. Complete the steps in "Launching the Resource CD Main Menu"
  2. Select the option 2. Install SAS 5/E Adapter Driver. Follow the instructions

on the screen to complete installation. The following message appears: 

DKMS: install Completed
You must restart your computer for the new settings to take effect.

For more information view documentation on the main menu.

Installing the SAS 5/E HBA Drivers via Dell Repository

Ensure you have configured your systems to access the Dell Software Repository. For information on configuring access to the repository, see the section Installing Dell Community Repositories in the System section. Install the SAS 5/E driver: 

[root]# yum install mptlinux

Storage Configuration

Connect your SAS cables to your nodes that were disconnected during the operating system installation in the System section.

Initially Configuring the Shared Storage System

The following steps only need to be performed during the initial setup of the storage array. For more information on all procedures below, consult the Dell PowerVault Modular Disk Storage Manager Users's Guide.

Discovering your Storage Array

NOTE: Prior to this step, ensure that the Storage Array management ports are connected to the private network. Refer to Cabling the Management Ports for more details.

  1. On a management node, open the PowerVault Modular Disk Storage Manager application. For more information on installing this software, refer to Installing the PowerVault Modular Disk Storage Manager.
  2. Click New at the top of the Manager window. This will launch the Add New Storage Array wizard.
  3. Choose Automatic. This option will scan your network for MD3000 Storage Arrays.
  4. When the scan is complete, your MD3000 Storage Array should be visible. If it is not, check all physical connections to ensure that the management node and the MD3000 management ports are on the same private network.

NOTE: If this process fails, consult the section Adding Storage Arrays in the Dell PowerVault Modular Disk Storage Manager Users's Guide.

Initial Setup Tasks

The Initial Setup Tasks window will open when the storage array is first disovered. If it does not open, you can click the Initial Setup Tasks link at the top of the MD Storage Manager window. This window contains links to the basic commands needed to configure your Storage Array. These steps can also be completed manually at any time. Follow these steps to complete the basic configuration of your Storage Array. For more information on any of these steps, see the section Setting Up Your Storage Array in the Dell PowerVault Modular Disk Storage Manager Users's Guide.

Perform the following steps:

  • Rename the Storage Array
  • Set a Storate Array Password
  • Set up Alert Notifications
  • Configure Host Access (automatic)
    • If the MD Agent is running properly on the nodes, you can configure host access from this initial setup task wizard. However, if no host initiators are displayed, skip this step for now. You will need to perform Manual host access setup later.
  • Configure Storage Array
    • Hot Spare assignment
    • Virtual Disks and Disk Groups
      • In most cases one large Virtual Disk is adequate. However, create as many Virtual Disks as required by your application.
  • Configure Ethernet Management Ports
    • This is the last task under the Optional heading
    • Assign IP addresses on the private network.

Updating the Firmware

For the latest PowerVault MD3000 firmware, see the Dell Support website at support.dell.com. Use the PowerVault Modular Disk Storage Manager to perform the firmware update. See the section Installing the PowerVault Modular Disk Storage Manager and the documentation that came with your storage array for more information.

Completing Shared Storage System Configuration

For more information on all procedures below, consult the section About Your Hosts in the Dell PowerVault Modular Disk Storage Manager Users's Guide.

Configuring Host Access

  • Create Host Group
  • Associate Host SAS address with Host Group
    • Use the SAS Address recorded in the step Configuring the SAS HBAs to match SAS HBA to node
    • Ensure all cluster nodes are part of the same Host Group
  • Associate Virtual Disk with Host Group
    • Ensure all Virtual Disks to be accessed by the cluster are part of the same Host Group

Verification Checklist

Item Verified
Physical setup
Physical cabling
SAS HBA driver (mptlinux package)
Multi-Path Proxy driver (linuxrdac package)
Dell™ PowerVault™ MD3000 storage array configured
PowerVault MD3000 Storage Manager installed on a node
Virtual Disk created
Host Groups defined and mapped to Virtual Disks
Retrieved from "http://linux.dell.com/wiki/index.php/Projects/DellRedHatHALinuxCluster/Documentation/Storage/MD3000"
Personal tools
Distributions