Readme for Dell Roll

Version 4.1.1-2.0

October 25 2006

Platform Computing

Contents

[ Top ]

About the Dell Roll

The Dell roll provides drivers and utilities for DellTM PowerEdge servers. It provides support for IPMI and Dell OpenManageTM (OM).

The Dell roll also contains a script that can be used to configure the BMC and BIOS of the cluster nodes during deployment. For systems that are not supported by OM, such as the PE SC 1435, only the BMC can be configured using the Linux ipmitool utility. For systems that are supported by OM, such as the PE1950, 2950, 1955, and 6950, the script allows configuration of the system BMC and BIOS using the omconfig utility provided by OM. Once the system is configured, features such as remote power cycling of servers and console redirection can be used. The script must be enabled to execute on the compute nodes during deployment.

[ Top ]

Setting up BMC and BIOS using a Script in the Dell Roll

Detailed below are instructions for using a script to configure the BMC and BIOS of the cluster nodes. For systems that are not supported by Dell OpenManage (OM), such as the PE SC 1435 you can only configure the BMC by using the Linux ipmitool utility. For systems that are supported by Dell OpenManage, the script allows you to configure the system BMC and BIOS using the omconfig utility provided by OM. Once the system is configured, you can use features such as remote power cycling of machines and console redirection. The script can be set to execute on your compute nodes to configure the BMC (Baseboard Management Controller) and BIOS automatically during deployment.

Script file

dell-bmcbios-setup.xml

This script can be used to configure the BMC and BIOS for PE 1950, PE 2950, PE 1955, and PE 6950. PE SC1435 does not support Dell OpenManage (OM) and the script can only be used to configure the BMC.

What does it do?

On systems supported by Dell OpenManage (PE1950, PE1955, PE2950, and PE6950), for each node, as part of the post-install procedure, this script

  1. Copies the OM tar and extracts it in /opt/omsa
  2. Installs OM in /opt/dell/
  3. Uses OM's omconfig utility to reset the BMC to defaults
  4. Uses OM's omconfig utility to configure the BMC of the node
  5. Uses OM's omconfig utility to configure the BIOS of the node for console redirection (CR)
    1. PE1950, PE2950 and PE6950: CR is setup through the BMC
    2. PE1955: CR is setup through the DRAC/MC
  6. Turns off OM services for all init levels using chkconfig utility

This script configures the BMC. It also configures the BIOS for console redirection. Operating system files are not edited by default. Console redirection at boot-up and once the machine has booted (login prompt) is disabled by default. To enable this and other options, please read the following sections.

On systems not supported by Dell OpenManage (PE SC1435), for each node, as part of the post-install procedure, this script uses the Linux ipmitool utility to configure the BMC of the node.

ipmi shell

To install ipmi shell (ipmish) on the management node, copy and install the OSA BMC Utility RPM (osabmcutil) manually using rpm -ivh. This RPM can be obtained from the OM CD or from the support.dell.com Web site:

Download ipmi shell from http://support.dell.com/support/index.aspx?c=us&cs=555&l=en&s=biz&~ck= mn. Browse to Drivers & Downloads for your specific server with Enterprise Linux 4. The BMU rpm that contains ipmish can be found by navigating to Systems Management and clicking on the Dell Linux Remote Access Utilities link.

Note

The osabmcutil RPM provides a solproxy (Serial-Over-LAN) that uses port 623.

To install this RPM on a cluster node such as the front end node, run: 

% rpm -ivh <osa_rpm_package>
Before using the script, complete the following steps
  1. If you do not have the OM CD (or do not wish to use it), follow these steps to download and extract the OM package:
    1. Download the OM file from http://support.dell.com/support/index.aspx?c=us&cs=555&l=en&s=biz&~ck=mn by browsing to Drivers & Downloads for your specific server with Enterprise Linux 4.

      You can find the software package by navigating to Systems Management and clicking on the Dell OpenManage Server Administrator Managed Node link.

    2. Extract the OM package from the file. 
      # tar -zxvf <om_software_package>
  2. From the appropriate OM CD or from the OM software package that you downloaded from the link given above, browse to the parent of the linux directory on the CD or tar file.
  3. Create the new om.tar file of everything under and including the Linux directory. 
    # tar -cf /tmp/om.tar linux/
  4. For systems that are supported by Dell OpenManage, place the new om.tar file in the appropriate path on the front end. 
    # cp /tmp/om.tar /export/home/install/contrib/4.1.1/x86_64/RPMS/
  5. Connect the dell-bmcbios-setup.xml to the client.xml node. 
    # cd /export/home/install
# cp rocks-dist/lan/x86_64/build/graphs/default/dell.xml \
site-profiles/4.1.1/graphs/default/
    Edit site-profiles/4.1.1/graphs/default/dell.xml and uncomment the three sections in bold: 
      <edge from="client">
        <to>dell-base</to>
        <to>dell-compute</to>
        <to>dell-misc</to>
        <!-- uncomment the following for BIOS/BMC configuration -->

        <!--
        <to>dell-bmcbios-setup</to>
        <to>dell-grub</to>
        -->
</edge>

<edge from="grub-server">
        <to>dell-misc</to>

</edge>

<!-- uncomment the following for BIOS/BMC configuration -->
<!--
<order head="snmp-server">
        <tail>dell-bmcbios-setup</tail>
</order>
-->

<order head="grub-client">
        <tail>dell-misc</tail>
</order>

<!-- uncomment the following for BIOS/BMC configuration 
<edge from="dell-misc">
        <to>dell-grub</to>
</edge>

-->
  6. (Optional) This section lists the script's default BMC and BIOS settings and other parameters. To change any of the defaults, follow the instructions below.

    The following tables summarize the configuration parameters for the BMC and BIOS, along with the script's default values for each parameter. Each of these parameters is set in the dell-bmcbios-setup.xml file.

    The first table contains configuration parameters for the BMC that are used for all server models. The second table contains configuration parameters that are used only for servers that are supported by Dell OpenManage. Note these parameters may not be supported on all server types. For example, the PE6950 does not have a parameter for HyperThreading.

    Common configuration parameters for all servers

    Configuration Parameter Default Value
    BMC gateway
    		20.255.1.1
    BMC netmask
    		255.255.0.0
    BMC IP address
    		The BMC IP address will be generated by the script according to the BMC gateway,
    BMC netmask and the nodes PXE installation IP: By default the BMC IP will be
    20.255.x.x where x.x are the last two octets of the node's PXE installation IP.
    For example, node 10.255.255.254 will have a BMC IP of 20.255.255.254.

    Configuration parameters for servers supported by Dell OpenManage

    Configuration Parameter Default Value
    BMC LAN alert
    		Not set
    OM Services on reboot
    		Disabled
    Root login via Console Redirection
    		Disabled
    Redirection of kernel and init messages at boot time to allow the login console over the serial port.
    		Disabled
    HyperThreading
    		Ignore
    Demand Based Power Management (DBS)
    		Ignore

    If you want to change any of these default values, you must first copy dell-bmcbios-setup.xml file to the site-profiles directory: 

    # cd /export/home/install
# cp 
rocks-dist/lan/x86_64/build/nodes/dell-bmcbios-setup.xml site-profiles/4.1.1/no
des

    For each configuration parameter you want to change, follow the appropriate set of instructions below and make the appropriate changes in the "Configuration values" section at the beginning of the dell-bmcbios-setup file.

    1. (Optional) Edit the xml file with BMC netmask, and BMC gateway.

      To change the defaults, edit /export/home/install/site-profiles/4.1.1/nodes/dell-bmc bios-setup.xml

      by making the following changes: 

      <var name="bmc_gw" val="new_BMC_gateway"/>
<var name="bmc_mask" val="netmask_for_BMC"/>
    2. (Optional) Add the BMC LAN Alert destination.

      By default, the script does not set this variable. To set it, edit /export/home/install/site-profiles/4.1.1/nodes/dell-bmc bios-setup.xml

      and make the following change: 

      <var name="bmc_lan_alert_dest" val="management_node_IP_address"/>

      where managment_node_IP_address is the IP address of the node to which the IPMI Platform Event Traps should be directed. This node is typically the Dell ITA (IT Assistant) management node in the cluster.

    3. (Optional) Turn Chkconfig OM services ON.

      By default, the script chkconfigs all OM services OFF for all init levels.

      To leave the services ON, set disable_om_services to no in /export/home/install/site-profiles/4.1.1/nodes/dell-bmc bios-setup.xml

      For example 

      <var name="disable_om_services" val="no"/>
    4. (Optional) Allow root to login by CR.

      Default settings do not allow root login by CR. If this is desired, set root_console_access to yes in /export/home/install/site-profiles/4.1.1/nodes/dell-bmc bios-setup.xml

      For example 

      <var name="root_console_access" val="yes"/>
    5. (Optional) Redirect kernel and init messages at startup (edits to grub.conf). Enabling login console over serial port (edits to /etc/inittab).

      By default, the script does not edit grub.conf for redirection of grub messages at startup or edit the /etc/inittab file to enable a login console through redirection. To redirect grub and enable a login console over serial port, edit /export/home/install/site-profiles/4.1.1/nodes/dell-bmc bios-setup.xml

      by making the following change: 

      <var name="enable_os_console_redirect" val="yes"/>

      Note that by setting this variable to "yes", the script configures init messages at boot to appear only on the serial console and not to the terminal.

    6. (Optional) Enable or disable HyperThreading in the BIOS.

      By default, the script does not modify the hyperthreading setting in the system BIOS.

      To disable hyperthreading, set enable_ht to no in /export/home/install/site-profiles/4.1.1/nodes/dell-bmc bios-setup.xml

      For example: 

      <var name="enable_ht" val="no"/>

      To enable hyperthreading, set enable_ht to yes in /export/home/install/site-profiles/4.1.1/nodes/dell-bmc bios-setup.xml

      For example 

      <var name="enable_ht" val="yes"/>
    7. (Optional) To enable or disable Demand Based Power Management (DBS) in the BIOS.

      By default, the script does not modify the DBS setting in the system BIOS.

      To disable DBS, set enable_dbs to no in /export/home/install/site-profiles/4.1.1/nodes/dell-bmc bios-setup.xml

      For example: 

      <var name="enable_dbs" val="no"/>

      To enable DBS, set enable_dbs to yes in /export/home/install/site-profiles/4.1.1/nodes/dell-bmc bios-setup.xml

      For example 

      <var name="enable_dbs" val="yes"/>
  7. Rebuild the Platform OCS distribution. 
    # cd /export/home/install
# rocks-dist dist
  8. Install your node.

[ Top ]

Notes

  1. Dell OpenManage documentation is available from the following:

    http://support.dell.com/support/edocs/software/smsom/.

    Dell's BMC documentation is available from the following:

    http://support.dell.com/support/edocs/software/smbmcmu/.

    Dell's DRAC/MC documentation is available from the following:

    http://support.dell.com/support/edocs/software/smdrac3/dracmc/index.h tm.

  2. To use ipmi shell (ipmish) from a management station to talk to a managed node's BMC, the OS-level IP of the management station should be on the same subnet as the BMC IP.

    For example:

    • Managed node: BMC 20.255.255.255, netmask 255.255.0.0.
    • The management station should have an IP on the 20.255.0.0 subnet on the interface that is connected to the managed node's BMC.
    • On a Linux management station, ifconfig eth0:0 20.255.1.1/16 (assuming eth0 (NIC1) is connected to the managed node's BMC).
  3. On servers with DRAC5 (Dell Remote Access Controller) cards, ensure that the DRAC5 port is cabled to the switch. With a DRAC5 present, by default all IPMI traffic to the BMC flows through the DRAC5 port. If there is no DRAC5 card, the BMC is contacted through the LOM1 (NIC1) interface.

    This is because the default NIC Selection option with a DRAC5 card present is Dedicated. In this mode, the BMC and DRAC can be contacted only through the cable connected to the DRAC. To contact the BMC over LOM1 even with a DRAC5 card present, the NIC Selection option should be changed to Shared.

    If there is no DRAC5 card in the server, the default NIC Selection option is Shared and the BMC can be contacted over LOM1 (NIC1).

    To change the default NIC Selection, follow these steps:

    1. Manually enter the BMC menu at boot-up by pressing Ctrl+E and change the NIC Selection option as needed.
    2. Use the Linux omconfig utility provided by Dell OpenManage to run the following command (requires OM services to be running on the server): 
      % omconfig chassis remoteaccess config=nic 
nicselection=shared
  4. OM also provides a utility called omreport which can be used to read the current configuration of the BMC and BIOS. OM services must be running to use this utility. 
    /opt/omsa/linux/supportscripts/srvadmin-services.sh 
status|start|stop

    For usage information: omreport -?

  5. omconfig allows system- and chassis-level configuration. For more information use: omconfig -?. Other system and chassis configuration commands can be added to this script or issued using the cluster-fork command. OM services must be running to use this utility. 
    /opt/omsa/linux/supportscripts/srvadmin-services.sh 
status|start|stop
  6. Once OS Console Redirection (the redirection of kernel and init messages at boot time to allow console login over the serial port) is configured, compute node reinstallation initiated by commands:

    shoot-node or /boot/kickstart/cluster-kickstart will not display any console messages after "VFS: Mounted root (ext2 filesystem)". All the rest of the installation messages will be redirected over the serial port.

[ Top ]

Learn About Platform Products

World Wide Web

The latest information about all supported releases of any Platform product is available on the Platform Web site at http://www.platform.com. Forums, e-support, documentation and release information is available at http://my.platform.com/.

If you have problems accessing the Platform Web site or the Platform FTP site, contact support@platform.com.

Platform training

Platform's Professional Services training courses can help you gain the skills necessary to effectively install, configure and manage your Platform products. Courses are available for both new and experienced users and administrators at our corporate headquarters and Platform locations worldwide. Customized on-site course delivery is also available.

Find out more about Platform Training at http://www.platform.com/Services/Training, or contact trng@platform.com for details.

Technical support

Contact Platform Computing at support@platform.com for technical support. When contacting Platform, please include the full name of your company.

See the Platform Web site at http://www.platform.com/Company/Contact.Us.htm for other contact information.

Platform documentation

Roll readmes are packaged with the rolls, and are accessible from the Roll Call page in your front end node: http://localhost/homepage/list-rolls.cgi.

[ Top ]

Get Technical Support

Contact Platform

Contact Platform Computing or your Platform OCS vendor for technical support. Use one of the following to contact Platform technical support:

Email

support@platform.com

World Wide Web

http://www.platform.com

Mail

Platform Support
Platform Computing Corporation
3760 14th Avenue
Markham, Ontario
Canada L3R 3T7

When contacting Platform, please include the full name of your company.

See the Platform Web site at http://www.platform.com/Company/Contact.Us.htm for other contact information.

Get patch updates and hotfixes

Obtain the latest patches and hotfixes for Platform OCS from the following page:

http://my.platform.com/products/platform-ocs

To obtain a user name and password, contact Platform Computing technical support at support@platform.com

[ Top ]

Copyright and Trademarks

© 1994-2006 Platform Computing Corporation. All Rights Reserved.

Although the information in this document has been reviewed, Platform Computing Corporation ("Platform") does not warrant it to be free of errors or omissions. Platform reserves the right to make corrections, updates, revisions or changes to the information in this document.

UNLESS OTHERWISE EXPRESSLY STATED BY PLATFORM, THE PROGRAM DESCRIBED IN THIS DOCUMENT IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT WILL PLATFORM COMPUTING BE LIABLE TO ANYONE FOR SPECIAL, COLLATERAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING WITHOUT LIMITATION ANY LOST PROFITS, DATA, OR SAVINGS, ARISING OUT OF THE USE OF OR INABILITY TO USE THIS PROGRAM.

Trademarks

This product includes software developed by the Rocks Cluster Group at the San Diego Supercomputer Center at the University of California, San Diego and its contributors.

® LSF and LSF HPC are trademarks or registered trademark of Platform Computing Corporation in the United States and in other jurisdictions.

TM ACCELERATING INTELLIGENCE, PLATFORM COMPUTING, and the PLATFORM and Platform OCS logos are trademarks of Platform Computing Corporation in the United States and in other jurisdictions.

®TM Dell and Dell OpenManage are service marks or registered trademarks of Dell Inc. in the United States and in other countries.

Other products or services mentioned in this document are identified by the trademarks or service marks of their respective owners.

[ Top ]

      Date Modified: July 06, 2006
Platform Computing: www.platform.com

Platform Support: support@platform.com
Platform Information Development: doc@platform.com

© 1994-2006 Platform Computing Corporation. All rights reserved.