Skip Headers
Oracle® Grid Infrastructure Installation Guide
11g Release 2 (11.2) for Microsoft Windows

Part Number E10817-01
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
View PDF

2 Advanced Installation Oracle Grid Infrastructure for a Cluster Preinstallation Tasks

This chapter describes the system configuration tasks that you must complete before you start Oracle Universal Installer (OUI) to install Oracle grid infrastructure.

This chapter contains the following topics:

2.1 Installation Differences Between Windows and Linux or UNIX

If you are experienced with installing Oracle components in Linux or UNIX environments, then note that many manual setup tasks required on Linux or UNIX are not required on Windows. The key differences between Windows and Linux or UNIX and installations are:

See Also:

"Oracle Database Windows/UNIX Differences," in Oracle Database Platform Guide for Microsoft Windows

2.2 Reviewing Upgrade Best Practices

Caution:

Always create a backup of existing databases before starting any configuration change.

If you have an existing Oracle installation, then record your version numbers, patches, and other configuration information. Before proceeding with installation of Oracle Grid infrastructure, review the Oracle upgrade documentation to decide the best method of upgrading your current software installation.

Note:

If you want to upgrade Oracle Clusterware Release 10.2 to Oracle Clusterware Release 11g, then you must first apply the 10.2.0.3 or later patchset.

You can upgrade a clustered Oracle ASM installation without shutting down an Oracle RAC database by performing a rolling upgrade either of individual nodes, or of a set of nodes in the cluster. However, if you have a standalone database on a cluster that uses Oracle ASM, then you must shut down the standalone database before upgrading.

If you have an existing standalone, or non-clustered, Oracle Automatic Storage Management (Oracle ASM) installation, then review Oracle upgrade documentation. The location of the Oracle ASM home changes in this release, and you may want to consider other configuration changes to simplify or customize storage administration.

During rolling upgrades of the operating system, Oracle supports using different operating system binaries when both versions of the operating system are certified with the Oracle Database release you are using.

Note:

Using mixed operating system versions is only supported for the duration of an upgrade, over the period of a few hours. Oracle does not support operating a cluster with mixed operating systems for an extended period. Oracle does not support running Oracle grid infrastructure and Oracle Real Application Clusters on heterogeneous platforms (servers with different chip architectures) in the same cluster.

To find the most recent software updates, and to find best practices recommendations about preupgrade, postupgrade, compatibility, and interoperability, refer to "Oracle Upgrade Companion." "Oracle Upgrade Companion" is available through Note 785351.1 on My Oracle Support:

https://support.oracle.com

2.3 Checking Hardware and Software Certification

The following sections list the following certification information:

2.3.1 View Certification Information at My Oracle Support

The hardware and software requirements included in this installation guide were current at the time this guide was published. However, because new platforms and operating system software versions might be certified after this guide is published, review the certification matrix on the My Oracle Support Web site for the most up-to-date list of certified hardware platforms and operating system versions. This Web site also provides compatible client and database versions, patches, and workaround information for bugs.

The My Oracle Support Web site is available at the following URL:

http://support.oracle.com/

You must register online before using My Oracle Support. After logging in, click the More... tab then select Certifications. In the Find Certification Information field, choose the following:

  • Product Line: Oracle Database Products

  • Product Family: Oracle Database

  • Product Area: Oracle Database

  • Product: Oracle Server - Enterprise Edition

  • Product Release: 11gR2 RAC

  • Product Version: 11gR2 RAC

  • Platform: Microsoft Windows x64 (64-bit)

After you have made these selections, click Search. Click the Certified link next to the value of Platform Version that matches your operating system, for example, 2008 R2.

Click the link for Certification notes to check the Certification Matrix for Oracle RAC to ensure that your hardware configuration is supported for use with Oracle Clusterware and Oracle RAC. My Oracle Support contains guidance about supported hardware options that can assist you with your purchasing decisions and installation planning.

In addition to specific certified hardware configurations, the Certify page provides support and patch information, and general advice about how to proceed with an Oracle Clusterware with Oracle RAC 11g release 2 (11.2) installation, including important information about configuration issues. View the Product Notes and Platform Notes to view this additional information.

Note:

Contact your Oracle sales representative if you do not have a My Oracle Support account.

2.3.2 Web Browser Support

On 64-bit Windows systems, Microsoft Internet Explorer 6.0 and higher is supported for Oracle Enterprise Manager Database Control and Oracle Enterprise Manager Grid Control.

2.3.3 Windows Telnet Services Support

Windows Server 2003 and Windows Server 2003 R2 can use a Telnet Service to enable remote users to log on to the operating system and run console programs using the command line. Oracle supports the use of database command line utilities such as sqlplus, export, import and sqlldr using this feature, but does not support the database GUI tools such as Oracle Universal Installer, Database Configuration Assistant, and Oracle Net Configuration Assistant.

Note:

Ensure that the Telnet service is installed and started.

2.4 Checking the Hardware Requirements

Each system must meet the following minimum hardware requirements:

2.5 Checking the Disk Space Requirements

The requirements for disk space on your server are described in the following sections:

2.5.1 Disk Format Requirements

Oracle recommends that you install Oracle software, or binaries, on NTFS formatted drives or partitions. Because it is difficult for OUI to estimate NTFS and FAT disk sizes on Windows, the system requirements documented in this section are likely more accurate than the values reported on the Oracle Universal Installer Summary screen.

Note:

Oracle Grid Infrastructure software is not supported on NFS.

You cannot use NTFS formatted disks or partitions for Oracle Clusterware files or data files because they cannot be shared. Oracle Clusterware shared files and Oracle Database data files can be placed on unformatted (raw) basic disks that are managed by Oracle ASM or Oracle Cluster File System (OCFS) for Windows.

Oracle ASM is recommended for managing Oracle Clusterware and Oracle Database data files.

2.5.2 Disk Space Requirements for Oracle Home Directories

3 GB of disk space for the grid infrastructure home (Grid home) The Grid home includes Oracle Clusterware and Oracle Automatic Storage Management (Oracle ASM) software, configuration, and log files. Additional disk space on a cluster file system or shared disks is required for the Oracle cluster registry (OCR) and voting files used by Oracle Clusterware.

To determine the amount of free disk space, open My Computer, right-click the drive where the Oracle software is to be installed, and choose Properties.

If you are installing Oracle Database, then you must configuration additional disk space for:

  • The Oracle Database software and log files

  • The shared data files and, optionally, the shared Fast Recovery Area on either a file system or in an Oracle Automatic Storage Management disk group

2.5.3 TEMP Disk Space Requirements

The amount of disk space available in the TEMP directory is equivalent to the total amount of free disk space, minus what will be needed for the Oracle software to be installed.

You must have 1 GB of disk space available in the TEMP directory. If you do not have sufficient space, then first delete all unnecessary files. If the temp disk space is still less than the required amount, then set the TEMP environment variable to point to a different hard drive.

To modify the TEMP environment variable open the System control panel, select the Advanced tab, and click Environment Variables.

Note:

The temporary directory must reside in the same directory path on each node in the cluster.

2.6 Checking the Network Requirements

Review the following sections to check that you have the networking hardware and internet protocol (IP) addresses required for an Oracle grid infrastructure for a cluster installation:

Note:

For the most up-to-date information about supported network protocols and hardware for Oracle RAC installations, refer to the Certify pages on the My Oracle Support Web site at the following URL:
https://support.oracle.com

2.6.1 Network Hardware Requirements

The following is a list of requirements for network configuration:

  • The host name of each node must use only the characters a-z, A-Z, 0-9, and the dash or minus sign (-). Host names using underscores (_) are not allowed.

  • Each node must have at least two network adapters or network interface cards (NICs): one for the public network interface, and one for the private network interface (the interconnect).

    To use multiple NICs for the public network or for the private network, Oracle recommends that you use NIC teaming. Use separate teaming for the public and private networks, because during installation each network connection is defined as a public or private interface.

  • The private and public network connection names must be different from each other and cannot contain any multibyte language characters. The names are case-sensitive.

  • The public network connection names associated with the network adapters for each network must be the same on all nodes, and the private network connection names associated with the network adapters must be the same on all nodes.

    For example: With a two-node cluster, you cannot configure network adapters on node1 with NIC1 as the public network connection name, but on node2 have NIC2 as the public network connection name. Public network connection names must be the same, so you must configure NIC1 as public on both nodes. You must also configure the network adapters for the private network connection with the same network connection name. If PrivNIC is the private network connection name for node1, then PrivNIC must be the private network connection name for node2.

  • In Windows Networking Properties, the public network connection on each node must be listed first in the bind order (the order in which network services access the node). The private network connection should be listed second.

    To ensure that your public adapter is first in the bind order, follow these steps:

    1. Right-click My Network Places and choose Properties.

    2. In the Advanced menu, click Advanced Settings.

    3. If the public adapter name is not the first name listed under the Adapters and Bindings tab, then select it and click the arrow to move it to the top of list

    4. Click OK to save the setting and then exit network setup dialog

  • For the public and private networks, each network adapter must support TCP/IP.

  • For the private network, the interconnect must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP (minimum requirement 1 Gigabit Ethernet).

    Note:

    UDP is the default interconnect protocol for Oracle RAC, and TCP is the interconnect protocol for Oracle Clusterware. You must use a switch for the interconnect. Oracle recommends that you use a dedicated switch.

    Oracle does not support token-rings or crossover cables for the interconnect.

  • Windows Media Sensing must be disabled for the private network connection.

    To disable Windows Media Sensing for TCP/IP, you must set the value of the DisableDHCPMediaSense parameter to 1 on each node. Because you need to modify the Windows registry to disable Media Sensing, you should first backup the registry and confirm that you can restore it, using the methods described in your Windows documentation. Disable Media Sensing by completing the following steps on each node of your cluster:

    1. Backup the Windows registry.

    2. Use Registry Editor (Regedt32.exe) to view the following key in the registry:

      HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters
      
    3. Add the following registry value of type DWORD:

      Value Name: DisableDHCPMediaSense
      Data Type: REG_DWORD -Boolean
      Value: 1
      
    4. Restart the computer.

  • For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. There should be no node that is not connected to every private network interface. You can test if an interconnect interface is reachable using ping.

  • During installation, you are asked to identify the planned use for each network connection name that OUI detects on your cluster node. You must identify each network connection name as a public or private network connection name, and you must use the same private network connection names for both Oracle Clusterware and Oracle RAC.

    You can team separate interfaces to a common network connection to provide redundancy in case of a NIC failure. Oracle RAC and Oracle Clusterware will share this connection.

    IP addresses on the subnet you identify as private are assigned as private IP addresses for cluster member nodes. You do not need to configure these addresses manually in a hosts file.

2.6.2 IP Address Requirements

Before starting the installation, you must have at least two network interfaces configured on each node: One for the private IP address and one for the public IP address.

You can configure IP addresses with one of the following options:

  • Oracle Grid Naming Service (GNS) using a static public node address and dynamically allocated IPs for the Oracle Clusterware provided VIP addresses, DHCP server assigned, and resolved using a multicast domain name server configured within the cluster.

  • Static addresses that network administrators assign on a network domain name server (DNS) for each node. Selecting this option requires that you request network administration updates when you modify the cluster.

Note:

Oracle recommends that you use a static hostname for all server node public hostnames.

Public IP addresses and virtual IP addresses must be in the same subnet.

2.6.2.1 IP Address Requirements with Grid Naming Service

If you enable Grid Naming Service (GNS), then name resolution requests are delegated to the GNS service through its virtual IP address. You define this address in the DNS domain before installation. The DNS delegates name resolution requests for cluster names to the GNS. The GNS processes the requests and responds with the list of addresses for the names.

To use GNS, before installation the DNS administrator must establish DNS Lookup to direct DNS resolution of a subdomain to the cluster.

See Also:

"DNS Configuration for Domain Delegation to Grid Naming Service" for information on how to configure DNS delegation

2.6.2.2 IP Address Requirements for Manual Configuration

If you do not enable GNS, then the public and virtual IP addresses for each node must be static IP addresses, configured before installation for each node, but not currently in use. Public and virtual IP addresses must be on the same subnet.

Oracle Clusterware manages private IP addresses in the private subnet on network interfaces you identify as private during the installation interview.

The cluster must have the following addresses configured:

  • A public IP address for each node

  • A virtual IP address for each node

  • A single client access name (SCAN) configured on the domain name server (DNS) for Round Robin resolution to three addresses (recommended) or at least one address.

The single client access name (SCAN) is a host name used to provide service access for clients to the cluster. Because the SCAN is associated with the cluster as a whole, rather than to a particular node, the SCAN makes it possible to add or remove nodes from the cluster without needing to reconfigure clients. It also adds location independence for the databases, so that client configuration does not have to depend on which nodes are running a particular database. Clients can continue to access the cluster in the same way as with previous releases, but Oracle recommends that clients access the cluster using the SCAN.

SCAN addresses should be defined in a domain name service (DNS) to resolve to the SCAN. The SCAN addresses must be on the same subnet as virtual IP addresses and public IP addresses. The SCAN must resolve to at least one address. For high availability and scalability, Oracle recommends that you configure the SCAN to use Round Robin resolution to three addresses. The name for the SCAN cannot begin with a numeral.

Note:

If you manually configure SCAN VIP addresses, then Oracle strongly recommends that you do not configure SCAN VIP addresses in the system hosts file. Use DNS resolution for SCAN VIPs. If you use the system hosts file to resolve SCANs, then you will only be able to resolve to one IP address and you will have only one SCAN address.

See Also:

Appendix C, "Understanding Network Addresses"for more information about network addresses

2.6.3 DNS Configuration for Domain Delegation to Grid Naming Service

If you plan to use GNS, then before grid infrastructure installation, you must configure your domain name server (DNS) to send to GNS any name resolution requests for the subdomain served by GNS. This subdomain represents the cluster member nodes.

You must configure the DNS to send GNS name resolution requests using DNS forwarders. If the DNS server is running on Windows server that you administer, then the following steps need to be performed to configure DNS:

  1. Click Start, then select Programs. Select Administrative Tools and then click DNS manager. The DNS server configuration wizard starts automatically. Use the wizard to create an entry for the GNS virtual IP address. For example:

    gns-server.clustername.com: 192.0.2.1
    

    The address you provide must be static and routable.

  2. To configure DNS forwarders, click Start, select Administrative Tools, and then select DNS.

  3. Right-click ServerName, where ServerName is the name of the server, and then click the Forwarders tab.

  4. Click New, then type the name of the DNS domain for which you want to forward queries in the DNS domain box, for example, clusterdomain.example.com. Click OK.

  5. In the selected domain's forwarder IP address box, type the GNS virtual IP address, and then click Add.

  6. Click OK to exit.

If the DNS server is running on a different operating system, then refer to the Oracle Clusterware Installation Guide for that platform, or your operating system documentation.

Note:

Experienced DNS administrators may want to create a reverse lookup zone to enable resolution of reverse lookups. A reverse lookup resolves an IP address to a host name with a PTR or Pointer Resource record. If you have your reverse DNS zones configured, then you can automatically create associated reverse records when you create your original forward record.

2.6.4 Grid Naming Service Configuration Example

If you use GNS, then you need to specify a static IP address for the GNS VIP address, and delegate a subdomain to be delegated to that static GNS VIP address.

As nodes are added to the cluster, your organization's DHCP server can provide addresses for these nodes dynamically. These addresses are then registered automatically in GNS, and GNS provides resolution within the subdomain to cluster node addresses registered with GNS.

Because allocation and configuration of addresses is performed automatically with GNS, no further configuration is required. Oracle Clusterware provides dynamic network configuration as nodes are added to or removed from the cluster. The following example is provided only for information.

With a two node cluster where you have defined the GNS VIP, after installation you might have a configuration similar to the following, where the cluster name is mycluster, the GNS parent domain is example.com, the subdomain is grid.example.com, 192.0.2 in the IP addresses represents the cluster public IP address network, and 192.168.0 represents the private IP address network:

Table 2-1 Example of a Grid Naming Service Network

Identity Home Node Host Node Given Name Type Address Address Assigned By Resolved By

GNS VIP

n/a

Selected by Oracle Clusterware

mycluster-gns.example.com

Virtual

192.0.2.1

Fixed by net administrator

DNS

Node 1 Public

node1

node1

node1Foot 1 

Public

192.0.2.101

Fixed

GNS

Node 1 VIP

node1

Selected by Oracle Clusterware

node1-vip

Virtual

192.0.2.104

DHCP

GNS

Node 1 Private

node1

node1

node1-priv

Private

192.168.0.1

Fixed or DHCP

GNS

Node 2 Public

node2

node2

node2Footref 1

Public

192.0.2.102

Fixed

GNS

Node 2 VIP

node2

Selected by Oracle Clusterware

node2-vip

Virtual

192.0.2.105

DHCP

GNS

Node 2 Private

node2

node2

node2-priv

Private

192.168.0.2

Fixed or DHCP

GNS

SCAN VIP 1

n/a

Selected by Oracle Clusterware

mycluster-scan.grid.example.com

Virtual

192.0.2.201

DHCP

GNS

SCAN VIP 2

n/a

Selected by Oracle Clusterware

mycluster-scan.grid.example.com

Virtual

192.0.2.202

DHCP

GNS

SCAN VIP 3

n/a

Selected by Oracle Clusterware

mycluster-scan.grid.example.com

Virtual

192.0.2.203

DHCP

GNS


Footnote 1 Node hostnames may resolve to multiple addresses, including any private IP addresses or VIP addresses currently running on that host.

2.6.5 Manual IP Address Configuration Example

If you choose not to use GNS, then before installation you must configure public, virtual, and private IP addresses. Also, check that the default gateway can be accessed by a ping command. To find the default gateway, use the ipconfig command, as described in your operating system's help utility.

For example, with a two node cluster where each node has one public and one private interface, and you have defined a SCAN domain address to resolve on your DNS to one of three IP addresses, you might have the configuration shown in the following table for your network interfaces:

Table 2-2 Manual Network Configuration Example

Identity Home Node Host Node Given Name Type Address Address Assigned By Resolved By

Node 1 Public

node1

node1

node1Foot 1 

Public

192.0.2.101

Fixed

DNS

Node 1 VIP

node1

Selected by Oracle Clusterware

node1-vip

Virtual

192.0.2.104

Fixed

DNS, hosts file

Node 1 Private

node1

node1

node1-priv

Private

192.168.0.1

Fixed

DNS, hosts file, or none

Node 2 Public

node2

node2

node2Footref 1

Public

192.0.2.102

Fixed

DNS

Node 2 VIP

node2

Selected by Oracle Clusterware

node2-vip

Virtual

192.0.2.105

Fixed

DNS, hosts file

Node 2 Private

node2

node2

node2-priv

Private

192.168.0.2

Fixed

DNS, hosts file, or none

SCAN VIP 1

n/a

Selected by Oracle Clusterware

mycluster-scan

Virtual

192.0.2.201

Fixed

DNS

SCAN VIP 2

n/a

Selected by Oracle Clusterware

mycluster-scan

Virtual

192.0.2.202

Fixed

DNS

SCAN VIP 3

n/a

Selected by Oracle Clusterware

mycluster-scan

Virtual

192.0.2.203

Fixed

DNS


Footnote 1 Node hostnames may resolve to multiple addresses.

You do not need to provide a private name for the interconnect. If you want name resolution for the interconnect, then you can configure private IP names in the system hosts file or the DNS. However, Oracle Clusterware assigns interconnect addresses on the interface defined during installation as the private interface (Local Area Connection 2, for example), and to the subnet used for the private subnet.

The addresses to which the SCAN resolves are assigned by Oracle Clusterware, so they are not fixed to a particular node. To enable VIP failover, the configuration shown in the preceding table defines the SCAN addresses and the public and VIP addresses of both nodes on the same subnet, 192.0.2.

Note:

All host names must conform to the RFC 952 standard, which permits alphanumeric characters. Host names using underscores ("_") are not allowed.

2.6.6 Network Interface Configuration Options

The precise configuration you choose for your network depends on the size and use of the cluster you want to configure, and the level of availability you require.

If certified Network-attached Storage (NAS) is used for Oracle RAC and this storage is connected through Ethernet-based networks, then you must have a third network interface for NAS I/O. Failing to provide three separate interfaces in this case can cause performance and stability problems under heavy system loads.

2.7 Identifying Software Requirements

Depending on the products that you intend to install, verify that the following operating system software is installed on the system.

Note:

OUI performs checks your system to verify that it meets the listed operating system requirements. To ensure that these checks complete successfully, verify the requirements before you start OUI.

Oracle does not support running different operating system versions on cluster members, unless an operating system is being upgraded. You cannot run different operating system version binaries on members of the same cluster, even if each operating system is supported.

Table 2-3 lists the software requirements for Oracle Grid Infrastructure and Oracle RAC 11g Release 2 (11.2).

Table 2-3 Oracle Grid Software Requirements for Windows Systems

Requirement Value

System Architecture

Processor: AMD64, or Intel Extended memory (EM64T)

Note: Oracle provides only 64-bit (x64) versions of Oracle Database with Oracle Real Application Clusters (Oracle RAC) for Windows.

The 64-bit (x64) version of Oracle RAC runs on the 64-bit version of Windows on AMD64 and EM64T hardware. For additional information, visit My Oracle Support at the following URL:

http://support.oracle.com/

Operating system for 64-bit Windows

Oracle Grid Infrastructure and Oracle RAC for x64 Windows:

  • Windows Server 2003 x64 with service pack 1 or higher.

  • Windows Server 2003 R2 x64.

  • Windows Server 2008 x64 Standard, Enterprise, Datacenter, Web, and Foundation editions.

  • Windows Server 2008 R2 x64 Standard, Enterprise, Datacenter, Web, and Foundation editions.

The Windows Multilingual User Interface Pack and Terminal Services are supported.

NOTE: Oracle Clusterware, Oracle ASM and Oracle RAC 11g release 2 are not supported on any 32-bit Windows operating systems.

Compiler for x64 Windows

Pro*Cobol has been tested and is certified with Micro Focus Net Express 5.0. Object Oriented COBOL (OOCOBOL) specifications are not supported.

The following components are supported with the Microsoft Visual C++ .NET 2005 9.0 and Intel 10.1 C compilers:

  • Oracle Call Interface (OCI)

  • Pro*C/C++

  • External callouts

  • Oracle XML Developer's Kit (XDK)

Oracle C++ Call Interface is supported with:

  • Microsoft Visual C++ .NET 2005 8.0

  • Microsoft Visual C++ .NET 2008 9.0 - OCCI libraries are installed under ORACLE_HOME\oci\lib\msvc\vc9. When developing OCCI applications with MSVC++ 9.0, ensure that the OCCI libraries are correctly selected from this directory for linking and executing.

  • Intel 10.1 C++ compiler with the relevant Microsoft Visual C++ .NET STLs

Network Protocol

Oracle Net foundation layer uses Oracle protocol support to communicate with the following industry-standard network protocols:

  • TCP/IP

  • TCP/IP with SSL

  • Named Pipes


If you are currently running an operating system version that is not supported by Oracle Database 11g release 2 (11.2), such as Windows 2000, then you must first upgrade your operating system before upgrading to Oracle Database 11g Real Application Clusters.

If you are currently running a cluster with Oracle9i Clusterware and wish to continue to use it, then you must upgrade to the latest patchset for Oracle9i to ensure compatibility between Cluster Manager Services in Oracle9i and Oracle Database 11g release 2 (11.2).

2.7.1 Windows Firewall Feature on Windows Servers

When installing Oracle grid infrastructure software or Oracle RAC software on Windows servers, it is mandatory to disable the Windows Firewall feature. If the windows firewall is enabled, then remote copy and configuration assistants such as VIPCA, NETCA and DBCA will fail during Oracle RAC installation. Thus, the firewall must be disabled on all the nodes of a cluster prior to performing Oracle RAC installation.

Note:

The Windows Firewall should never be enabled on a network interface card (NIC) that is used as a cluster interconnect (private network interface).

After the installation is successful, you can enable the Windows Firewall for the public connections. However, to ensure correct operation of the Oracle software, you must add certain executables and ports to the Firewall exception list on all the nodes of a cluster. See Section 5.1.2, "Configure Exceptions for the Windows Firewall" for details.

Additionally, the Windows Firewall must be disabled on all the nodes in the cluster before performing any cluster-wide configuration changes, such as:

  • Adding a node

  • Deleting a node

  • Upgrading to patch release

  • Applying a one-off patch

If you do not disable the Windows Firewall before performing these actions, then the changes might not be propagated correctly to all the nodes of the cluster.

2.8 Network Time Protocol Setting

Each node in the cluster needs to use the same time reference. Follow the instructions in one of the following sections to configure time synchronization for your cluster nodes:

2.8.1 Configuring the Windows Time Service

The Windows Time service (W32Time) provides network clock synchronization on computers running Microsoft Windows. If you are using Windows Time service, and you prefer to continue using it instead of Cluster Time Synchronization Service, then you need to modify the Windows Time service settings to prevent the time from being adjusted backward. Restart the Windows Time service after you complete this task.

To configure Windows Time service, use the following command on each node:

C:\> W32tm /register

To modify the Windows Time service to prevent it from adjusting the time backwards, perform the following steps:

  1. Open the Registry Editor (regedit).

  2. Locate the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config key.

  3. Set the value for MaxNegPhaseCorrection to 0.

  4. To put the change into effect, use the following command:

    C:\>W32tm /config /update
    

2.8.2 Configuring Network Time Protocol

The Network Time Protocol (NTP) is a client/server application. Each server must have NTP client software installed and configured to synchronize its clock to the network time server. The Windows Time service is not an exact implementation of the Network Time Protocol (NTP), but it based on the NTP specifications.

If you decide to use NTP instead of the Windows Time service, then, after you have installed the NTP client software on each node server, you must start the NTP service with the -x option to prevent time from being adjusted backward.

To ensure the NTP service is running with the -x option, perform the following steps:

  1. Use the registry editor to edit the value for the ntpd executable under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTP

  2. Add the -x option to the ImagePath key value, behind %INSTALLDIR%\ntpd.exe.

  3. Restart the NTP service using the following commands:

    net stop NTP
    net start NTP
    

2.8.3 Configuring Cluster Time Synchronization Service

If you want to use Cluster Time Synchronization Service to provide synchronization service in the cluster, then disable the Windows Time service and stop the Network Time Protocol service.

When the installer finds that neither the Windows Time service or NTP service are active, the Cluster Time Synchronization Service is installed in active mode and synchronizes the time across the nodes. If the Windows Time service or NTP service is active, then the Cluster Time Synchronization Service is started in observer mode, and no active time synchronization is performed by Oracle Clusterware within the cluster.

To confirm that the Cluster Time Synchronization Service is active after installation, enter the following command as the Grid installation owner:

crsctl check ctss

2.9 Enabling Intelligent Platform Management Interface (IPMI)

Intelligent Platform Management Interface (IPMI) provides a set of common interfaces to computer hardware and firmware that system administrators can use to monitor system health and manage the system. With Oracle Database 11g release 2, Oracle Clusterware can integrate IPMI to provide failure isolation support and to ensure cluster integrity.

You can configure node-termination with IPMI during installation by selecting a node-termination protocol, such as IPMI. You can also configure IPMI after installation with crsctl commands.

See Also:

Oracle Clusterware Administration and Deployment Guide for information about how to configure IPMI after installation

2.9.1 Requirements for Enabling IPMI

You must have the following hardware and software configured to enable cluster nodes to be managed with IPMI:

  • Each cluster member node requires a Baseboard Management Controller (BMC) running firmware compatible with IPMI version 1.5 or greater, which supports IPMI over LANs, and configured for remote control using LAN.

    Note:

    On servers running Windows 2008, you may have to upgrade the BIOS, system firmware, and BMC firmware before you can use IPMI. Refer to Microsoft Support Article ID 950257 (http://support.microsoft.com/kb/950257) for details.
  • Each cluster member node requires an IPMI driver installed on each node.

  • The cluster requires a management network for IPMI. This can be a shared network, but Oracle recommends that you configure a dedicated network.

  • Each cluster member node's Ethernet port used by BMC must be connected to the IPMI management network.

  • Each cluster member must be connected to the management network.

  • Some server platforms put their network interfaces into a power saving mode when they are powered off. In this case, they may operate only at a lower link speed (for example, 100 MB, instead of 1 GB). For these platforms, the network switch port to which the BMC is connected must be able to auto-negotiate down to the lower speed, or IPMI will not function properly.

2.9.2 Configuring the IPMI Management Network

You can configure the BMC for DHCP, or for static IP addresses. Oracle recommends that you configure the BMC for dynamic IP address assignment using DHCP. To use this option, you must have a DHCP server configured to assign the BMC IP addresses.

Note:

If you configure IPMI, and you use Grid Naming Service (GNS), then you still must configure separate addresses for the IPMI interfaces. As the IPMI adapter is not seen directly by the host, the IPMI adapter is not visible to GNS as an address on the host.

2.9.3 Configuring the IPMI Driver

For Oracle Clusterware to communicate with the BMC, the IPMI driver must be installed permanently on each node, so that it is available on system restarts. On Windows systems, the implementation assumes the Microsoft IPMI driver (ipmidrv.sys), which is included on Windows Server 2008 and later versions of the Windows operating system. The driver is included as part of the Hardware Management feature, which includes the driver and the Windows Management Interface (WMI).

Note:

  • The ipmidrv.sys driver is not supported by default on Windows Server 2003. It is available for Windows 2003 R2, but is not installed by default.

  • An alternate driver (imbdrv.sys) is available from Intel as part of Intel Server Control, but this driver has not been tested with Oracle Clusterware.

2.9.3.1 Configuring the Hardware Management Component

Hardware Management is not installed and enabled by default on Windows Server 2003 systems. Hardware management is installed using the Add/Remove Windows Components Wizard.

  1. Press Start, then select Control Panel.

  2. Select Add or Remove Programs.

  3. Click Add/Remove Windows Components.

  4. Select (but do not check) Management and Monitoring Tools and click the Details button to bring up the detailed components selection window.

  5. Select the Hardware Management option.

    If a BMC is detected via the SMBIOS Table Type 38h, then a dialog box will be displayed instructing you to remove any third party drivers. If no third party IPMI drivers are installed or they have been removed from the system, then click OK to continue.

    Note:

    The Microsoft driver is incompatible with other drivers. Any third party drivers must be removed
  6. Click OK to select the Hardware Management Component, and then click Next.

    Hardware Management (including WinRM) will be installed.

After the driver and hardware management have been installed, the BMC should be visible in the Windows Device Manager under System devices with the label "Microsoft Generic IPMI Compliant Device". If the BMC is not automatically detected by the plug and play system, then the device must be created manually.

To create the IPMI device, run the following command:

rundll32 ipmisetp.dll,AddTheDevice

2.9.3.2 Configuring the BMC Using ipmiutil on Windows 2003 R2

For IPMI-based fencing to function properly, the BMC hardware must be configured for remote control through LAN. The BMC configuration may be effected from the boot prompt (BIOS), using a platform specific management utility or one of a number of publicly available utilities, which can be downloaded from the Internet, such as:

IPMIutil, which is available for Linux and Windows:

http://ipmiutil.sourceforge.net

Refer to the documentation for these configuration tools for instructions on how to configure the BMC.

When you configure the BMC on each node, you must complete the following:

  • Enable IPMI over LAN, so that the BMC can be controlled over the management network.

  • Enable dynamic IP addressing using DHCP, or configure a static IP address for the BMC.

  • Establish an administrator user account and password for the BMC

  • Configure the BMC for VLAN tags, if you will use the BMC on a tagged VLAN.

The configuration tool you use does not matter, but these conditions must be met for the BMC to function properly.

Example of BMC Configuration Using ipmiutil on Windows 2003 R2

The following is an example of configuring BMC using ipmiutil (version 2.2.3):

  1. Open a command window while logged in as a member of the Administrators group.

  2. After the driver is loaded and the device special file has been created, verify that ipmiutil is capable of communicating with the BMC via the driver:

    C:\> ipmiutil lan
    impiutil ver 2.23
    <PEFilter parameters displayed> . . .  
    pefconfig, GetLanEntry for channel 1 . . .
    Lan Param(0) Set in progress: 00
    . . . <remaining Lan Param info displayed>
    

    The following steps establish the required configuration parameters described in this example.

    Note:

    If you use the -l option, then ipmiutil sets certain LAN parameters only in the context of enabling IPMI over LAN. This can have the undesired effect of resetting to default values of some previously established parameters if they are not supplied on the command line. Thus, the order of the following steps could be critical.
  3. Establish remote LAN access with Administrator privileges (-v 4) and the desired user name and password (ipmiutil will find the LAN channel on its own):

    C:\> ipmiutil lan -l -v 4 -u user_name -p password
    
  4. Configure dynamic or static IP address settings for BMC:

    • Using dynamic IP addressing (DHCP)

      Dynamic IP addressing is the default assumed by Oracle Universal Installer. Oracle recommends that you select this option so that nodes can be added or removed from the cluster more easily, as address settings can be assigned automatically.

      Note:

      Use of DHCP requires a DHCP server on the subnet.

      Set the channel. For example, if the channel is 1, then enter the following command to enable DHCP:

      C:\> ipmiutil lan set -l -D
      
    • Using static IP Addressing

      If the BMC shares a network connection with the operating system, then the IP address must be on the same subnet. You must set not only the IP address, but also the proper values for the default gateway and the netmask for your network configuration. for example:

      C:\> impiutil lan -l -I 192.168.0.55   (IP address)
      C:\> ipmiutil lan -l -G 192.168.0.1    (gateway IP address)
      C:\> ipmiutil lan -l -S 255.255.255.0  (netmask)
      

      The specified address (192.168.0.55) will be associated only with the BMC, and will not respond to normal pings.

      Note:

      Enabling IPMI over LAN with the -l option will reset the subnet mask to a value obtained from the operating system. Thus, when setting parameters one at a time using the impiutil lan -l command, as shown above, the -S option should be specified last.
  5. Verify the setup.

    After the previous configuration steps have been completed, you can verify your settings on the node being configured as follows (the items in bold text reflect the settings just made):

    C:\> impiutil lan
    
    ipmiutil ver 2.23
    peconfig ver 2.23
    -- BMC version 1.40, IPMI version 1.5
    pefconfig, GetPefEntry ...
    PEFilter(01): 04 h : event ... <skipping PEF entries>
    ...
    
    pefconfig, GetLanEntry for channel 1 ...
    Lan Param(0) Set in progress: 00
    Lan Param(1) Auth type support: 17 : None MD2 MD5 Pswd
    Lan Param(2) Auth type enables: 16 16 16 16 00
    Lan Param(3) IP address: 192 168 0 55
    Lan Param(4) IP address src: 01 : Static
    Lan Param(5) MAC addr: 00 11 43 d7 4f bd
    Lan Param(6) Subnet mask: 255 255 255 0
    Lan Param(7) IPv4 header: 40 40 10
    GetLanEntry: completion code=cc
    GetLanEntry(10), ret = -1
    GetLanEntry: completion code=cc
    GetLanEntry(11), ret = -1
    Lan Param(12) Def gateway IP: 192 168 0 1
    Lan Param(13) Def gateway MAC: 00 00 0c 07 ac dc
    ...
    Get User Access(1): 0a 01 01 0f : No access ()
    Get User Access(2): 0a 01 01 14 : IPMI, Admin (user_name)
    Get User Access(3): 0a 01 01 0f : No access ()
    pefconfig, completed successfully
    
  6. Finally, you can verify that the BMC is accessible and controllable from a remote node in your cluster:

    C:\> ipmiutil health -N 192.168.0.55 -U user_name -P password
    ipmiutil ver 2.23
    bmchealth ver 2.23
    Opening connection to node 192.168.0.55 ...
    Connected to node racnode1.example.com 192.168.0.31
    BMC version 1.23, IPMI version 1.5
    BMC manufacturer = 0002a2 (Dell), product = 0000
    Chassis Status   = 01   (on, restore_policy=stay_off)
    Power State      = 00   (S0: working)
    Selftest status  = 0055 (OK)
    Channel 1 Auth Types: MD2 MD5
             Status = 14, OEM ID 000000 OEM Aux 00
    bmchealth, completed successfully
    

2.10 Checking Individual Component Requirements

This section contains these topics:

2.10.1 Oracle Advanced Security Requirements

You must meet hardware and software requirements to use authentication support with Oracle components. Some Oracle Advanced Security components can use a Lightweight Directory Access Protocol (LDAP) such as Oracle Internet Directory.

2.10.2 Oracle Enterprise Manager Requirements

All Oracle Enterprise Manager products that you use on your system must be of the same release. Older versions of Enterprise Manager are not supported with the current release.

Note:

All Oracle Enterprise Manager products, except Oracle Enterprise Manager Database Control, are released on the Enterprise Manager Grid Control installation media. Enterprise Manager Database Control is available on the Oracle Database installation media.

See Also:

Oracle Enterprise Manager Grid Control Installation and Basic Configuration available on the Enterprise Manager Grid Control installation media

2.11 Configuring User Accounts

To install the Oracle software, you must use a user that is a member of the Administrators group. If you use a local user account for the installation, then the user account must exist on all nodes in the cluster and the user name and password must be the same on all nodes.

If you use a domain account for the installation, then the domain user must be explicitly declared as a member of the local Administrators group on each node in the cluster. It is not sufficient if the domain user has inherited membership from another group. The user performing the installation must be in the same domain on each node. For example, you cannot have a dba1 user on the first node in the DBADMIN domain and a dba1 user on the second node in the RACDBA domain.

For example, assume that you have one Oracle installation owner, and the user name for this Oracle installation owner is oracle. The oracle user must be either a local Administrator user or a domain user, and the same user must exist (same user name, password, and domain) on each node in the cluster.

If you intend to install Oracle Database, then the oracle user must be part of the ORA_DBA group. During installation, the user performing the software is automatically added to the ORA_DBA group. If you use a domain user, then you must make sure the domain user on each node is a member of the ORA_DBA group.

2.11.1 Managing User Accounts with User Account Control

To ensure that only trusted applications run on your computer, Windows Server 2008 and Windows Server 2008 R2 provide User Account Control. If you have enabled this security feature, then depending on how you have configured it, Oracle Universal Installer prompts you for either your consent or your credentials when installing Oracle Database. Provide either the consent or your Windows Administrator credentials as appropriate.

You must have Administrator privileges to run some Oracle tools, such as Database Configuration Assistant, Net Configuration Assistant, and OPatch, or to run any tool or application that writes to any directory within the Oracle home. If User Account Control is enabled and you are logged in as the local Administrator, then you can successfully run each of these commands. However, if you are logged in as "a member of the Administrator group," then you must explicitly invoke these tasks with Windows Administrator privileges.

All of the Oracle shortcuts that require Administrator privileges are invoked as "Administrator" automatically when you click the shortcuts. However, if you run the previously mentioned tools from a Windows command prompt, then you need to run them from an Administrative command prompt. OPatch does not have a shortcut and has to be run from an Administrative command prompt.

2.12 Verifying Cluster Privileges

Before running Oracle Universal Installer, from the node where you intend to run the Installer, verify that the user account you are using for the installation is configured as a member of the Administrators group on each node in the cluster. To do this, enter the following command for each node that is a part of the cluster where nodename is the node name:

net use \\nodename\C$

If you will be using other disk drives in addition to the C: drive, then repeat this command for every node in the cluster, substituting the drive letter for each drive you plan to use.

The installation user must also be able to update the Windows registry on each node in the cluster. To verify the installation user is configured to do this, perform the following steps:

  1. Run regedit from the Run menu or the command prompt.

  2. From the 'File' menu choose: 'Connect Network Registry'

  3. In the 'Enter the object name…' edit box enter the name of a remote node in the cluster, then click OK.

  4. Wait for the node to appear in the registry tree.

If the remote node does not appear in the registry tree or you are prompted to fill in a username and password, then you must resolve the permissions issue at the OS level before proceeding with the Oracle Grid infrastructure installation.

Note:

For the installation to be successful, you must use the same user name and password on each node in a cluster or use a domain user. If you use a domain user, then you must have explicitly granted membership in the local Administrators group to the domain user on all of the nodes in your cluster.