Skip to main content

Setting Up a 2-node Fail-over Cluster with OES 1

· 16 min read

Preamble: This is a post on documenting the steps that I took to deploy a 2-node fail-over cluster for a client. No identifying information specific to the client is contained in this post. The deployment was done in August 2007 and I'm only posting it in December, nearly four months since the date I said I would put it online. Well, now that Open Enterprise Server (OES) 2 is already available, this post might not be that useful to others. Nevertheless, perhaps this will help someone somewhere out there ;-)

Setup​

The objective is a 2-node cluster to serve iPrint and iFolder services with fail-over capability. In my particular deployment, there were two Dell rack servers and a SAN storage server. To deploy this setup, the multipath tool had to be used to be able to address the SAN storage partitions over optics. I'm not familiar with SAN - the knowledge that I have is based on information I get from the Internet.

Step 1: Install OES on Both Servers​

I used OES1 SP2 on CDs (4 x SP2 and 3 x core) to install OES on both servers. Note that the servers have to be patched before doing anything to cluster them together. Below are the things to note.

Important! Do not install Novell Storage Services (NSS) or Novell Cluster Services (NCS) during the installation — the multipath-tools package should be installed and then configured in order for the shared storage (the SAN) to be recognised by NSS. And multipath-tools can only be configured after the installation. The multipath-tools package is required because the SBD (Split Brain Detector) partition is needed to create the cluster, and the SBD must be created on the SAN partition.

When I went through the documentation, I was puzzled because it didn't say explicitly that the multipath-tools has to be configured AFTER installation. Although it seems rather obvious here, believe me, when you are caught in the thick of things, simple logic can be very complicated to the puzzled mind.

Configure OES services during installation.

Step 2: Patch the Systems​

Do NOT patch the system in one shot! The first time I did this, I had to download a whopping 2GB worth of patches - and they didn't work! Patching everything in one shot actually kills the systems - which means you have to reinstall.

The correct way to go about doing this is to install patch-11371 and then patch-11461 before the rest of the patches. The effect of applying the first patch (11371) is it helps remove a large part of the patches that have to be downloaded. In my case, instead of 2GB, I only needed about 900MB of patches. After applying this patch, restart the Red Carpet daemon. The commands to do so are described below:

#Displays the services

rug sl

#If not available from the output from above, add the service

rug sa https://update.novell.com/data

#activate the product

rug act -s 1 123465879 admin@company.com

#displays available channels

rug ch

#subscribe to the channel 'oes'

rug sub oes

#displays all available patches (installed and non-installed)

rug pl

#install patch-11371

rug pin patch-11371

#after the patch is applied successfully, restart the daemon

rcrcd restart

After the daemon is restarted, install patch-11461. This is a security patch.

rug refresh

rug pin patch-11461

#it is better to restart the daemon again

rcrcd restart

Finally install the rest of the patches. In my case, the patches took four hours at around 50kbps.

#the listing of patches is so much smaller now compared to before

rug refresh

rug pl

#finally, update the system

rug pin --entire-channel oes

Once the patching is complete, reboot the system. This is needed because there might be kernel updates which require a reboot.

In my case the update did not complete successfully as there was an error about restarting the daemon, which I suspect is referring to the Red Carpet daemon. I was unable to verify this but after rebooting, the system turned out fine.

For more information, refer to Novell's TID #3739116

Step 3: Install Multipath-tools​

This tool is required to handle multipathing on the storage side.

Each of my servers only has one Host Bus Adapter (HBA) card with one port. However at the storage side there were two paths to the storage. As a result Linux showed two sets of the same storage (e.g. sdb-sdc-sdd and sde-sdf-sdg).

Install the package from the software management option from YaST. After installing you should be looking at Red Carpet for patches. This time, you can use the Red Carpet GUI (ZENworks Linux Management).

To use ZENworks Linux Management, first refresh the channel for any new patches. The filter should show you the patches for the multipath tools. Install those patches.

Then run the multipath programs:

#makes the following programs run at boot time

chkconfig boot.multpath on

chkconfig multipathd on

#start the programs NOW

/etc/init.d/boot.multipath start

/etc/init.d/multipath start

#check that the services are indeed started at boot time

chkconfig -l | grep multi

Take note that the chkconfig commands might not work correctly - double check that the services are started at boot time. If they are not (which was the case with boot.multipath for me), use the GUI tool from YaST to enable it.

Once boot.multipath has started running, the device mapper will create a directory /dev/disk/by-name with the WWN names of the storage LUNs in there. These names will be the means how the storage devices are accessed.

At this time you can invoke the following command to see if the LUNs are detected.

multipath -l

What I encountered was that the devices are detected but are not active yet. As I mention above, there are two paths to the device so at least one should be active. After invoking the commands above, the two paths to the device are shown as "enabled" but not active. I'd not found any faster way to make the path active other than rebooting.

Of course you KNOW that you need to install multipath-tool on both servers

Step 4: Install NSS & NCS on Both Servers​

The clustering services should be installed by invoking the GUI options from YaST because the process adds dependency checking so that the required packages are actually installed. Where clustering is concerned, the required packages are open-ipmi, heartbeat-stonith, heartbeat, novell-nss-admin-session-openwbem-provider, novell-nss and novell-cluster-services-km (I think).

Prior to installing you should make sure that the following packages are of the indicated version number or later.

novell-ncp-5.0.1-48

novell-ncpserv-1.3-6

novell-nss-4.9.20-1

evms-2.3.3-0.43

novell-cluster-services-km-1.8.2.57

Personally I will take the following steps to set up clustering while ensuring the packages are up-to-date. First I will go to the first server and invoke YaST > System > Novell Storage Services. This step will do the dependency checking and configure NSS for the cluster. There may be additional packages that need to be installed. If this is the case, be sure to invoke the Red Carpet GUI to update the newly installed packages.

The next step is to then configure NCS from YaST > System. Similarly, additional packages may need to be installed. Configure the cluster as descibed by Novell's documents. If any new packages were installed, they should be updated using Red Carpet after configuration.

I then configured the Role Based Service (RBS) Configuration using iManager. This is to set up the privileges for the different users and modules in eDirectory.

Install NSS on the second server. After doing that (and updating), check the directory /opt/novell/oes_install for a file nssid.sh. If this file is present, run it. It supposedly adjusts the permissions of some files of the server which might be messed up by the installation of NSS.

Finally install NCS for the second server. There is no need to configure RBS as both servers are in the same tree. Reboot both servers after installing and configuring the cluster.

Step 5: Configure the Shared Storage​

This is a two step process. First the storage devices have to be configured for use through EVMS. Then the devices have to be cluster-enabled through iManager.

5.1. Configure EVMS to Handle the Shared Storage​

Before using EVMS, make sure that NCS is installed on ALL nodes that are in the cluster.

Then (preferably) from the master node start EVMSGUI. This will invoke the evmsd_worker process on the rest of the nodes. This is how the SAN storage devices are locked by EVMS and why EVMSGUI should not be left running unnecessarily.

Follow the steps as described in Novell's documnetation for configuring the clustered resources. Below is a summary:

  1. Check from the Volumes tab to see if the volume is there. If it is there, make sure to remove the Compatibility status from it.
  2. Remove the Segment Manager from the disks using the Disks tab. Save.
  3. Create a new Container and choose Cluster Segment Manager for it. It should be a private type.
  4. Make this newly created container active by modifying its properties from the Container tab. As insignificant as this step may appear, you should NOT skip it! Save. Once saved, the 'Dirty' mark on the container should disappear.
  5. Create a new EVMS volume on this container. Save.
  6. Go to the Volumes tab and create a file system on the volume.

Cluster-enable the Volume​

After configuring EVMS to be the volume manager for the storage, the next step is to use iManager to configure the volumes that can be failed over between the nodes. This step is clearly described with Novell's documentation.

Step 6: Create the iPrint Resource​

From iManager, create a cluster resource using the iPrint_Template. Edit the load/unload scripts accordingly.

I named this resource iprint_store.

Change the timeout value to 3 minutes and make sure that the Ignore Quorum option is unchecked.

The scripts are located in /etc/opt/novell/ncs and /var/opt/novell/ncs. I don't know which one is the one used by iManager because the scripts in both locations should be identical.

Finally start the iPrint resource.

Step 7: Create the Driver Store and Print Manager​

Before creating the Driver Store, create an organiztion unit (ou) Printers.main to hold the printer objects (where main is the root of the tree). The purpose of doing so is more for organisation than anything else.

From iManager > iPrint, click on Create Driver Store.

After the Driver Store is created, proceed to create the Print Manager (iManager > iPrint > Create Print Manager).

There might be an eror with the certificate when the Driver Store is first created. Click on the link and accept the certificate.

Edit the following files and remove the extra double quotes (") at the end of the opening <OBJECT> tag at lines 12, 13 or 14:

  • CreatePrinterDriverProfile2.jsp
  • ModifyPrinterDriverProfile2.jsp
  • PrinterDriverProfile.jsp
  • ResourceDriversPage.jsp
I forgot where these files are located, but you can easily use the find or locate command to find these files.

Step 8: Add Printer Drivers​

From iManager > iPrint > Manage Driver Store, click on Drivers.

If you are using Xerox Phaser printers, download the Windows drivers from Xerox's website. Look out for the Microsoft PPD set of drivers. Unzip this file and then use the INF file to upload to iManager to install the printer driver.

Step 9: Create a Printer​

iManager > iPrint > Create Printer

If the printer driver is available, it can be assigned to the printer immediately. Otherwise, it can be assigned later by clicking Manager Printer > Drivers.

Optionally, a printer profile may be created and subsequently assigned to the printer.

Step 10: Create a Printer Map​

Make sure that ippsrvr.nlm is loaded in iManager.

Copy the map image to the location /var/opt/novell/iprint/htdocs/images/maps and the printer icons to /var/opt/novell/iprint/htdocs/images/printers.

Start Map Designer at http://<dns_name_of_iprint_service>/maptool.htm

Once the map is saved, upload it to the /var/opt/novell/iprint/htdocs folder.

(Next page "Install & Configure iFolder Enterprise & iFolder Web Access")

Step 11: Install & Configure iFolder Enterprise & iFolder Web Access​

Install but do not configure iFolder Enterprise and iFolder Web Access on all the nodes.

Before configuring iFolder, prepare the shared storage device.

Follow the procedure in Step 5 to create and cluster enable the SAN device.

Configuring iFolder Enterprise​

Before configuring iFolder on the master node, the shared volume must be mounted on the node. This is because the configuration will place some files onto the storage deivce which has to be shared with other nodes later.

From YaST, configure iFolder Enterprise Server. Take note that the System Store Path should be the (mounted) share volume created above. In addition append a data directory at the end of the mount point for the shared volume because it is recommended not to store the data files at the root of the shared volume.

Something peculiar about the setting here is that the example given for the iFolder Admin DN has only the cn and o components. In practice, you need another context, an Organization Unit under Organization to host the iFolder Admin account. It may be necessary for you to create the ou manually.

At the end of the configuration procedure, Apache will be restarted. test if the service is working by entering the URL of the master node http://<dns_name_of_master_node>/simias10/Simias.asmx

Manual Configuration​

Copy the /var/lib/wwwrun/.local directory to the mount point of the shared volume:

cp -a /var/lib/wwwrun/.local /mnt/mount_ifolder

Remove the /var/lib/wwwrun/.local directory and replcae it with a symbolic link (of an identical name) to the .local directory which was copied to the shared volume. Then change the ownership.

rm -f /var/lib/wwwrun/.local
ln -s /mnt/mount_ifolder/.local /var/lib/wwwrun/
chown -R wwwrun:www /var/lib/wwwrun/.local

Finally restart Apache and go to the URL http://<dns_name_of_node_1>/simias10/Simias.asmx to check that you get an authentication prompt.

Configuring iFolder Web Accessss​

The alias that you use for node 1 should be the same for the other node i.e. /ifolder

For the iFolder server URL use https://localhost

Finally unmount the shared storage so that other nodes can mount it.

Configure Other Nodes​

Start YaST and configure iFolder Enterprise Server. For these non-master nodes, set the System Store Path to /tmp/ifolder. This value will be replaced later.

It is important that you do not specify the shared storage - stick with the temporary folder for the configuration.

Check the URL after the configuration to ensure that iFolder was configured correctly.

Configure iFolder Web Acces. Use https://localhost as well.

Stop the Apache service. Copy the file /top/novell/ifolder3/etc/simias-server-bootstrap.config file from the master node (the first one that was configured) and replace the existing one on the other nodes.

Delete the /var/lib/wwwrun/.local directory and replace it with a symbolic link to the shared volume's .local directory. Change the ownership of this link as well.

Configure the iManager Plug-in for iFolder​

After iFolder is installed, iManager should show on the homepage that there is a plug-in waiting to be installed. Follow the link. Alternatively, you can click on the Configure icon, click on Module Installatoin > Available Novell Plug-in Modules. There should only be one - iFolder3.npm. Select it and click on Install.

Close iManager and restart Tomcat:

rcnovell-tomcat4 restart

Provision Users for iFolder 3.x​

If ther are no users for the system, create their accounts and a container to hold them. From iManager, click on View Objects at the top. Go to the container ifcontext. Click on this object and select Create Object. Choose Organization Unit. Give this object the name Users.

Then create users from iManager > Roles and Tasks. Select Users > Create User. Put these users into the Users container created under the context ifcontext.

From iManager, Novell iFolder 3 > System, provide the highly available IP for iFolder as the iFolder server.

Enter the iFolder user name ifolderadmin and the password for authenticatoin and click OK.

Click OK again to import the security certificate.

Click on the LDAP tab and click Modify.

The server host should be the highly available DNS name of the master resource (i.e. the DNS name of the cluster).

In the Search DN's field, select the object Users.ifcontext.main

Click on OK and then Update and Synchronize Now.

Install the Client​

From the cluster URL, click on the iFolder 3.0 link on the left and click on iFolder 3.0 Windows Client. Install the client and restart the workstation. (Of course this requires that you have administrative privileges to the workstation.)

Make sure that the firewall in Windows XP is not blocking the MOno-XSP server.

Configure iFolder Accounts and Create iFolders​

Make sure that the user accounts are already created in eDirectory. Log in to a workstation with the user account. You will be prompted whether you want to set up an iFolder account.

After logging into the server, you will be able to cconvert your folders into iFolders

Useful Info​

  • /opt/novell/ifolder3/etc/simias_server_bootstrap.config is modified by the YaST configuratoin tool for iFolderEnterprise
  • iFolder3.npm can be found at /var/opt/novell
  • Security certificates for iPrint are stored in /var/opt/novell/iManager/nps/portal/modules/iPrintX/certstore. Removing the file in this directory will revoke the certificates
  • /var/opt/novell/iprint/htdocs/images/{maps | printers} are where the maps and printer icons should be placed
  • http://<dns_name_of_iprint_store>/maptool.htm is the map designer. The created map should be placed in /var/opt/novell/iprint/htdocs
  • The backup database for iPrint is in /var/opt/novell/iprint/psmdb
  • iPrint can be made to look for more than one eDirectory server by adding DSServer2, DSServer3, etc. to the files /etc/opt/novell/iprint/iprint_mgr.ipsmd.conf (or idsd..conf I forget)
  • iFolder setup script is located at /opt/novell/ifolder3/bin/simias-server-setup
  • The PID file for iFolder is /var/opt/novell/ifolder/run/ifolder.pid
  • A file iprint.ini is in each server where iPrint is installed. This file controls whether the user is prompted before the updated client is installed without user intervention. Look out for the line "Allow Auto Update".
  • TIDs
    • #10100002 Troubleshooting OES SP2 Patch Process
    • #3377050 Guide to patching Novell Linux products
    • #3045794 How to correctly patch an OES Linux via RUG
    • #3003435 How to create an OES cluster resource with a native Linux file system
    • #10100285 Config Report tool
    • #3248305 Correcting Expired Certificates in the iFolder 3.x Keystore
    • #10098796 Export eDirectory Certificate and import into OpenSSL for iFolder
    • #3130715 Cannot login to iFolde3
    • #3827266 lputil does not discover Emulex cards after upgrading to SLES9 SP3