How to build Oracle VM Manager 3.1.1


Jephe Wu - http://linuxtechres.blogspot.com

Environment: Dell R610 server, Oracle Linux 6.3, Oracle VM Manager 3.1.1 with patch 399, Oracle database enterprise edition 11.2.0.3 with the latest patchset, Ksplice
Objective: install a fully patched Oracle VM Manager 3.1.1 production host


Steps:
1. BIOS config
Enable logical processor(hyper trending), virtualization, Turbo mode, disable C1E, C states, node interleaving(enable NUMA)

 C-state
In Oracle VM server 3.1.1, to check if C-state is enabled or not, use the following command to check if the total C-state is not more than 2.

# xenpm get-cpuidle-states | grep total | uniq
total C-states : 2

or in Linux, check it :
C-state: cat /proc/acpi/processor/CPU*/power
P-state: cat /sys/devices/system/cpu/cpu*/cpufreq/stats/total_tran
         cat /sys/devices/system/cpu/cpu*/cpufreq/stats/time_in_state

NUMA
# numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
node 0 size: 49139 MB
node 0 free: 33832 MB
node distances:
node 0
0: 10

How to disable NUMA in Linux OS and Oracle:
pass numa=off to kernel boot parameter in grub.conf, you can check the current kernel parameter in /proc/cmdline.
According to MOS: Enable Oracle NUMA support with Oracle Server Version 11.2.0.1 [ID 864633.1]:  Starting with 11.2.0.1, Oracle NUMA support is disabled by default.
example output: NUMA is disabled.

[root@util01 ~]# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                24
On-line CPU(s) list:   0-23
Thread(s) per core:    2
Core(s) per socket:    6
CPU socket(s):         2
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 44
Stepping:              2
CPU MHz:               2660.346
BogoMIPS:              5319.79
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              12288K
NUMA node0 CPU(s):     0-23

[root@util01 ~]# numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
node 0 size: 49139 MB
node 0 free: 40698 MB
node distances:
node   0 
  0:  10 

power mode - use maximum performance mode
References:
http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.0.pdf  check general bios settings and power management settings part.

2.  Configuring iDRAC ip address, netmask and gateway

After you have done this part, you are able to change it through DRAC and dell openmanage for Linux later on. The default DRAC username and password is root/calvin.

How to reset system by sshing into DRAC?
# racadm serveraction hardreset


3.  OS installation through kickstart by foreman

Preparing tftp/dhcp/[dns] server first by PXELinux, DNSmasq or cobber for foreman, we use traditional PXELinux here.
service tftp
{
    socket_type     = dgram
    protocol        = udp
    wait            = yes
    user            = root
    server          = /usr/sbin/in.tftpd
    server_args     = -vvv -s /tftpboot
    disable         = no
    per_source      = 11
    cps             = 100 2
    flags           = IPv4
}


under /tftpboot/pxelinux.cfg, put the following kernel files:

-rw-r--r-- 1 root root 30481341 Aug 20 15:27 OL-6.3-x86_64-initrd.img
-rw-r--r-- 1 root root  3986992 Aug 20 15:27 OL-6.3-x86_64-vmlinuz

# more /tftpboot/pxelinux.cfg/default
default linux
label linux
localboot 0  (important, boot into local hard disk if tftp failed, although DHCP is okay)

Preparing foreman
# more /etc/foreman/settings.yaml
:modulepath: /etc/puppet/modules/
:tftppath: /tftpboot/
:unattended: true

Preparing foreman-proxy

# cat /etc/foreman-proxy/settings.yml| grep -v ^# | grep -v ^$
:daemon: true
:daemon_pid: /var/run/foreman-proxy/foreman-proxy.pid
:port: 8443
:tftp: true
:tftproot: /tftpboot
:tftp_servername: 192.168.0.2
:dns: false
:dhcp: true
:dhcp_vendor: isc
:dhcp_config: /etc/dhcp/dhcpd.conf
:dhcp_leases: /var/lib/dhcpd/dhcpd.leases
:puppetca: true
:puppet: true
:log_file: /tmp/proxy.log
:log_level: DEBUG

Preparing DHCP server

#  more /etc/dhcp/dhcpd.conf
allow booting;
allow bootp;
omapi-port 7911;
deny unknown-clients;


subnet 192.168.0.0 netmask 255.255.255.0 {
  option broadcast-address 192.168.0.255;
  option routers 192.168.0.1;
  option domain-name "jephe";
    option domain-name-servers 192.168.0.254;
  next-server 192.168.0.2;
  filename "pxelinux.0";

}

After adding util01.jephe by foreman, it appears in /var/lib/dhcpd/dhcpd.leases as follows:

server-duid "xxxxxx";

host util01.jephe {
  dynamic;
  hardware ethernet d4:be:d9:ef:aa:bb;
  fixed-address 192.168.0.78;
        supersede server.filename = "pxelinux.0";
        supersede server.next-server = aa:bb:cc:dd; #actually, it's hex code of tftp server ip addr in /etc/foreman-proxy/settings.yml
        supersede host-name = "util01.jephe";
}


access DeLL iDRAC at https://dracipaddress , system, properties, EMBEDDED NIC MAC ADDRESSES to get onboard nic1 MAC address
so that you can configure foreman.

4. foreman configuration in GUI interface
kickstart config for Oracle Linux 6.3:
install
lang en_US
keyboard us
skipx
text
reboot
url --url http://repo.jephe/OL/6.3/os/x86_64/
rootpw --iscrypted $6$ruyiAJ82a/9Bazd7$zspMbTz4e7omVlb12uTa.bjuvc8SuxcGoVHgJv3Y3r1m/4.7tONDUzPVpJpYWA85uXaaaaaaaaaaaaa
firewall --disabled
selinux --disabled
authconfig --enableshadow --enablemd5
timezone --utc Australia/Sydney
bootloader --location=mbr
<%= @host.diskLayout %>
%packages
@base
-iptables
ntp
lrzsz
telnet
hwloc
numactl
# Oracle recommended tigervnc instead of its own one
tigervnc
oracle-rdbms-server-11gR2-preinstall

%post --log=/root/kickstart-post.log
echo "Syncing system date and update bios date"
ntpdate puppet.jephe
clock --systohc

# Disable most things. Puppet will activate these if required.
echo "Disabling various system services..."
<% %w{autofs gpm atd abrt-ccpp abrt-oops abrtd acpid cpuspeed rpcbind rpcgssd rpcidmapd sendmail cups iptables ip6tables auditd arptables_jf xfs pcmcia isdn rawdevices hpoj bluetooth

openibd avahi-daemon avahi-dnsconfd hidd hplip pcscd restorecond mcstrans rhnsd yum-updatesd}. each do |service| -%>
  /sbin/chkconfig --level 345 <%= service %> off 2>/dev/null
<% end -%>
     
echo "Running puppet..."
#/usr/sbin/puppetd --debug --no-daemonize --verbose --onetime
sync

# Force the right hosts file
echo "generating correct /etc/hosts"
IP=`ifconfig em1 | grep "inet " | cut -d : -f 2 | cut -d " " -f 1`
cat > /etc/hosts <<EOF
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1     localhost.localdomain localhost
::1           localhost6.localdomain6 localhost6
`echo -n $IP`         $(hostname)  $(hostname -s)
EOF

# generate dell openmanage script
cat > /root/dell_openmanage.sh <<END
#!/bin/sh
export http_proxy=http://puppet.jephe:8080
wget -q -O - http://linux.dell.com/repo/hardware/latest/bootstrap.cgi | bash
yum -y -e 0 install srvadmin-all
yum -y -e 0 install dell_ft_install
yum -y -e 0 install \$(bootstrap_firmware)
echo "now you can run  inventory_firmware then update_firmware"
END

chmod +x /root/dell_openmanage.sh

echo "Informing Foreman that we are built.."
wget -q -O /dev/null --no-check-certificate http://puppet.jephe:3000/unattended/built

After install dell openmanager, you can access it through https://serveripaddress:1311, login with root password

a. refresh cdrom installation anaconda config for VM server 3.1.1 (use eth4 as ovsagent interface and add to vlan 1001)

install
eula Accepted
cdrom
lang en_US.UTF-8
keyboard us
network --device eth4 --bootproto static --ip 192.168.0.2 --netmask 255.255.255.0 --gateway 192.168.0.254 --nameserver 192.168.0.3,8.8.8.8 --hostname util01.jephe
ovsagent --iscrypted X12345TkAjLZk+TLABCDEFGHJIg=
ovsmgmntif eth4.1001
rootpw --iscrypted $1$abcde$O5ngOGU.0eXsXIfk6QB2B.
firewall --enabled --port=21:tcp --port=22:tcp --port=53:udp --port=53:tcp --port=80:tcp --port=2049:tcp --port=5900-7999:tcp --port=8002:tcp --port=8003:tcp --port=8899:tcp --port=7777:tcp
authconfig --enableshadow --enablemd5
selinux --disabled
timezone --utc Australia/Sydney
bootloader --location=mbr --dom0_mem=5800 --driveorder=sda
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
#clearpart --all --drives=sda
#part /boot --fstype ext3 --size=100 --ondisk=sda
#part / --fstype ext3 --size=3072 --ondisk=sda
#part swap --size=1024 --ondisk=sda

%packages
@base
@core
@ovs-virtualization

Note:
[root@util01 init.d]# openssl passwd -1 -salt abcde "password"
$1$abcde$O5ngOGU.0eXsXIfk6QB2B.

According to Oracle support, you can use the following in %post to display on screen
%post --log=/root/kickstart-post.log
#actually, if you specify tee /root/ks-post_install.log, you can ignore this --log as it won't generate anything inside

exec < /dev/tty3 > /dev/tty3
chvt 3
(
.....

) 2>&1 | /usr/bin/tee /root/ks-post_install.log
echo
chvt 1


b. without static ip address, use generate kickstart config file for everything
eula Accepted
reboot
install
lang en_US.UTF-8
keyboard us
skipx
timezone Australia/Sydney
ovsagent --iscrypted XM3Ve5TkAjLZAAAAAAAAAAAAAAA
ovsmgmntif eth0
url --url http://repo.jephe/xen/3.1.1/os/x86_64/
rootpw --iscrypted $1$/NL6hNln$mVk8AAAAAAAAAAAAAA.
firewall --disabled
selinux --disabled
authconfig --enableshadow --enablemd5
timezone  Australia/Sydney
bootloader --location=mbr
<%= @host.diskLayout %>
%packages
@core
@base
@ovs-virtualization

%post --log=/root/kickstart-post.log
echo "Informing Foreman that we are built.."
wget -q -O /dev/null --no-check-certificate http://puppet.jephe:3000/unattended/built

And diskLayout is as follows:
clearpart --all --drives=sda
part /boot --fstype ext3 --size=100 --ondisk=sda
part / --fstype ext3 --size=3072 --ondisk=sda
part swap --size=1024 --ondisk=sda

c. /tftpboot/pxelinux.cfg/01-MACaddress


[root@puppet pxelinux.cfg]# more 01-d4-ae-52-aa-bb-cc
default linux
label linux
kernel boot/OVS-OVS3.1-vmlinuz
append initrd=boot/OVS-3.1-OVS3.1-initrd.img ks=http://puppet.jephe:3000/unattended/provision ksdevice=bootif network kssendmac
ipappend 2

or use the following for specific ip address
append initrd=boot/OVS-3.1-OVS3.1-initrd.img ks=http://puppet.jephe:3000/unattended/provision ksdevice=bootif
network --device eth5 --bootproto static --ip 192.168.0.2 --netmask 255.255.255.0 --gateway 192.168.0.254 --nameserver 192.168.0.1,8.8.8.8 --hostname xens01.jephe kssendmac

or you can use general pxelinux.cfg, but in foreman provision script, use if-then to dynamic generate ipaddress for different hostname as folows:
echo "configuring correct IP address for bond1.2091 for `hostname`"
if [ `hostname` == 'xens01.jephe' ]
then
  cat >> ifcfg-bond1.91 <<EOF
IPADDR=192.168.0.2
GATEWAY=192.168.0.254
NETMASK=255.255.255.0
EOF
elif [ `hostname` == 'xens02.jephe' ]
then
   cat >> ifcfg-bond1.91 <<EOF
IPADDR=192.168.0.3
GATEWAY=192.168.0.254
NETMASK=255.255.255.0
EOF
fi

5. ksplice and ULN
quickstart guide: https://oss.oracle.com/ksplice/docs/ksplice-quickstart.pdf
firstly, get access key for ksplice,
then run the following command

wget -N https://www.ksplice.com/uptrack/install-uptrack
sh install-uptrack 123445555kklkkddkdkakdk
uptrack-upgrade -y

configuring https_proxy in /etc/uptrack/uptrack.conf


Uninstall of Uptrack is very easy. You should consider removing the updates with
uptrack-remove --all followed by #yum remove uptrack.


[root@util01 uptrack]# yum remove uptrack
Loaded plugins: dellsysid, rhnplugin, security
Setting up Remove Process
Resolving Dependencies
--> Running transaction check
---> Package uptrack.noarch 0:1.2.2-0.el6 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

==================================================================================================================================================================================================================
 Package                                        Arch                                          Version                                               Repository                                               Size
==================================================================================================================================================================================================================
Removing:
 uptrack                                        noarch                                        1.2.2-0.el6                                           @ksplice-uptrack                                        1.1 M

Transaction Summary
==================================================================================================================================================================================================================
Remove        1 Package(s)

Installed size: 1.1 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Erasing    : uptrack-1.2.2-0.el6.noarch                                                                                                                                                                     1/1
warning: /etc/uptrack/uptrack.conf saved as /etc/uptrack/uptrack.conf.rpmsave
  Verifying  : uptrack-1.2.2-0.el6.noarch                                                                                                                                                                     1/1

Removed:
  uptrack.noarch 0:1.2.2-0.el6                                                                                                                                                                                  

Complete!

Note: this will clear /var/cache/uptrack directory .


ULN:
go to linux.oracle.com to register which binds your SSO to ULN, then run uln_register --proxy=1.2.3.4:8080 to register Oracle Linux to ULN, proxy is optional, during text gui interface, it will ask you SSO, CSI etc information for register.

run yum repolist to check if the following are listed.


ol6_x86_64_UEK_latest                                                                                          Latest Unbreakable Enterprise Kernel for Oracle Linux 6 (x86_64)                                                                                             71
ol6_x86_64_latest                                                                                              Oracle Linux 6 Latest (x86_64)  


then run yum update to update system.

6.  install Dell openmanage after os install
refer to http://www.fatmin.com/2010/08/installing-dell-openmanage-server-administrator-on-linux.html

Install OpenManage Server Administrator

Step 1: wget -q -O - http://linux.dell.com/repo/hardware/latest/bootstrap.cgi | bash
Step 2: yum -y install srvadmin-all
Step 3: Start  Systems Management Data Engine - /etc/init.d/dataeng start
Step 4: Start webinterface - /etc/init.d/dsm_om_connsvc start

Your system homepage is now availible via https://hostname:1311, where hostname is the hostname of your server.

Install Firmware/Bios Management Tools

Step 1: yum install dell_ft_install
Step 2: yum install $(bootstrap_firmware)


Now run either inventory_firmware or inventory_firmware_gui to view your firmware versions, and perform firmware upgrades. Also, you can access server ip dell openmanage at https://serveripaddress:1311/


7. Dell Nagios plugin installation for hardware monitoring

Go to http://fedoraproject.org/wiki/EPEL/FAQ to download epel

rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm
then download perl-Config-Tiny rpm from public yum repository:

then install Dell openmanage nagios plugin by running

yum install nagios-plugins-openmanage

[root@util02 yum.repos.d]# /usr/lib64/nagios/plugins/check_openmanage
OK - System: 'PowerEdge R610 II', SN: '5HZXXX', 48 GB ram (6 dimms), 1 logical drives, 2 physical drives

8. Silent install database

unzip downloaded Oracle database 11.2.0.3 64bit, unzip it, find *.rsp file

[root@util01 database]# cat inst.rsp | grep -v ^$ | grep -v ^# | grep -v =$
oracle.install.responseFileVersion=/oracle/install/rspfmt_dbinstall_response_schema_v11_2_0
oracle.install.option=INSTALL_DB_AND_CONFIG
ORACLE_HOSTNAME=util01
UNIX_GROUP_NAME=dba
INVENTORY_LOCATION=/u01/app/oraInventory
SELECTED_LANGUAGES=en
ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
ORACLE_BASE=/u01/app/oracle
oracle.install.db.InstallEdition=EE
oracle.install.db.EEOptionsSelection=false
oracle.install.db.optionalComponents=oracle.rdbms.partitioning:11.2.0.3.0,oracle.oraolap:11.2.0.3.0,oracle.rdbms.dm:11.2.0.3.0,oracle.rdbms.dv:11.2.0.3.0,oracle.rdbms.lbac:11.2.0.3.0,oracle.rdbms.rat:11.2.0.3.0
oracle.install.db.DBA_GROUP=dba
oracle.install.db.OPER_GROUP=dba
oracle.install.db.config.starterdb.type=GENERAL_PURPOSE
oracle.install.db.config.starterdb.globalDBName=orcl
oracle.install.db.config.starterdb.SID=orcl
oracle.install.db.config.starterdb.characterSet=AL32UTF8
oracle.install.db.config.starterdb.memoryOption=true
oracle.install.db.config.starterdb.memoryLimit=4096
oracle.install.db.config.starterdb.installExampleSchemas=false
oracle.install.db.config.starterdb.enableSecuritySettings=true
oracle.install.db.config.starterdb.password.ALL=password
oracle.install.db.config.starterdb.control=DB_CONTROL
oracle.install.db.config.starterdb.automatedBackup.enable=false
oracle.install.db.config.starterdb.storageType=FILE_SYSTEM_STORAGE
oracle.install.db.config.starterdb.fileSystemStorage.dataLocation=/u01/oradata
oracle.install.db.config.starterdb.fileSystemStorage.recoveryLocation=/u01/oraarch
DECLINE_SECURITY_UPDATES=true
oracle.installer.autoupdates.option=SKIP_UPDATES

[root@outil01 database]# mkdir -p /u01/app/oracle/
[root@util01 database]# chown oracle:oinstall /u01/ -R
[root@util01 database]# su - oracle
[oracle@util01 database]$ pwd
/home/oracle/database
[oracle@util01 database]$ ./runInstaller -ignoreSysPrereqs -force -silent -responseFile /home/oracle/database/inst.rsp

9. configure database environment

create /usr/local/bin/oraset
[oracle@util01 bin]$ ls -l oraset
-rw-r--r-- 1 oracle oinstall 149 Aug 27 16:54 oraset
[oracle@util01 bin]$ more oraset
export PATH=$PATH:/home/oracle/bin
export NLS_DATE_FORMAT='DD-MON-YYYY HH24:MI:SS'
export ORACLE_SID=orcl

ORAENV_ASK=NO
. oraenv
unset ORAENV_ASK

[oracle@util01 bin]$ ls -l ~/.bashrc
-rw-r--r-- 1 oracle oinstall 149 Aug 27 16:54 /home/oracle/.bashrc

[oracle@util01 bin]$ more ~/.bashrc
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

# User specific aliases and functions

. /usr/local/bin/oraset >/dev/null 2>&1

Note: if you don't put >/dev/null 2>&1, the scp file to this host from other server may fail.

[root@util01 init.d]# more oracle
#!/bin/bash 
# chkconfig: 35 98 10
# description: Starts and stops Oracle processes 

# startup/shutdown sequence,  the lower the number is , the high priority it gets, for shutdown, 5 shutdown first, 10 shutdown later

LOCKFILE=/var/lock/subsys/oracle

case "$1" in 
  'start') 
    su - oracle -c "dbstart;lsnrctl start;emctl start dbconsole"
  touch $LOCKFILE
   ;; 
  'stop') 
    su - oracle -c "emctl stop dbconsole;lsnrctl stop;dbshut"
    rm -f $LOCKFILE
   ;; 
esac 

Note: those red color part is very important, otherwise, auto shutdown won't work although auto start will be working as Redhat-alike system doesn't think oracle is running when rebooting if it cannot see oracle in /var/lock/subsys/, so ignore it.

# chkconfig --add oracle
# chkconfig oracle --level 35 on

You also need to modify dbstart and dbshut file as follows:

[oracle@util01 bin]$ grep -B 1 ORACLE_HOME_LISTNER dbstart
ORACLE_HOME_LISTNER=${1:-$ORACLE_HOME}

[oracle@util01 bin]$ grep -B 1 ORACLE_HOME_LISTNER dbshut
ORACLE_HOME_LISTNER=${1:-$ORACLE_HOME}

[root@util01 init.d]# ls -l oracle
-rwxr-xr-x 1 root root 354 Aug 30 14:31 oracle

a more complete auto stop/start file is as follows: Please refer to http://tldp.org/HOWTO/Oracle-9i-Fedora-3-Install-HOWTO/sect_08.html
----------------

#!/bin/bash
#
# Run-level Startup script for the Oracle Instance and Listener
#
# chkconfig: 345 91 19
# description: Startup/Shutdown Oracle listener and instance

ORA_HOME="/u01/app/oracle/product/9.2.0.1.0"
ORA_OWNR="oracle"

# if the executables do not exist -- display error

if [ ! -f $ORA_HOME/bin/dbstart -o ! -d $ORA_HOME ]
then
        echo "Oracle startup: cannot start"
        exit 1
fi

# depending on parameter -- startup, shutdown, restart 
# of the instance and listener or usage display 

case "$1" in
    start)
        # Oracle listener and instance startup
        echo -n "Starting Oracle: "
        su - $ORA_OWNR -c "$ORA_HOME/bin/lsnrctl start"
        su - $ORA_OWNR -c $ORA_HOME/bin/dbstart
        touch /var/lock/subsys/oracle
        echo "OK"
        ;;
    stop)
  # Oracle listener and instance shutdown
        echo -n "Shutdown Oracle: "
        su - $ORA_OWNR -c "$ORA_HOME/bin/lsnrctl stop"
        su - $ORA_OWNR -c $ORA_HOME/bin/dbshut
        rm -f /var/lock/subsys/oracle
        echo "OK"
        ;;
    reload|restart)
        $0 stop
        $0 start
        ;;
    *)
        echo "Usage: $0 start|stop|restart|reload"
        exit 1
esac
exit 0


10. patch 11.2.0.3 to the latest patchset
download OPATH latest version first:
How To Download And Install The Latest OPatch Version [ID 274526.1]

install opatch

oracle@util01 ~]$ cd $ORACLE_HOME
[oracle@util01 db_1]$ mv OPatch/ OPatch.orig
[oracle@util01 db_1]$ unzip ~/p6880880_112000_Linux-x86-64.zip

11. Patch Oracle database to the latest pathset

sqlplus / as sysdba
sql> shutdown immediate;
sql> exit
lsnrctl stop
emctl stop dbconsole

[oracle@util01 13923374]$ export PATH=$PATH:/u01/app/oracle/product/11.2.0/db_1/OPatch
[oracle@util01 13923374]$ opatch apply

Check the readme.html file from the patch, run below for database upgrade.
cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
sql> connect / as sysdba
sql> startup
sql> @catbundle_PSU_ORCL_ROLLBACK.sql
sql> quit

check logfile for any errors: $ORACLE_HOME/cfgtoollogs/catbundle/catbundle_PSU_ORCL_ROLLBACK_timestamp.log

12. install Oracle VM Manager
Download OVMM original and patch 399 ISO
https://updates.oracle.com/ARULink/PatchDetails/process_form?aru=15337157

Preparing config.yml for silent OVMM installation

[root@util01 oracle]# more config.yml
db:
install : No
host : localhost
sys_password : oracle
port : 1521
sid : orcl
mgmt_port : 8080
user : ovs
password : oracle

webLogic:
install : Yes
user : weblogic
password : oracle

java:
install : Yes

ADF:
install : Yes

OVMCore:
install : Yes
user : admin
password : oracle

OVMConsole:
install : Yes

OVMShell:
install : Yes

onFailure:
cleanup : Yes

OVMUpgrade:
install : Yes

13  Preparing Oracle environment

[root@util01 ovmm]# mount Oracle_VM_Manager_3.1.1_64bit.iso ovmm -o loop
[root@util01 ovmm]# ./createOracle.sh
Adding group 'oinstall' with gid '54323' ...
groupadd: group 'oinstall' already exists
Adding group 'dba'
groupadd: group 'dba' already exists
Adding user 'oracle' with user id '54322', initial login group 'dba', supplementary group 'oinstall' and  home directory '/home/oracle' ...
User 'oracle' already exists ...
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba)
Creating user 'oracle' succeeded ...
Verifying user 'oracle' OS prerequisites for Oracle VM Manager ...
oracle    soft        nofile                    8192
oracle    hard       nofile                    8192
oracle    soft        nproc                    4096
oracle    hard       nproc                   4096
oracle    soft        core                       unlimited
oracle    hard       core                       unlimited
Setting  user 'oracle' OS limits for Oracle VM Manager ...
Altered file /etc/security/limits.conf
Original file backed up at /etc/security/limits.conf.orabackup
Verifying & setting of user limits succeeded ...
Modifying iptables for OVM
Adding rules to enable access to:
     7001  : Oracle VM Manager http
     7002  : Oracle VM Manager https
     15901 : Oracle VM Manager VM console proxy
     54321 : Oracle VM Manager core
     54322 : Oracle VM Manager core via SSL
       123 : NTP
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]
Rules added.

---
when OVMS and OVMM are not on the same network, port 7001 is needed ,
OVMM events shows BindingMismatchEvent critical python bindings mismatch .

14. actual installation of OVMM 3.1.1
/runInstaller.sh --config=/home/oracle/config.yml --installtype=production –assumeyes

15.  Upgrade Oracle VM manager to 399
mount iso file as -o loop, then
#./runUpgrader.sh as root


access OVM at  http://ip:7001/ovm/console

16. database archive log mode and rman
oracle@util01 u01]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.3.0 Production on Wed Aug 22 15:41:19 2012
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> alter system set log_archive_dest_1='location=/u01/oraarch' scope=both;
SQL> alter system set log_archive_format='%t_%s_%r.arc' scope=spfile;
SQL> shutdown immediate;
SQL> startup mount;
SQL> alter database archivelog;
SQL> alter database open;
SQL>exit;

$ rman target /
Rman> show all;
Rman> configure controlfile autobackup on;
Rman> backup database plus archivelog format ‘/u01/orarman/%U’

17. after renaming database host, re-generate em repository
modify the following files first.

./app/oracle/product/11.2.0/db_1/network/admin/listener.ora
/etc/hosts
/etc/sysconfig/network

[oracle@util01 bin]$ ./emca -deconfig dbcontrol db -repos drop
[oracle@util01 bin]$ ./emca -config dbcontrol db -repos create

18. known issue
If you configured vlan for em1 or bond0(em1 and em2), you might not able to start ovmm service with https port 7002.
The solutions is to disable ipv6 as follows:

a. put the following in /etc/modprobe/ipv6.conf
options ipv6 disable=1

b. append the following into /etc/sysconfig/network
NETWORKING_IPV6=off

19. references:

kickstart - http://en.wikipedia.org/wiki/Kickstart_(Linux)
http://fedoraproject.org/wiki/Anaconda/Kickstart
http://www.fatmin.com/2010/08/installing-dell-openmanage-server-administrator-on-linux.html
http://linux.dell.com/wiki/index.php/Repository/OMSA#Supported_Operating_Systems
http://www.fatmin.com/2010/08/installing-dell-openmanage-server-administrator-on-linux.html  4 steps to update dell firmware from Linux
http://folk.uio.no/trondham/software/check_openmanage.html#download  Nagios plugin to check dell openmanage

How to recover MS SQL 2008 database

Jephe Wu - http://linuxtechres.blogspot.com


Objective: understanding the recovery cases in MS SQL 2008
Environment: Windows 7 Professional SP1 64bit with 8G RAM, MS SQL server 2008 standard edition.



Prerequisites (refer to http://msdn.microsoft.com/en-us/library/dd207003.aspx for sql server 2012)
Perform the following tasks before you rebuild the system databases to ensure that you can restore the system databases to their current settings.
  1. Record all server-wide configuration values.
    SELECT * FROM sys.configurations;
    
  2. Record all service packs and hotfixes applied to the instance of SQL Server and the current collation. You must reapply these updates after rebuilding the system databases.
    SELECT
    SERVERPROPERTY('ProductVersion ') AS ProductVersion,
    SERVERPROPERTY('ProductLevel') AS ProductLevel,
    SERVERPROPERTY('ResourceVersion') AS ResourceVersion,
    SERVERPROPERTY('ResourceLastUpdateDateTime') AS ResourceLastUpdateDateTime,
    SERVERPROPERTY('Collation') AS Collation;
    
  3. Record the current location of all data and log files for the system databases. Rebuilding the system databases installs all system databases to their original location. If you have moved system database data or log files to a different location, you must move the files again.
    SELECT name, physical_name AS current_file_location
    FROM sys.master_files
    WHERE database_id IN (DB_ID('master'), DB_ID('model'), DB_ID('msdb'), DB_ID('tempdb'));
    
  4. Locate the current backup of the master, model, and msdb databases.
  5. If the instance of SQL Server is configured as a replication Distributor, locate the current backup of the distribution database.
  6. Ensure you have appropriate permissions to rebuild the system databases. To perform this operation, you must be a member of the sysadmin fixed server role. For more information, see Server-Level Roles.
  7. Verify that copies of the master, model, msdb data and log template files exist on the local server. The default location for the template files is C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\Binn\Templates. These files are used during the rebuild process and must be present for Setup to succeed. If they are missing, run the Repair feature of Setup, or manually copy the files from your installation media. To locate the files on the installation media, navigate to the appropriate platform directory (x86 or x64) and then navigate to setup\sql_engine_core_inst_msi\Pfiles\SqlServr\MSSQL.X\MSSQL\Binn\Templates.



Case 1:  Master database failure, but we can still startup with single user mode with master database.

Go to 'sql server configuration manager' - 'sql server service' - SQL server(MSSQLSERVER)- advanced tab, to append startup parameter with -m, so it becomes:

-m;-dC:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA\master.mdf;-eC:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\Log\ERRORLOG;-lC:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA\mastlog.ldf

stop all other services first, then start SQL server(MSSQLSERVER) service.

After starting up single user mode, you can use 'sqlcmd' in command prompt or SSMS to connect to it, when you use SSMS, remember not to login with database engine, cancel it, use 'new query' to get connected. Otherwise you might encounter error message "Only one administrator can connect at this time."

Now you can restore master database from your backup with the following command:


C:\Users\Ling Chen>sqlcmd
1> restore database master from disk = 'C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\Backup\master.bak'
2> go


Processed 376 pages for database 'master', file 'master' on file 1.
Processed 3 pages for database 'master', file 'mastlog' on file 1.
The master database has been successfully restored. Shutting down SQL Server.
SQL Server is terminating this process.


or just run:
restore database master from disk = 'C:\Program Files\Microsoft SQLServer\MSSQL10.MSSQLSERVER\MSSQL\Backup\master.bak'  in SSMS

FAQ:
a. C:\Users\Ling Chen>sqlcmd
Msg 18461, Level 14, State 1, Server FEIT3263, Line 1
Login failed for user 'FEIT3263\Ling Chen'. Reason: Server is in single user mod
e. Only one administrator can connect at this time.
Solution 1: change sql server to start it as your current Windows login user and password instead of system account.
Solution 2: logout current user although it belongs to administrator group, use 'Administrator' to login, then run 'sqlcmd'

Case 2: Master database corrupted,  we cannot even startup in single user mode

http://msdn.microsoft.com/en-us/library/dd207003(v=sql.100).aspx for how to rebuild db

In this case, we need to rebuild master database as follows:


Assuming MS SQL server 2008 standard edition installation disk is at H drive (Magic ISO to mount ISO file at H drive), the instance name is default MSSQLSERVER

H:\>setup /quiet /action=rebuilddatabase /instancename=MSSQLSERVER
Microsoft (R) SQL Server 2008 Setup 10.00.1600.22
Copyright (c) Microsoft Corporation.  All rights reserved.

The following error occurred:
Missing system administrator account. To continue, provide at least one Windows
account to provision as a SQL Server system administrator.

Error result: -2068578304
Result facility code: 1204
Result error code: 0

Please review the summary.txt log for further details


H:\>setup /quiet /action=rebuilddatabase /instancename=MSSQLSERVER /sqlsysadminaccounts=Administrator
Microsoft (R) SQL Server 2008 Setup 10.00.1600.22
Copyright (c) Microsoft Corporation.  All rights reserved.



try to restore master db from SSMS (in single user mode) after rebuild systemdb:
Msg 3154, Level 16, State 4, Line 1
The backup set holds a backup of a database other than the existing 'master' database.
Msg 3013, Level 16, State 1, Line 1
RESTORE DATABASE is terminating abnormally.

-----------
Updated on 26/01/2013:
For SQL server 2012 rebuild system database howto, please refer to http://msdn.microsoft.com/en-us/library/dd207003.aspx

Setup /QUIET /ACTION=REBUILDDATABASE /INSTANCENAME=InstanceName /SQLSYSADMINACCOUNTS=accounts [ /SAPWD= StrongPassword ] [ /SQLCOLLATION=CollationName]

-----------------

restore database master from disk = 'C:\Program Files\Microsoft SQLServer\MSSQL10.MSSQLSERVER\MSSQL\Backup\master.bak' with replace

After rebuilding master/system db, you can startup in single user mode, follow case 1.


Oracle database SCN and Recovery



Background:  Oracle 11.2 64bit with Rman
Objective: understanding the relationship between SCN and Rman recovery.


Diagram:  3 SCNs in controlfile and 1 SCN in individual datafile 






Cases:


1. when database is running normally, stop scn is null
SQL> show parameter FAST_START_MTTR_TARGET;

NAME     TYPE VALUE
------------------------------------ ----------- ------------------------------
fast_start_mttr_target     integer 0
SQL> select checkpoint_change# from v$database;

CHECKPOINT_CHANGE#
------------------
  1828954

SQL> select name,checkpoint_change# from v$datafile where name like '%users01%';

NAME
--------------------------------------------------------------------------------
CHECKPOINT_CHANGE#
------------------
/home/oracle/app/oracle/oradata/testdb/users01.dbf
  1828954


SQL> select name,checkpoint_change# from v$datafile_header where name like '%users01%';

NAME
--------------------------------------------------------------------------------
CHECKPOINT_CHANGE#
------------------
/home/oracle/app/oracle/oradata/testdb/users01.dbf
  1828954


SQL> select name,last_change# from v$datafile where name like '%users01%';

NAME
--------------------------------------------------------------------------------
LAST_CHANGE#
------------
/home/oracle/app/oracle/oradata/testdb/users01.dbf

2. clean shutdown shows all 4 numbers are same
SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.

Total System Global Area  217157632 bytes
Fixed Size    2211928 bytes
Variable Size  159387560 bytes
Database Buffers   50331648 bytes
Redo Buffers    5226496 bytes
Database mounted.
SQL> select checkpoint_change# from v$database;

CHECKPOINT_CHANGE#
------------------
  1840147

SQL> select name,checkpoint_change# from v$datafile_header where name like '%users01%';

NAME
--------------------------------------------------------------------------------
CHECKPOINT_CHANGE#
------------------
/home/oracle/app/oracle/oradata/testdb/users01.dbf
  1840147


SQL> select name,checkpoint_change#,last_change# from v$datafile where name like '%users01%';

NAME
--------------------------------------------------------------------------------
CHECKPOINT_CHANGE# LAST_CHANGE#
------------------ ------------
/home/oracle/app/oracle/oradata/testdb/users01.dbf
  1840147 1840147

3. simulate a media failure which need media recovery

SQL> select name,checkpoint_change#,last_change# from v$datafile;

NAME



CHECKPOINT_CHANGE# LAST_CHANGE#

/home/oracle/app/oracle/oradata/testdb/system01.dbf

 

1846335
/home/oracle/app/oracle/oradata/testdb/sysaux01.dbf

 

1846335
/home/oracle/app/oracle/oradata/testdb/undotbs01.dbf

 

1846335
/home/oracle/app/oracle/oradata/testdb/users01.dbf

 

1846335
/home/oracle/app/oracle/oradata/testdb/example01.dbf

 

1846335

SQL> select checkpoint_change# from v$database;

CHECKPOINT_CHANGE#
------------------
  1846335

RMAN> sql 'alter database datafile 4 offline';

sql statement: alter database datafile 4 offline

RMAN> restore datafile 4 ;

Starting restore at 04-AUG-12
using channel ORA_DISK_1

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00004 to /home/oracle/app/oracle/oradata/testdb/users01.dbf
channel ORA_DISK_1: reading from backup piece

/home/oracle/app/oracle/product/11.2.0/dbhome_1/dbs/05nhr2ul_1_1
channel ORA_DISK_1: piece handle=/home/oracle/app/oracle/product/11.2.0/dbhome_1/dbs/05nhr2ul_1_1

tag=TAG20120804T214452
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
Finished restore at 04-AUG-12



SQL> select * from v$recover_file;

     FILE# ONLINE  ONLINE_
---------- ------- -------
ERROR     CHANGE#
----------------------------------------------------------------- ----------
TIME
---------
4 OFFLINE OFFLINE
    1846297
04-AUG-12


SQL> alter database datafile 4 online;
alter database datafile 4 online
*
ERROR at line 1:
ORA-01113: file 4 needs media recovery
ORA-01110: data file 4: '/home/oracle/app/oracle/oradata/testdb/users01.dbf'


SQL> set line 32000
SQL> r
  1* select checkpoint_change#,last_change#,file#,name from v$datafile

CHECKPOINT_CHANGE# LAST_CHANGE#      FILE# NAME

  1856063 1856063 1 /home/oracle/app/oracle/oradata/testdb/system01.dbf
  1856063 1856063 2 /home/oracle/app/oracle/oradata/testdb/sysaux01.dbf
  1856063 1856063 3 /home/oracle/app/oracle/oradata/testdb/undotbs01.dbf
  1846335 1850112 4 /home/oracle/app/oracle/oradata/testdb/users01.dbf
  1856063 1856063 5 /home/oracle/app/oracle/oradata/testdb/example01.dbf

SQL> select * from v$datafile_header;

     FILE# STATUS  ERROR FORMAT REC FUZ

CREATION_CHANGE# CREATION_ TABLESPACE_NAME TS# RFILE# RESETLOGS_CHANGE# RESETLOGS

CHECKPOINT_CHANGE# CHECKPOIN CHECKPOINT_COUNT      BYTES BLOCKS NAME



SPACE_HEADER   LAST_DEALLOC_CHA UNDO_OPT_CURRENT_CHANGE#

1 ONLINE     10 NO  NO

       7 15-AUG-09 SYSTEM   0      1  1815157 06-MAY-12  

  1856063 05-AUG-12      244  723517440  88320 /home/oracle/app/oracle/oradata/testdb/system01.dbf

 4194306

  1013911    1811966
2 ONLINE     10 NO  NO

    2140 15-AUG-09 SYSAUX   1      2  1815157 06-MAY-12  

  1856063 05-AUG-12      244  660602880  80640 /home/oracle/app/oracle/oradata/testdb/sysaux01.dbf

 8388610

  1808221
3 ONLINE     10 NO  NO

  942603 15-AUG-09 UNDOTBS1   2      3  1815157 06-MAY-12  

  1856063 05-AUG-12      169  110100480  13440 /home/oracle/app/oracle/oradata/testdb/undotbs01.dbf

 12582914

  1771742    1811966
4 OFFLINE     10 YES NO

   17993 15-AUG-09 USERS   4      4  1815157 06-MAY-12  

  1846297 04-AUG-12      241    5242880    640 /home/oracle/app/oracle/oradata/testdb/users01.dbf

 16777218
5 ONLINE     10 NO  NO

  973209 03-NOV-11 EXAMPLE   6      5  1815157 06-MAY-12  

  1856063 05-AUG-12      165  104857600  12800 /home/oracle/app/oracle/oradata/testdb/example01.dbf

 20971522

  1387524

[oracle@jephe ~]$ rman target /

Recovery Manager: Release 11.2.0.1.0 - Production on Sun Aug 5 02:11:28 2012

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

connected to target database: TESTDB (DBID=2565863629)

RMAN> recover datafile 4;

Starting recover at 05-AUG-12
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=34 device type=DISK

starting media recovery
media recovery complete, elapsed time: 00:00:00

Finished recover at 05-AUG-12

RMAN> exit


Recovery Manager complete.
[oracle@jephe ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on Sun Aug 5 02:11:41 2012

Copyright (c) 1982, 2009, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> alter database datafile 4 online;

Database altered.
SQL> select checkpoint_change# from v$database;

CHECKPOINT_CHANGE#
------------------
  1856066

SQL> select file#,name,checkpoint_change#,last_change# from v$datafile;

     FILE# NAME

 

CHECKPOINT_CHANGE# LAST_CHANGE#
----------

1 /home/oracle/app/oracle/oradata/testdb/system01.dbf



1856066
2 /home/oracle/app/oracle/oradata/testdb/sysaux01.dbf



1856066
3 /home/oracle/app/oracle/oradata/testdb/undotbs01.dbf



1856066
4 /home/oracle/app/oracle/oradata/testdb/users01.dbf



1856340
5 /home/oracle/app/oracle/oradata/testdb/example01.dbf



1856066


SQL> select checkpoint_change# from v$database;

CHECKPOINT_CHANGE#
------------------
  1856385

SQL> select file#,name,checkpoint_change#,last_change# from v$datafile;

     FILE# NAME

 

CHECKPOINT_CHANGE# LAST_CHANGE#

1 /home/oracle/app/oracle/oradata/testdb/system01.dbf



1856385
2 /home/oracle/app/oracle/oradata/testdb/sysaux01.dbf



1856385
3 /home/oracle/app/oracle/oradata/testdb/undotbs01.dbf



1856385
4 /home/oracle/app/oracle/oradata/testdb/users01.dbf



1856385
5 /home/oracle/app/oracle/oradata/testdb/example01.dbf



1856385


SQL> select * from v$recover_file;

no rows selected


4.  check controlfile content 
SQL> oradebug setmypid
Statement processed.
SQL> oradebug unlimit;
Statement processed.
SQL> alter session set events 'immediate trace name controlf level 9';

Session altered.

SQL> oradebug tracefile_name;
/home/oracle/app/oracle/diag/rdbms/testdb/testdb/trace/testdb_ora_12972.trc
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production


[oracle@jephe ~]$ grep ' Database checkpoint' /home/oracle/app/oracle/diag/rdbms/testdb/testdb/trace/testdb_ora_12972.trc
 Database checkpoint: Thread=1 scn: 0x0000.001c5381
[oracle@jephe ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on Sun Aug 5 03:17:30 2012

Copyright (c) 1982, 2009, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select checkpoint_change# from v$database;

CHECKPOINT_CHANGE#
------------------
  1856385


[oracle@jephe ~]$ egrep "ENTRY|RECORD"

/home/oracle/app/oracle/diag/rdbms/testdb/testdb/trace/testdb_ora_12972.trc
DATABASE ENTRY
CHECKPOINT PROGRESS RECORDS
EXTENDED DATABASE ENTRY
REDO THREAD RECORDS
LOG FILE RECORDS
DATA FILE RECORDS
TEMP FILE RECORDS
TABLESPACE RECORDS
RMAN CONFIGURATION RECORDS
FLASHBACK LOGFILE RECORDS
THREAD INSTANCE MAPPING RECORDS
MTTR RECORDS
STANDBY DATABASE MAP RECORDS
RESTORE POINT RECORDS
ACM SERVICE RECORDS
LOG FILE HISTORY RECORDS
OFFLINE RANGE RECORDS
ARCHIVED LOG RECORDS
FOREIGN ARCHIVED LOG RECORDS
BACKUP SET RECORDS
BACKUP PIECE RECORDS
BACKUP DATAFILE RECORDS
BACKUP LOG RECORDS
DATAFILE COPY RECORDS
BACKUP DATAFILE CORRUPTION RECORDS
DATAFILE COPY CORRUPTION RECORDS
DELETION RECORDS
PROXY COPY RECORDS
INCARNATION RECORDS
RMAN STATUS RECORDS
DATAFILE HISTORY RECORDS
NORMAL RESTORE POINT RECORDS
DATABASE BLOCK CORRUPTION RECORDS