Reboot or Shutdown a Solaris System

The best and safest command to do a clean shutdown or a reboot of a Solaris server is the 2 commands below. There are many commands with which you can achieve this. But if you have many scripts in init.d for application like Oracle DBand Oracle RAC. Then shutdown command is the way to go, also it prompts the users to save and shutdown their sessions.

To Reboot

shutdown -y -i6 -g0

To shutdown

shutdown -y -i5 -g0

Create Virtual Interface Solaris 10

Ocassionally you need to create a quick one node RAC setup for testing and one of the requirements is a private virtual interface. Virtual interfaces allow a single ethernet interface to listen on additional IP addresses. You can quickly do it in Solaris 10 so that the grid infrastructure setup can see the private virtual interconnect. You can do as below

After adding entry for private interconnect in etc/hosts file you can proceed.

Given an ethernet interface bge0 (use ifconfig -a to identify the names of your interfaces), you can create a subinterface called bge0:1 with the following command:

— Configure Virtual Interface Solaris 10–

Virtual interfaces allow a single ethernet interface to listen on additional IP addresses.

Given an ethernet interface bge0 (use ifconfig -a to identify the names of your interfaces), you can create a subinterface called bge0:1 with the following command:

# dladm show-dev
bge0 link: up speed: 1000 Mbps duplex: full
bge1 link: unknown speed: 0 Mbps duplex: unknown
bge2 link: unknown speed: 0 Mbps duplex: unknown
bge3 link: unknown speed: 0 Mbps duplex: unknown

# dladm show-link
bge0 type: non-vlan mtu: 1500 device: bge0
bge1 type: non-vlan mtu: 1500 device: bge1
bge2 type: non-vlan mtu: 1500 device: bge2
bge3 type: non-vlan mtu: 1500 device: bge3

ifconfig bge0:1 plumb

You can set the IP address of the interface to 10.90.1.102 and turn on the interface with the following command:

ifconfig bge0:1 10.90.1.102 up

ifconfig -a

lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
bge0: flags=1000843 mtu 1500 index 2
inet 172.*.*.* netmask ffffff00 broadcast 172.*.*.*
ether 0:14:4f:95:e1:38

Unless you do some additional nonstandard things in your network, all of the subinterfaces on a physical interface need to be in the same subnet.

To make the virtual interface persist following a reboot, you can add the ip address or hostame from /etc/hosts in the file /etc/hostname.bge0:1

more /etc/hosts

#
# Internet host table
#
::1 localhost

10.90.1.102 hostname-priv

vi /etc/hostname.bge0:1

Add the entry hostname-priv from the host file to the virtual interface suncsftst-priv. And now it will persis after a reboot also.

Monitor User Processes and Send Email Shell Script Solaris10

Sometimes on a system it is important to know the maximum user processes does not increase so much that it consumes all shared memory. Recently we had an incident in our ORganization where a rogue application bug caused user processes to spike upto 20,000 and it consumed all memory and swap and the entire server crashed.  If we pro-actively monitored the system for increasing user processes we could have averted this disaster. Below is script to monitor user processes and send email.

swx – is name process name

500- threshold for permitted processes

#!/bin/ksh
VAL=`ps -ef | grep -c swx | awk ‘{if ( $1 > 500 ) print $1 }’`

if [ “$VAL” -ge “500” ]; then
echo “$VAL user processes found running with SWX user. Exceeded the Threshold,
Please take immediate action.” | /usr/bin/mailx -s “Alert !! $VAL SWX Applicati
on processes” email@test.com
fi

exit 0

You can add this to crontab like below:

* * * * * /u02/scripts/userprocmon.sh >/dev/null 2>&1

Failover Solaris Zone Cluster

root@swxapp1 # clrg status

=== Cluster Resource Groups ===

Group Name     Node Name           Suspended    Status
———-     ———           ———    ——
swxapp-rg      swxnode1:swxapp1    No           Online
swxnode2:swxapp2    No           Offline

root@swxapp1 # clrg switch -n swxnode2:swxapp2 swxapp-rg

Configuring DNS on Solaris 10 (Joining Solaris10 server to Domain)

################ Configure Solaris 10 machine to be on DNS##############

1. Check for file resolv.conf in /etc directory, if it doenst exist create it

2. add below information to the resolv.conf file

domain mydomain.com
nameserver 172.20.4.10
nameserver 172.20.4.166
search mydomain.com

3. Check for file nsswitch.conf in /etc directory

hosts:      files
ipnodes:    files

## edit the above 2 lines and add ‘dns’ in front of file

hosts:      files dns
ipnodes:    files dns

save and exit

$ nslookup swxscan

Server:         172.20.4.10
Address:        172.20.4.10#53

Name:   swxscan.mydomain.com
Address: 172.21.51.18
Name:   swxscan.mydomain.com
Address: 172.21.51.17
Name:   swxscan.mydomain.com
Address: 172.21.51.19

Voila

Avoid out of memory error on Solaris 10 for Oracle Database

Out of Memory Problems on Oracle 10 / Solaris 10

Many kernel parameters have been replaced by so called resource controls in Solaris 10. It is possible to change resource controls using the prctl command. All shared memory and semaphore settings are now handled via resource controls, so any entries regarding shared memory or semaphores (shm & sem) in /etc/system will be ignored.

Here is the procedure we followed to modify the kernel parameters on Solaris 10 / Oracle 10.2.0.2.

Unlike earlier releases of Solaris, most of the system parameters needed to run Oracle are already set properly, so the only one you need is the maximum shared memory parameter. In earlier versions this was called SHMMAX and was set by editing the /etc/system file and rebooting. With Solaris 10 you set this by modifying a «Resource Control Value». You can do this temporarily by using prctl, but that is lost at reboot so you will need to add the command to the oracle user’s .profile.

The other option is to create a default project for the oracle user.

# projadd -U oracle -K \
“project.max-shm-memory=(priv,4096MB,deny)” user.oracle

What this does:

Makes a project named “user.oracle” in /etc/project with the user oracle as it’s only member.

# cat /etc/project

system:0::::
user.root:1::::
noproject:2::::
default:3::::
group.staff:10::::
user.oracle:100::oracle::project.max-shm-memory
=(priv,4294967296,deny)

Because the name was of the form “user.username” it becomes the oracle user’s default project.

The value of the maximum shared memory is set to 4GB, you might want to use a larger value here if you have more memory and swap.

No reboot is needed, the user will get the new value
at their next login.

Now you can also modify the max-sem-ids Parameter:

# projmod -s -K “project.max-sem-ids=(priv,256,deny)” \
user.oracle

Check the Paramters as User oracle

$ prctl -i project user.oracle

project: 100: user.oracle
NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT
project.max-contracts
privileged 10.0K – deny –
system 2.15G max deny –
project.max-device-locked-memory
privileged 125MB – deny –
system 16.0EB max deny –
project.max-port-ids
privileged 8.19K – deny –
system 65.5K max deny –
project.max-shm-memory
privileged 4.00GB – deny –
system 16.0EB max deny –
project.max-shm-ids
privileged 128 – deny –
system 16.8M max deny –
project.max-msg-ids
privileged 128 – deny –
system 16.8M max deny –
project.max-sem-ids
privileged 256 – deny –
system 16.8M max deny –
project.max-crypto-memory
privileged 498MB – deny –
system 16.0EB max deny –
project.max-tasks
system 2.15G max deny –
project.max-lwps
system 2.15G max deny –
project.cpu-shares
privileged 1 – none –
system 65.5K max none –
zone.max-lwps
system 2.15G max deny –
zone.cpu-shares
privileged 1 – none –

Oracle Database 11g installation on Solaris 10

I had installed Oracle Database 11g on Solaris 10 few days back. I thought of documenting the steps for easy reference. Please find below steps for the same

Installation Pre-Requisite

Refer to Oracle Database Installation Guide 11g Release 1 (11.1) for Solaris Operating System for checking Hardware and Software Requirements.

User Creation and Environment Settings

1)Create groups for Oracle account

#groupadd oinstall
#groupadd dba
#groupadd oper

2)Create Oracle Default Home directory

# mkdir /export/home
# mkdir /export/home/oracle

3)Create Oracle user

# useradd -g oinstall -G dba -d /export/home/oracle -s /usr/bin/bash oracle
# chown oracle:oinstall /export/home/oracle

4)Create Project for Oracle for setting the kernel parameters

In case of Solaris 10, you can use projects to configure the kernel parameters instead of /etc/system file. This can be done as following

# projadd -U oracle -K "project.max-shm-memory=(priv,4g,deny)" oracle
# projmod -sK "project.max-sem-nsems=(priv,256,deny)" oracle
# projmod -sK "project.max-sem-ids=(priv,100,deny)" oracle
# projmod -sK "project.max-shm-ids=(priv,100,deny)" oracle

There are many more ways of creating project entries such as group.group-name or user.user-name. For more details refer to Solaris Administration documents.

Update

Last three settings made by projmod command are not required as these values are lower than the default. This was pointed by Mike Madland and he also gave a Sun documentation link

http://docs.sun.com/app/docs/doc/819-2724/6n50b0795?l=en&a=view#chapter1-33
You can check the values for max-sem-ids and max-shm-ids with this command:

prctl -n project.max-sem-ids -i task `ps -o taskid= -p $$`

5)Create .bash_profile for Oracle user

#Oracle Environment Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_BASE=/u03/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.1.0/db_1; export ORACLE_HOME
ORACLE_SID=TESTDB11G; export ORACLE_SID
PATH=$PATH:/usr/local/bin:/usr/ccs/bin:/usr/sfw/bin:$ORACLE_HOME/bin

Now Set the Display to a X-windowing enabled system.

$ export DISPLAY=192.168.4.47:0.0

Also allow the host to accept the connection by

$xhost +

Oracle Software Installation

Go to the Oracle dump location and run runInstaller as Oracle user

$./runInstaller

This will open Oracle Universal Installer(OUI) screen. If Oracle Universal Installer is not displayed, then ensure DISPLAY variable is set correctly. Select “Software only” option and install the software. If any of the pre-requisite’s are not met , then installation will fail. You would be required to make necessary changes to proceed.

Database Creation

We will be using ASM for the Database files. For this we need to perform some configuration

1)Prepare the Raw device for using as ASM Disks

# ls -l
total 0
crw------- 1 root root 125, 1 Jun 20 10:39 1

Disk should be owned by Oracle user and should have permission set to 660

# chown oracle:dba 1
# chmod 660 1

- # ls -ltr
total 0
crw-rw---- 1 oracle dba 125, 1 Jun 20 10:39 1

2)Configure CSS Service

In case of Solaris 10, we need to use Service Management Facility (SMF) for configuring CSS service else it will not start. Refer to my earlier post for this step i.e 11.1.0.6 ASM installation on Solaris fails -II

3) Configure ASM Instance

a)Go to $ORACLE_HOME/bin

b)Execute dbca from this directory (ensure dbca is properly set)

$./dbca

c) Select Configure ASM Instance option. This will create ASM instance for you. After this you can create Diskgroups using GUI or else use sqlplus to do the same.

4)Now continue creating database normally and enter Diskgroup Name after selecting Oracle Managed files as database file location.

While you navigate through GUI screens, it will prompt you to Specifying Security Settings

– Keep the enhanced 11g security settings(recommended)
– Revert to pre 11g settings

Select the 11g settings which will enable Auditing by default and also enable Case sensitive passwords with Stronger password hashing algorithm.

I have not discussed GUI screens for DBCA and OUI in this article. These are pretty much standard screens. In case you need more information about it, then you can refer to : Oracle 11g Install guide for Solaris

Install Oracle10gR2 on Solaris 10 in few easy steps.

Directions :: First do gunzip then cpio of the file you downloaded from oracle.com
1. Run “gunzip <filename>” on all the files.   Eg.: gunzip solarisx86_DB_10_1_0_3_Disk1.cpio.gz
2. Extract the cpio archives with the command “cpio -idcmv < filename>”

Eg.: cpio -idcmv <solarisx86_DB_10_1_0_3_Disk1.cpio

A. Create Oracle Group and User:
—————-
A. create group name “dba”, oracle inventory group “oinstall” and “oracle” user

# /usr/sbin/groupadd oinstall
# /usr/sbin/groupadd dba

{Determine oracle user exist or not
# id -a oracle
{if exist, should be look like this=
uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)

{create oracle user=
# useradd -d /export/home/oracle -g dba -G oinstall -m -s /bin/ksh oracle

{set password=

# passwd -r files oracle

/*to delete user if it exists command is “userdel”

#mkdir /export/home/oracle
#chown oracle:dba /export/home/oracle

{to determine nobody user=
# id nobody
# /usr/sbin/useradd nobody >>run if does not exist

B. EDIT FILE /export/home/oracle/.profile
————————————–
umask 022
TMP=/tmp
TMPDIR=$TMP
DISPLAY=localhost:0.0
export TMP TMPDIR DISPLAY
ORACLE_BASE=/u01/app/oracle [replace with ur Oracle base Directory]
ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1 [replace with ur Oracle home Directory]

ORACLE_SID=jktdb [replace with your database]
PATH=$ORACLE_HOME/bin:$PATH
export ORACLE_BASE ORACLE_HOME ORACLE_SID PATH

C. Configure Kernel Parameter

AS A ROOT USER:-

projadd -U oracle -K “project.max-shm-memory=(priv,4096MB,deny)” user.oracle

AS A ORACLE USER:-

# su – oracle

$ prctl -n project.max-shm-memory $$

For User details:-

AS A ROOT USER:-

# more /etc/project

D. Make Directories in /u01 and give permissions to /u01

# mkdir /u01/app/oracle

#mkdir /u01/app/oracle/product/10.2.0/db_ 1

Give oracle:dba to /u01/app/oracle/product/10.2.0/

AS ROOT:-

# chmod -R 777 /u01

# chown –R oracle:dba /u01

Put a parameter –f if few files not changing ownership

E. Performing the Oracle Installation

$su – oracle

$xhost +

$export DISPLAY=localhost:0.0

$xhost +localhost

/*localhost is the ip of your current machine or laptop from where you are installing*/

$ cd /export/home/database/ [the source unzipped here]

As Root user:

# chown oracle:dba /export/home/database/runInstaller

./runInstaller

F. Finishing Installation

Run the 2 scripts as root user

When you run root.sh and this message comes, just hit enter

Enter the full pathname of the local bin directory: [/usr/local/bin]:

G. Post Installation Tasks

Set $ORACLE_HOME and $ORACLE_BASE

As a Root User

# ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1

# export ORACLE_HOME

# cd $ORACLE_HOME

# ORACLE_BASE=/u01/app/oracle

# export ORACLE_BASE

# cd $ORACLE_BASE

You can run dbca and create the database if you chose not to install the database while using the runinstaller.

Simply from any Xwindows client. I prefer using NetSarangs Xbrowser.

$  su – oracke

$ dbca

And continue with creating the database…

Add disk Solaris

How to configure just newly added disks in Solaris 10? It’s really simple.
1. first show all attached targets

bash-3.00# cfgadm -al

Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 disk         connected    configured   unknown
c0::dsk/c0t1d0                 disk         connected    configured   unknown
c0::dsk/c0t6d0                 CD-ROM       connected    configured   unknown
c1                             fc-fabric    connected    configured   unknown
c1::200500a0b8169d18           disk         connected    configured   unknown
c1::5006016008065109           disk         connected    configured   unknown
c2                             fc-fabric    connected    configured   unknown
c2::200400a0b8169d18           disk         connected    configured   unknown
c2::5006016108065109           disk         connected    unconfigured unknown
c2::5006016908065109           disk         connected    configured   unknown
c2::500604843d489c84           disk         connected    configured   unknown
c3                             scsi-bus     connected    unconfigured unknown

bash-3.00#

2.  now I attached SCSI JBOD to c3 – let’s configure them

bash-3.00# cfgadm -c configure c3
bash-3.00# cfgadm -al

Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 disk         connected    configured   unknown
c0::dsk/c0t1d0                 disk         connected    configured   unknown
c0::dsk/c0t6d0                 CD-ROM       connected    configured   unknown
c1                             fc-fabric    connected    configured   unknown
c1::200500a0b8169d18           disk         connected    configured   unknown
c1::5006016008065109           disk         connected    configured   unknown
c2                             fc-fabric    connected    configured   unknown
c2::200400a0b8169d18           disk         connected    configured   unknown
c2::5006016108065109           disk         connected    unconfigured unknown
c2::5006016908065109           disk         connected    configured   unknown
c2::500604843d489c84           disk         connected    configured   unknown
c3                             scsi-bus     connected    configured   unknown
c3::dsk/c3t8d0                 disk         connected    configured   unknown
c3::dsk/c3t9d0                 disk         connected    configured   unknown
c3::dsk/c3t10d0                disk         connected    configured   unknown
c3::dsk/c3t11d0                disk         connected    configured   unknown
c3::dsk/c3t12d0                disk         connected    configured   unknown
c3::dsk/c3t13d0                disk         connected    configured   unknown
c3::dsk/c3t14d0                disk         connected    configured   unknown

3. Now we have to create a partition using format utility and then create a new filesystem for it.

format>

AVAILABLE DISK SELECTIONS:
0. c0d0 <DEFAULT cyl 4092 alt 2 hd 128 sec 32>
/pci@0,0/pci-ide@7,1/ide@0/cmdk@0,0
1. c0d1 <DEFAULT cyl 4092 alt 2 hd 128 sec 32>
/pci@0,0/pci-ide@7,1/ide@0/cmdk@1,0
2. c1d1 <DEFAULT cyl 2556 alt 2 hd 128 sec 32>
/pci@0,0/pci-ide@7,1/ide@1/cmdk@1,0
3. c2t0d0 <DEFAULT cyl 4092 alt 2 hd 128 sec 32>
/pci@0,0/pci15ad,1976@10/sd@0,0
4. c2t1d0 <DEFAULT cyl 3914 alt 2 hd 255 sec 63>
/pci@0,0/pci15ad,1976@10/sd@1,0
Specify disk (enter its number): 4

format>fdisk
No fdisk table exists. The default partition for the disk is:

a 100% “SOLARIS System” partition

Type “y” to accept the default partition,  otherwise type “n” to edit the
partition table.
y

format>partition         //alternately you can quit format, select disk number again and enter ‘p’ twice
format>print

Part      Tag    Flag     Cylinders        Size            Blocks
0 unassigned    wm       0               0         (0/0/0)           0
1 unassigned    wm       0               0         (0/0/0)           0
2     backup    wu       0 – 3912       29.98GB    (3913/0/0) 62862345
3 unassigned    wm       0               0         (0/0/0)           0
4 unassigned    wm       0               0         (0/0/0)           0
5 unassigned    wm       0               0         (0/0/0)           0
6 unassigned    wm       0               0         (0/0/0)           0
7 unassigned    wm       0               0         (0/0/0)           0
8       boot    wu       0 –    0        7.84MB    (1/0/0)       16065
9 unassigned    wm       0               0         (0/0/0)           0

partition> 0
Part      Tag    Flag     Cylinders        Size            Blocks
0 unassigned    wm       0               0         (0/0/0)           0

Enter partition id tag[unassigned]:
Enter partition permission flags[wm]:
Enter new starting cyl[0]:
Enter partition size[0b, 0c, 0e, 0.00mb, 0.00gb]: 29.98gb
`29.98gb’ is out of range
Enter partition size[0b, 0c, 0e, 0.00mb, 0.00gb]: 29.00gb
partition> p
Current partition table (unnamed):
Total disk cylinders available: 3913 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders        Size            Blocks
0 unassigned    wm       0 – 3785       29.00GB    (3786/0/0) 60822090
1 unassigned    wm       0               0         (0/0/0)           0
2     backup    wu       0 – 3912       29.98GB    (3913/0/0) 62862345
3 unassigned    wm       0               0         (0/0/0)           0
4 unassigned    wm       0               0         (0/0/0)           0
5 unassigned    wm       0               0         (0/0/0)           0
6 unassigned    wm       0               0         (0/0/0)           0
7 unassigned    wm       0               0         (0/0/0)           0
8       boot    wu       0 –    0        7.84MB    (1/0/0)       16065
9 unassigned    wm       0               0         (0/0/0)           0

partition> label
Ready to label disk, continue? yes

format>quit

$ newfs /dev/rdsk/c2t1d0s0
bash-3.00# newfs /dev/rdsk/c2t1d0s0
newfs: construct a new file system /dev/rdsk/c2t1d0s0: (y/n)? y
Warning: 3510 sector(s) in last cylinder unallocated
/dev/rdsk/c2t1d0s0:     60822090 sectors in 9900 cylinders of 48 tracks, 128 sectors

$ fsck /dev/rdsk/c2t1d0s0
** /dev/rdsk/c2t1d0s0
** Last Mounted on
** Phase 1 – Check Blocks and Sizes
** Phase 2 – Check Pathnames
** Phase 3a – Check Connectivity
** Phase 3b – Verify Shadows/ACLs
** Phase 4 – Check Reference Counts
** Phase 5 – Check Cylinder Groups
2 files, 9 used, 29950474 free (18 frags, 3743807 blocks, 0.0% fragmentation)

4. Now the final step is mounting the filesystem we created.

And then mount the partition. In this case, I’m making a /rman partition:
bash-3.00# mkdir /rman
bash-3.00# mount /dev/dsk/c2t1d0s0 /rman
bash-3.00# df -h

Next, add the proper line to vfstab, else it will unmount when you reboot the server.

$ vi /etc/vfstab

/dev/dsk/c2t1d0s0 /dev/rdsk/c2t1d0s0     /rman ufs     2       yes     –

That’s it, Simple and easy!