Use ACL and SGID to make a group of users to work on the same directory and Linux file permission

Jephe Wu - http://linuxtechres.blogspot.com

Objective:  a parent directory called /shared is owned by db2inst1 (umask 022 only),  and 2 other users called jephe and zhitan are going to work on the this directory /shared. They should be able to read/write any files and directories under /shared each other, also be able to read/write files created by db2inst1 which means any files created by db2inst1 will have group-writable permission although its umask is 022.

Environment: RHEL 5 or CentOS 5

Preparation:
You can mount the partition with acl option by using 'mount / -o remount,acl'.
or

cd /path/to/directory; df . -> find out which mount directory, let's say it's /data, then mount /data -o remount,acl



Steps:
1. configure the permission 
When a new file is created on a Unix-like system, its permissions are determined from the umask of the process that created it.

[root@linuxtest /]# ls -ld shared  (let its group to have write permission so that jephe and zhitan can create files and directories under /shared, but any files created by db2inst1 itself will not be group writable because its umask is 022)
drwxrwsr-x 2 db2inst1 its 4096 Feb 20 10:22 shared

[root@linuxtest /]# usermod -G its jephe
[root@linuxtest /]# id jephe

uid=500(jephe) gid=500(jephe) groups=500(jephe),502(its)
[root@linuxtest /]# usermod -G its zhitan
[root@linuxtest /]# id zhitan
uid=501(zhitan) gid=501(zhitan) groups=501(zhitan),502(its)


 2. configure default acl permissions
[root@linuxtest /]# setfacl -d -m u:jephe:rwx,u:zhitan:rwx /shared
[root@linuxtest /]# setfacl -R -m u:jephe:rwx,u:zhitan:rwx /shared

Note:
The default ACL will be applied only for newly created files or directories under /shared directory. If you use only setfacl -d for the parent directory and user jephe and zhitan are still not able to write for this directory,

[root@linuxtest /]# getfacl shared
# file: shared
# owner: db2inst1
# group: its
user::rwx
group::rwx
other::r-x
default:user::rwx
default:user:jephe:rwx
default:user:zhitan:rwx
default:group::rwx
default:mask::rwx  (the mask defines the maximum permissions that can be given to users or group, if it's --x, means the user and group can get maximum permission is --x, if the user or group itself doesn't have x permission, then mean user and group will get effective permission as ---, aka nothing)
default:other::r-x

Now, any files or directories created by db2inst1 will have group writable permission although the umask for db2inst1 is 022, this is different from the case before setting setfacl. Also, user jephe and zhitan will be able to create files or directories under /shared.

or you can just set up mask permission as rwx, then any files created by db2inst1 will have group write permissions.
[root@linuxtest /]# setfacl -R -m d:m:rwx /shared  (modify mask as rwx)

A directory may contain default ACL entries. If a file or directory is created in a directory that contains default ACL entries, the newly created file will have permissions generated according to the intersection of the default ACL entries and the permissions requested at creation time. The umask will not be applied if the directory contains default ACL entries.

--------------umask explanation by example--------------
[root@linuxtest /]# setfacl -m m:r-- shared  (set shared directory itself mask as r--)
[root@linuxtest /]# getfacl shared
# file: shared
# owner: db2inst1
# group: its
user::rwx
group::rwx                      #effective:r--
mask::r--

other::r-x
default:user::rwx
default:user:jephe:--x
default:group::rwx
default:mask::rwx
default:other::r-x

[root@linuxtest /]# setfacl -d -m m:r-- shared  (set default mask as r--, means for any newly files/directories created under it )
[root@linuxtest /]# getfacl shared
# file: shared
# owner: db2inst1
# group: its
user::rwx
group::rwx                      #effective:r--
mask::r--

other::r-x
default:user::rwx
default:user:jephe:--x          #effective:---
default:group::rwx              #effective:r--
default:mask::r--

default:other::r-x
[root@linuxtest /]# ls -ld shared
drwxr-Sr-x+ 7 db2inst1 its 4096 Feb 20 13:18 shared

------------------end------------------------------------


3. remove all acl permissions
setfacl -R -b /shared

To remove all the permissions for a user, group, or others, use the -x option and do not specify any permissions:

setfacl -x <rules> <files>
To remove all permissions from the user jephe
setfacl -x u:jephe /shared
 
Linux file permissions:

Any program uses system calls to access files and directories.

1. directory:
You can imagine directory is like a datafile which conains a table, each row has filename and its inode number:

Read:  
This permissioin can only find out the name of the files under it , not inode and not other things stated in inode data structure
.

Write:  
This permission allows you to add, rename, or delete files within, not modifying file directly like
# echo testing > file1 
# > file1
 
But vim can modify files. The following is from http://vimdoc.sourceforge.net/htmldoc/editing.html 
*write-readonly*
When the 'cpoptions' option contains 'W', Vim will refuse to overwrite a
readonly file.  When 'W' is not present, ":w!" will overwrite a readonly file,
if the system allows it (the directory must be writable).
 
 
Also, all above actions also require changing or at least reading the inodes of the affected files, 
so search permission is also needed to add/rename/delete files within

Execute: 
This permission means search, grants the ability to traverse its tree in orde to access files/subdirectories. 
It's required to access the "inode" information of the files within.


2. File:
Read: system call read()
You need to get file's inode number to read it.
Only file owner or root user can modify information in the inode such as the owner and group name and permissions.

Write:system call write()  to modify file content

Execute: system call  exec()
If file is a program, you can only need x without r to execute that program, but if the file is a shell script, 
you also must have r permission to execute it. The proper permissions on a script are both read and execute

Example: cat /home/jephe/file1, firstly you need to have read permission for file1 to read the content, then 
you need x permission on / /home /home/jephe to locate inode of file1and thus to read it. 
 
References:

a. Unix File and Directory Permissions and Modes - http://content.hccfl.edu/pollock/AUnix/FilePermissions.htm

b. http://en.wikipedia.org/wiki/Filesystem_permissions 

Some advanced usages on Putty and OpenSSH

Jephe Wu - http://linuxtechres.blogspot.com

Objective: explore advanced usage on Putty on Windows and OpenSSH on Linux
Environment: Putty, OpenSSH


Usage:
1.  use putty or openssh client to create a secure socks web proxy tunnel
You can create socks proxy server by using putty or openssh.

Putty:
create a normal ssh session, enable compression and ssh version 2, under tunnel menu, create a auto and dynamic source port 8080.
then configure your favorite browser to use socks5 proxy at 127.0.0.1:8080,try socks4 proxy if socks5 doesn't work. Configure 'no proxy for' part with 'localhost,127.0.0.1'.

Note: For data you retrieved through browser, most of them is text or HTML data, the compression rate is very high, so, enable compression is better.

OpenSSH:
The following command uses compression, SSH2, Quite, Force pseudo-tty allocation, Redirect stdin from /dev/null, and use 'master' mode for ssh client for connection sharing. 

ssh -C2qTnN -D 8080 jephe@server.domain.com
 or

ssh -C2qTnN -L 8080:localhost:3333 jephe@server.domain.com
(ssh tunnel and local forwarding to ssh server at port 3333(squid proxy)  
 
You can try to access http://www.whatismyip.com/ to get the external source IP. 

login as root to your ssh server, then use above command to ssh into the destination server to create socks proxy tunnel, then use 127.0.0.1:8080 as socks5 proxy to access Internet.

DNS query issue:
when using above socks5 proxy, by default, firefox/thunderbird requires your local Windows pc must be able to resolve DNS request. If not, you can make changes for firefox/thunderbird to enable remote DNS (open the about:config page, and change network.proxy.socks_remote_dns to true), which also secure DNS queries)

Multiple tabs:
network.http.max-persistent-connections-per-proxy 25

Note: some socksifier program
a. http://widecap.com/
b. http://www.proxycap.com/
c. http://tsocks.sourceforge.net/(Linux)

2.  use http proxy software in ssh proxycommand option to ssh into Internet server directly
In office environment, you might not be able to direct ssh into some servers on the Internet. If the ssh server is listening at port 80 or 443, usually squid proxy server is allowing that, if not, you might need to do something to establish ssh connection, either by changing the destination server port to port 80/443 or enabling the squid to allow port 22.

For openssh/cygwin, you can use http://proxytunnel.sourceforge.net/ to use squid proxy server to tunnel your ssh connection.
In /etc/ssh/ssh_config, put
Host jephe
    Proxycommand /usr/bin/proxytunnel -p 10.0.0.1:8080 -d jephe.domain.com:22
   
then you can use 'ssh -v username@jephe -R 2222:localhost:22 -R 3389:10.0.0.2:3389' to do remote port forwarding to RDP/ssh to your office pc from home.

Another options is to use corkscrew -  http://www.agroman.net/corkscrew/
 
Please refer to another article for details at How to access office server and admin desktop from home - http://linuxtechres.blogspot.com/2010/12/how-to-access-office-server-and-admin.html

3. use nc in proxycommand option to directly ssh into server on Internet through firewall 
Case: You are not able to ssh directly to Internet , you have to ssh into firewall/proxy server, then you can ssh to Internet from firewall itself.

Solution: use nc to make it one step only.
on /etc/ssh/ssh_config, put the following line
Host external_ssh_server.domain.com
        ServerAliveInterval 60
        ServerAliveCountMax 600
    ProxyCommand ssh jephe@firewall_ip nc %h %p



Then run 'ssh username@jephe' to ssh directly to host on the Internet. 
 Note:
You might need to configure public key authentication without password for firewall and external_ssh_server.domain.com, otherwise, you might get something like 'write pipe error'.



Note: OpenSSH 5.4 onwards supports netcat mode with option -W host:port. See 
http://www.openssh.org/txt/release-5.4 


* Added a 'netcat mode' to ssh(1): "ssh -W host:port ..." This connects
   stdio on the client to a single port forward on the server. This
   allows, for example, using ssh as a ProxyCommand to route connections
   via intermediate servers.



The following example for using nc is by Fabian Arrotin from  http://planet.centos.org/
You need to ssh/scp from your pc to HostC directly. (normal path: your pc->hostA->hostB->hostC)
==================================

Host HostB
Hostname the.known.fqdn.as.resolvable.by.HostA
User arrfab
ForwardAgent yes
Port 22
ProxyCommand ssh remoteuser@HostA.with.ssh.access nc %h %p



And what if you need to reach HostC, which itself is only reachable by HostB ? Let’s just define a new Host section in the ~/.ssh/config and another ProxyCommand !

Host HostC
Hostname the.known.fqdn.as.resolvable.by.HostB
User arrfab
ForwardAgent yes
Port 22
ProxyCommand ssh remoteuser@HostB nc %h %p


====================================


4. use zmodem transfer with leputty (http://leputty.sourceforge.net/)

It's much faster for you to upload/download files directly with putty, without opening winscp to do it.


You can use leputty, configuring sz/rz path for default putty settings so that all session created later on will have this settings automatically.


When you need to upload file from Windows pc to server, just use 'Zmodem upload' in Leputty.
When you need to download a file, ssh into server with Leputty, then type in 'sz filename', then click on menu 'Zmodem receive' to transfer to the predefined directory on Windows PC.


It's faster then using winscp.



References:
a. https://calomel.org/firefox_ssh_proxy.html

Oracle pfile, spfile and parameters, as well as AMM

Jephe Wu - http://linuxtechres.blogspot.com

Objective: Checking the current running parameters for Oracle database and configuring AMM (automatic memory managgement)
Environment: Oracle 11g 64bit, RHEL 5


Commands:
1.  Checking if the database is using pfile or spfile
sqlplus / as sysdba
select name,value from v$parameter where name='pfile';
select name,value from v$parameter where name='spfile';


2. generate pfile from spfile or memory
sqlplus / as sysdba
create pfile from spfile;
create pfile from memory;
create spfile from pfile; (if the database starts up with pfile initially, then need to change to spfile)


3.  checking pfile,spfile and memory
sqlplus / as sysdba
show parameter memory_target;
show parameter pga;
show parameter sga;
show sga;

AMM configuration:
1.  Configure tmpfs size
error message:
a. ORA-845: MEMORY_TARGET not supported on this system
b. Starting ORACLE instance (normal)
WARNING: You are trying to use the MEMORY_TARGET feature.
This feature requires the /dev/shm file system to be mounted for at
Least <size> bytes.The /dev/shm is either not mounted or is mounted
With available space less than this size.
Please fix this so that MEMORY_TARGET can work as expected.
Current available is <size> and used is <size> bytes.memory_target needs larger /dev/shm

If ORA-04031 is seen in the alert log, sometimes you can not establish new connections due to this problem.

Solutions: 
(ORA-00845 When Starting Up An 11g Instance With AMM Configured. [ID 460506.1]) 
1. If you are installing Oracle 11g on a Linux system, note that Memory Size (SGA and PGA), which sets
the initialization parameter MEMORY_TARGET or MEMORY_MAX_TARGET, cannot be greater than the shared memory filesystem (/dev/shm) on your operating system. To resolve the current error, increase the /dev/shm file size. For example:
# mount -t tmpfs tmpfs -o size=12g /dev/shm
Also, to make this change persistent across system restarts, add an entry in /etc/fstab similar to the following:
tmpfs /dev/shm tmpfs size=12g 0


note: tmpfs is previously called shmfs - http://en.wikipedia.org/wiki/Tmpfs

2.  Configure Automatic Memory Management(AMM) on 11g [ID 443746.1]
Check the current values configured for SGA_TARGET and PGA_AGGREGATE_TARGET.

SQL>SHOW PARAMETER TARGET
Add the values of pga_aggregate_target and sga_target. In our case it is 12g

3.Decide on a maximum amount of memory that you would want to allocate to the database which will determine the maximum value for the sum of the SGA and instance PGA sizes. In our case we decide to set to 12g

4. apply changes to spfile
SQL>ALTER SYSTEM SET MEMORY_MAX_TARGET = 12g SCOPE = SPFILE;
SQL>ALTER SYSTEM SET MEMORY_TARGET = 12g SCOPE = SPFILE;
SQL>ALTER SYSTEM SET SGA_TARGET =0 SCOPE = SPFILE;
SQL>ALTER SYSTEM SET PGA_AGGREGATE_TARGET = 0 SCOPE = SPFILE;


5.  restart database
SQL> shutdown immediate;
sql> startup (or startup mount, then alter database open)

sql> show parameter target;

-------------
Updated on 29 Nov 2012


1. How to look at Linux free command output

https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Tuning_and_Optimizing_Red_Hat_Enterprise_Linux_for_Oracle_9i_and_10g_Databases/chap-Oracle_9i_and_10g_Tuning_Guide-Memory_Usage_and_Page_Cache.html



2. does -/+ buffers/cache used column include SGA Oracle shared memory

testing below:

[oracle@oratest ~]$ sqlplus  / as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on Thu Nov 29 20:35:45 2012

Copyright (c) 1982, 2009, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> !free
             total       used       free     shared    buffers     cached
Mem:       2050676    1938452     112224          0      32112     869344
-/+ buffers/cache:    1036996    1013680
Swap:      5144560      49032    5095528

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> !free
             total       used       free     shared    buffers     cached
Mem:       2050676    1265780     784896          0      32144     339040
-/+ buffers/cache:     894596    1156080
Swap:      5144560      49032    5095528

SQL> !df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       48G   30G   16G  65% /
/dev/sda1              99M   23M   71M  25% /boot
tmpfs                1002M     0 1002M   0% /dev/shm

SQL> startup;
ORACLE instance started.

Total System Global Area  835104768 bytes
Fixed Size    2217952 bytes
Variable Size  637536288 bytes
Database Buffers  192937984 bytes
Redo Buffers    2412544 bytes
Database mounted.
Database opened.
SQL> !free
             total       used       free     shared    buffers     cached
Mem:       2050676    1872012     178664          0      32200     870120
-/+ buffers/cache:     969692    1080984
Swap:      5144560      49032    5095528

SQL> !df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       48G   30G   16G  65% /
/dev/sda1              99M   23M   71M  25% /boot
tmpfs                1002M  520M  482M  52% /dev/shm

[oracle@oratest ~]$ echo "(870120-339040)/1024" | bc
518


Note: When using Oracle, shared memory, which is used by Oracle process, (ipcs), it's caculated by cached column of 'free' command.

3. ipcs or /dev/shm for Oracle shared memory 


Refer to http://www.toadworld.com/Newsletter/TWPIPELINEMay2009/PIPEMay09Oracle/tabid/575/Default.aspx

In Oracle 11g, When you use Automatic Memory Management (AMM) by memory_target or memory_max_target parameters. ipcs -m doesn't show any shared memory. It actually used /dev/shm for small files.


Home network troubleshooting for using cable modem, laptop/pc and wireless router

Jephe Wu - http://linuxtechres.blogspot.com

Environment: home cable modem network in Singapore (ISP: Starhub), Mortolona cable modem, Linksys WRT54 ver2.2 wireless router and laptop/pc
Objective: home network issue troubleshooting for accessing Internet.

Network diagram:
cable wall socket->cable modem->network cable RJ45 -> Linksys wireless router -> laptop/pc connected through physical cable or wireless access point
Issue: able to connect to wireless router through physical cable or access point and get IP address from wireless router itself, but cannot access Internet.

Troubleshooting steps:
1. basic troubleshooting to find out the problem
Everything seems okay between  laptop and wireless router, laptop can get IP address and can access wireless router admin page at http://192.168.1.1(default linksys wireless router IP address).

2. check admin page router status page
Find out router itself cannot get WAN IP address actually.


3. Check cable modem itself (sequence is important, otherwise, you might not be able to get WAN IP address)
Disconnect network cable between cable modem and wireless router.
Connect a dell laptop to cable modem by RJ45 cable.
Shutdown both cable modem and laptop first (totally power down)
Power on the cable modem to wait for the first 4 lights become solid green (if standby light is on after that, please press the button on the top of the modem)
Now power on the laptop and wait for the PC/Activitity light is on
Check if the laptop is getting WAN ip address from ISP through DHCP (ipconfig /all)

In my case, the cable modem is okay and laptop is getting WAN IP and is able to access Internet.

4. Solve the problem of wireless router
Try above steps in point 3, shutdown power for both cable modem and wireless router, connect cable modem and wireless router through RJ45 network cable, power on cable modem first then wireless router later, check admin page, this time, the wireless router can get WAN IP address but client laptop/pc are still not able to access Internet(try to ping 8.8.8.8, cannot).

Try to check Linksys website to upgrade firmware if it's not the latest one. It's already latest in my case.
Use wireless admin page at http://192.168.1.1 to reset to factory default settings.
Then modify the settings such as wireless router admin password, SSID and WPA2 personal/AES password.

Optional - to disable Block Anonymous Internet Requests  This setting prevents the router from being able to be pinged or otherwise connected to on the external interface, unless you have defined a port-forwarding filter. This should be enabled, but keep in mind that not being able to ping the router can make it more difficult to troubleshoot.

Disable UPnP under Administration page.

With above setting is disabled, then configure port forwarding settings.

5. References
http://www.ciscopress.com/articles/article.asp?p=598649&seqNum=5