Thursday, October 23, 2014

Bash interpreting a Command

By default bash is the default shell in Linux. Lets see how bash shell interprets the command we type in.

It not as simple as checking the binaries in directories specified in the $PATH variable. It flows through the following steps:
  1. Redirection
  2. Aliases
  3. Expansion
  4. Shell Function
  5. Shell Built-in
  6. Hash table
  7. PATH variable
Lets go through the steps :

1. Redirection.
This comes into picture when you use redirection : ">" and ">>".

You can see that a file list.txt is automatically created before the "ls" command is executed with redirection ">". How does this happened ? Redirection is done and the o/p file is created before the command "ls" is executed.  

2. Alias.
Alias is actually shortcuts or easy to remember alternatives of a command.

Here we have defined alias "ls=cat", now when we type "ls list.txt" , ls is replaced with cat and contents of list.txt file is displayed.
Just FYI, we can make the shell ignore alias by preceding "\" with the command.
3. Expansion : 
Expansion stands for command substitution, variable and parameter expansion, tilde expansion, brace expansion, arithmetic expansion, word splitting, and path name expansion. 

Like this ~, *,? etc will be replaced accodingly.

4. Functions defined in the Shell
Shell also allows to define functions like other programming languages. After defining functions you can execute it just like any other commands.

As you can see, here I have shown you the order of interpretation. Alias has precedence over Functions!. When we first executed "ls" the alias /bin/ls is executed. Then ignored alias with "\" and function is executed. Then we deleted the alias "unalias" command. Now since there is no alias, text candidate to run is function.

5. Shell Builtin
The bash code contains some commands as well. These commands gets precedence over other binaries. The type command can be used to identify who the command is interpreted. 

Lets play a bit with this :

[jk@rhel1 ~]$ type echo
echo is a shell builtin

[jk@rhel1 ~]$ which echo

[jk@rhel1 ~]$ type ls
ls is a function
ls ()
    echo "I am a Function"

[jk@rhel1 ~]$ alias ls="echo GNU/Linux"
[jk@rhel1 ~]$ type ls
ls is aliased to `echo GNU/Linux'
[jk@rhel1 ~]$ ls

[jk@rhel1 ~]$ unalias ls
[jk@rhel1 ~]$ ls
I am a Function

[jk@rhel1 ~]$ unset ls
[jk@rhel1 ~]$ type ls
ls is hashed (/bin/ls)
[jk@rhel1 ~]$

I shall explain why "hashed" is displayed in the last command.

6. Hash Table 
The concept of hash table is similar to cashing in Linux, shell store the full path of all the executed commands to speed things up. 
Lets see how hast table works. We have started a new shell and executed "ls", "cat" and "echo" commands. Then we checked the hash table entries. The hash table, as already mentioned, contains two columns, first column contains the cache hits and the 2nd column contains the full path of the command. Also note that the shell built in command "echo" is not listed in the hash table. Remember the purpose the hash table is to speed up the execution. 

You can see that when the command "ls" is executed the second time, the hits value is increased.

7. Path Variable
After all the above steps are over or not available, shell searches for the given command in the directories specified in the PATH variable. 

[jk@rhel1 ~]$ echo $PATH
[jk@rhel1 ~]$

Wednesday, October 8, 2014

Disk Metadata : Superblock, Directory and Inodes

Filesystem blocks are user for 2 purposes : To store User data and Metadata
  • User data - stores actual data contained in files
  • Metadata - stores file system structural information such as superblock, inodes, directories
Metadata describes the structure of the file system. Most common metadata structure are superblock, inode and directories.

Every FS has a superblock which contains info about filesystems such as :
  • File system type
  • Size
  • Status
  • Information about other metadata structures
    • For filesystems with 1k blocksizes, a backup superblock can be found at block 8193
    • For filesystems with 2k blocksizes, at block 16384
    • For 4k blocksizes, at block 32768.
List backup superblocks:
# dumpe2fs /dev/hda3 | grep -i superblock

If Superblock  is corrupted, restore with backup :
# e2fsck -f -b 8193 /dev/sda3
 An inode is a data structure on a Linux Unix FS which stores stores basic information about a regular file, directory, or other file system objects.  

Monday, April 28, 2014

Recover deleted files used by any process in Linux from RAM memory.

Every thing in Linux is a File. A file in turn is actually a pointer to inode which contain the actual data on the disk, permissions, ownership. Now what happens when a file is deleted ? Only the link is removed by not the inode or the actual data. if a process is using the file, or if the file is open , the inode is not released for overwriting util the process is done with the file. Such files will remain in the server memory (RAM). 

Lets discuss how to recover such deleted files which is being used by a process.

Lets do it with an example.

Create a test file.
# touch testfile.txt

Echo some random data on it.
# cat /dev/random > testfile.txt

Open the file using some command like below.
# less  testfile.txt

# ps -ef | grep -i less
less 4607 root  4r  REG 254,4   21  
           8880214 /root/testing.txt (deleted)

All the open files remain in the memory and hence in the /proc filesystem. The important columns in the above output are the second one, which gives you the PID of the process that has the file open (4607), and the fourth one, which gives you the file descriptor (4). Now, we go look in /proc, where there will still be a reference to the inode, from which you can copy the file back.

# ls -l /proc/4607/fd/4
lr-x------ 1 root root 64 Apr  7 03:19 
             /proc/4607/fd/4 -> /root/testing.txt (deleted)
To recover the deleted file in memory, just copy as below.
 #cp /proc/4607/fd/4 testing.txt.bk

Hurray you got your file back. Just make sure not to use "-a" switch while copying the file as this will copy the broken softlink.



Tuesday, April 8, 2014

Sendmail Error : NOQUEUE: SYSERR(root): No local mailer defined NOQUEUE: SYSERR(root): QueueDirectory (Q) option must be set

Error using m4 macro while building from on RHEL 6 / CentOS 6 flavors?

[root@cent1]# m4 /etc/mail/ > /etc/mail/
m4:/etc/mail/ cannot open `/usr/share/sendmail-cf/m4/cf.m4': No such file or directory

The above error is because the package sendmail-cf is not installed.

You will have the below error messages on the /var/log/maillog file also. 

sendmail[3215]: gethostbyaddr( failed: 2
sendmail[3215]: NOQUEUE: SYSERR(root): No local mailer defined
sendmail[3215]: NOQUEUE: SYSERR(root): QueueDirectory (Q) option must be set

The above error is because the is not configured properly. 

Fix is to install the package sendmail-cf, build from and restart sendmail service.

Install the sendmail-cf

# yum install -y sendmail-cf

Comment the entry which makes the sendmail listens only to loopback address as below by adding "dnl" on /etc/mail/

# vi /etc/mail/
dnl # DAEMON_OPTIONS(`Port=smtp,Addr=, Name=MTA')dnl

# m4 /etc/mail/ > /etc/mail/

# service sendmail restart

Check if sendmail is started properly.

[root@cent1 ~]# netstat -tulnp |  grep -i :25
tcp        0      0        *                   LISTEN      3748/sendmail
[root@cent1 ~]#

Hope this helps.


Change Default MTA in RHEL 6 / Cent OS 6

By default Postfix is set as the default mail transfer agent in Redhat Enterprise Linux 6 and Cent OS 6 flavors.

However you can change the default Mail transfer Agent in RHEL 6 using the below commands.

* Install Sendmail if not yet installed.

# yum install sendmail -y

Change the default MTA.

# alternatives --config mta

There are 2 programs which provide 'mta'.
  Selection    Command
 + 1           /usr/sbin/sendmail.postfix
*  2           /usr/sbin/sendmail.sendmail
Enter to keep the current selection[+], or type selection number: 2

Congrats, now your default MTS is changed from postfix to Sendmail.


Thursday, October 10, 2013

How to configure Veritas Volume Manager VxVM 6.0.2

The disks should be discovered by OS before it can be added to Veritas Volume Manager.

1. List the disks detected by VxVM.

# vxdisk -o alldgs list

Eg :

[root@rhel61 ~]# vxdisk -o alldgs list
DEVICE       TYPE            DISK         GROUP        STATUS
sda          auto:none       -            -            online invalid
sdb          auto:none       -            -            online invalid
sdc          auto:cdsdisk    mydg01       mydg         online
sdd          auto:cdsdisk    mydg02       mydg         online
[root@rhel61 ~]#

sda and sbd are not configured / initialized for VxVM; where as sdc and sdd are initialized and added to a disk group.

2. Initialize disks.

vxdisksetup -i [attributes]

# vxdisksetup -i sdc
# vxdisksetup -i emc0_dd1

Initializing a disk will create Privae and Public regions in the disks, Meta data is stored in private region and public reagion is available for general purpose data usage. 32 MB is reserved for private region.

Friday, July 5, 2013

Configure DNS Client in Solaris 11.1

Solaris 11.1 saw a lot of changes. A number of configurations has been moved from config files to Service Management Framework , SMF.

To enable DNS client you should be come an Administrator.

1. List the current DNS Client configuration.

root@solaris11.1:~# svccfg -s network/dns/client listprop config
config                      application
config/value_authorization astring
config/domain              astring     Home
config/nameserver          net_address

2. Update the SMF repository with the DNS server name. Let and be the primary and secondary DNS server.

root@solaris11.1:~# svccfg -s network/dns/client setprop config/nameserver = net_address: "("

3. change the Domain name.
root@solaris11.1:~# svccfg -s network/dns/client setprop config/domain = astring: '("solaris.local")'

4. Change the search domains.
root@solaris11.1:~# svccfg -s network/dns/client setprop config/search = astring: '("solaris.local" test.solaris.local)'

5. Update the Name resolution order.
root@solaris11.1:~# svccfg -s name-service/switch setprop config/host = astring: '("files dns mdns")'

6. Review the modified DNS client configuration
root@solaris11.1:~# svccfg -s network/dns/client listprop config
config                      application
config/value_authorization astring
config/domain              astring     solaris.local test.solaris.local
config/nameserver          net_address

root@sol1:~# svccfg -s name-service/switch listprop config
config                      application
config/default             astring     files
config/value_authorization astring
config/host                astring     "files dns mdns"
config/printer             astring     "user files"