Bash interpreting a Command

By default bash is the default shell in Linux. Lets see how bash shell interprets the command we type in.

It not as simple as checking the binaries in directories specified in the $PATH variable. It flows through the following steps:

1. Redirection
2. Aliases
3. Expansion
4. Shell Function
5. Shell Built-in
6. Hash table
7. PATH variable

Lets go through the steps :

1. Redirection.

This comes into picture when you use redirection : ">" and ">>".

You can see that a file list.txt is automatically created before the "ls" command is executed with redirection ">". How does this happened ? Redirection is done and the o/p file is created before the command "ls" is executed.  

2. Alias.

Alias is actually shortcuts or easy to remember alternatives of a command.

Here we have defined alias "ls=cat", now when we type "ls list.txt" , ls is replaced with cat and contents of list.txt file is displayed.

Just FYI, we can make the shell ignore alias by preceding "\" with the command.

 

3. Expansion : 

Expansion stands for command substitution, variable and parameter expansion, tilde expansion, brace expansion, arithmetic expansion, word splitting, and path name expansion. 

 

Like this ~, *,? etc will be replaced accodingly.

4. Functions defined in the Shell

Shell also allows to define functions like other programming languages. After defining functions you can execute it just like any other commands.

As you can see, here I have shown you the order of interpretation. Alias has precedence over Functions!. When we first executed "ls" the alias /bin/ls is executed. Then ignored alias with "\" and function is executed. Then we deleted the alias "unalias" command. Now since there is no alias, text candidate to run is function.

5. Shell Builtin

The bash code contains some commands as well. These commands gets precedence over other binaries. The type command can be used to identify who the command is interpreted. 

Lets play a bit with this :

[jk@rhel1 ~]$ type echo
echo is a shell builtin

[jk@rhel1 ~]$ which echo
/bin/echo

[jk@rhel1 ~]$ type ls
ls is a function
ls ()
{
    echo "I am a Function"
}

[jk@rhel1 ~]$ alias ls="echo GNU/Linux"
[jk@rhel1 ~]$ type ls
ls is aliased to `echo GNU/Linux'

[jk@rhel1 ~]$ ls
GNU/Linux

[jk@rhel1 ~]$ unalias ls
[jk@rhel1 ~]$ ls
I am a Function

[jk@rhel1 ~]$ unset ls
[jk@rhel1 ~]$ type ls
ls is hashed (/bin/ls)
[jk@rhel1 ~]$

I shall explain why "hashed" is displayed in the last command.

6. Hash Table 

The concept of hash table is similar to cashing in Linux, shell store the full path of all the executed commands to speed things up. 

Lets see how hast table works. We have started a new shell and executed "ls", "cat" and "echo" commands. Then we checked the hash table entries. The hash table, as already mentioned, contains two columns, first column contains the cache hits and the 2nd column contains the full path of the command. Also note that the shell built in command "echo" is not listed in the hash table. Remember the purpose the hash table is to speed up the execution. 

You can see that when the command "ls" is executed the second time, the hits value is increased.

7. Path Variable

After all the above steps are over or not available, shell searches for the given command in the directories specified in the PATH variable. 

[jk@rhel1 ~]$ echo $PATH
/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/jk/bin
[jk@rhel1 ~]$

Disk Metadata : Superblock, Directory and Inodes

Metadata

Filesystem blocks are user for 2 purposes : To store User data and Metadata

• User data - stores actual data contained in files
• Metadata - stores file system structural information such as superblock, inodes, directories

Metadata describes the structure of the file system. Most common metadata structure are superblock, inode and directories.

Superblocks

Every FS has a superblock which contains info about filesystems such as :

• File system type
• Size
• Status
• Information about other metadata structures
• • For filesystems with 1k blocksizes, a backup superblock can be found at block 8193
  • For filesystems with 2k blocksizes, at block 16384
  • For 4k blocksizes, at block 32768.

List backup superblocks:

# dumpe2fs /dev/hda3 | grep -i superblock

If Superblock  is corrupted, restore with backup :

# e2fsck -f -b 8193 /dev/sda3

 

Inode

 

 An inode is a data structure on a Linux Unix FS which stores stores basic information about a regular file, directory, or other file system objects.  

Recover deleted files used by any process in Linux from RAM memory.

Every thing in Linux is a File. A file in turn is actually a pointer to inode which contain the actual data on the disk, permissions, ownership. Now what happens when a file is deleted ? Only the link is removed by not the inode or the actual data. if a process is using the file, or if the file is open , the inode is not released for overwriting util the process is done with the file. Such files will remain in the server memory (RAM). 

Lets discuss how to recover such deleted files which is being used by a process.

Lets do it with an example.

Create a test file.
# touch testfile.txt

Echo some random data on it.
# cat /dev/random > testfile.txt

Open the file using some command like below.
# less  testfile.txt

# ps -ef | grep -i less

less 4607 root  4r  REG 254,4   21  
           8880214 /root/testing.txt (deleted)

All the open files remain in the memory and hence in the /proc filesystem. The important columns in the above output are the second one, which gives you the PID of the process that has the file open (4607), and the fourth one, which gives you the file descriptor (4). Now, we go look in /proc, where there will still be a reference to the inode, from which you can copy the file back.

# ls -l /proc/4607/fd/4
lr-x------ 1 root root 64 Apr  7 03:19 
             /proc/4607/fd/4 -> /root/testing.txt (deleted)

To recover the deleted file in memory, just copy as below.
 #cp /proc/4607/fd/4 testing.txt.bk


Hurray you got your file back. Just make sure not to use "-a" switch while copying the file as this will copy the broken softlink.

JK

 

Sendmail Error : NOQUEUE: SYSERR(root): No local mailer defined NOQUEUE: SYSERR(root): QueueDirectory (Q) option must be set

Error using m4 macro while building sendmail.cf from sendmail.mc on RHEL 6 / CentOS 6 flavors?

[root@cent1]# m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

m4:/etc/mail/sendmail.mc:10: cannot open `/usr/share/sendmail-cf/m4/cf.m4': No such file or directory

[root@cent1]#

The above error is because the package sendmail-cf is not installed.

You will have the below error messages on the /var/log/maillog file also. 

sendmail[3215]: gethostbyaddr(10.0.3.15) failed: 2

sendmail[3215]: NOQUEUE: SYSERR(root): No local mailer defined

sendmail[3215]: NOQUEUE: SYSERR(root): QueueDirectory (Q) option must be set

The above error is because the sendmail.cf is not configured properly. 

Fix is to install the package sendmail-cf, build sendmail.cf from sendmail.mc and restart sendmail service.

Install the sendmail-cf

# yum install -y sendmail-cf

Comment the entry which makes the sendmail listens only to loopback address as below by adding "dnl" on /etc/mail/sendmail.mc

# vi /etc/mail/sendmail.mc

dnl # DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl

# m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

# service sendmail restart

Check if sendmail is started properly.

[root@cent1 ~]# netstat -tulnp |  grep -i :25

tcp        0      0 0.0.0.0:25                  0.0.0.0:*                   LISTEN      3748/sendmail

[root@cent1 ~]#

Hope this helps.

Jk