Thursday, October 23, 2014

Bash interpreting a Command

By default bash is the default shell in Linux. Lets see how bash shell interprets the command we type in.

It not as simple as checking the binaries in directories specified in the $PATH variable. It flows through the following steps:
  1. Redirection
  2. Aliases
  3. Expansion
  4. Shell Function
  5. Shell Built-in
  6. Hash table
  7. PATH variable
Lets go through the steps :

1. Redirection.
This comes into picture when you use redirection : ">" and ">>".


You can see that a file list.txt is automatically created before the "ls" command is executed with redirection ">". How does this happened ? Redirection is done and the o/p file is created before the command "ls" is executed.  

2. Alias.
Alias is actually shortcuts or easy to remember alternatives of a command.


Here we have defined alias "ls=cat", now when we type "ls list.txt" , ls is replaced with cat and contents of list.txt file is displayed.
Just FYI, we can make the shell ignore alias by preceding "\" with the command.
 
3. Expansion : 
Expansion stands for command substitution, variable and parameter expansion, tilde expansion, brace expansion, arithmetic expansion, word splitting, and path name expansion. 
 

Like this ~, *,? etc will be replaced accodingly.

4. Functions defined in the Shell
Shell also allows to define functions like other programming languages. After defining functions you can execute it just like any other commands.


As you can see, here I have shown you the order of interpretation. Alias has precedence over Functions!. When we first executed "ls" the alias /bin/ls is executed. Then ignored alias with "\" and function is executed. Then we deleted the alias "unalias" command. Now since there is no alias, text candidate to run is function.

5. Shell Builtin
The bash code contains some commands as well. These commands gets precedence over other binaries. The type command can be used to identify who the command is interpreted. 

Lets play a bit with this :

[jk@rhel1 ~]$ type echo
echo is a shell builtin

[jk@rhel1 ~]$ which echo
/bin/echo

[jk@rhel1 ~]$ type ls
ls is a function
ls ()
{
    echo "I am a Function"
}

[jk@rhel1 ~]$ alias ls="echo GNU/Linux"
[jk@rhel1 ~]$ type ls
ls is aliased to `echo GNU/Linux'
[jk@rhel1 ~]$ ls
GNU/Linux

[jk@rhel1 ~]$ unalias ls
[jk@rhel1 ~]$ ls
I am a Function

[jk@rhel1 ~]$ unset ls
[jk@rhel1 ~]$ type ls
ls is hashed (/bin/ls)
[jk@rhel1 ~]$

I shall explain why "hashed" is displayed in the last command.


6. Hash Table 
The concept of hash table is similar to cashing in Linux, shell store the full path of all the executed commands to speed things up. 
Lets see how hast table works. We have started a new shell and executed "ls", "cat" and "echo" commands. Then we checked the hash table entries. The hash table, as already mentioned, contains two columns, first column contains the cache hits and the 2nd column contains the full path of the command. Also note that the shell built in command "echo" is not listed in the hash table. Remember the purpose the hash table is to speed up the execution. 


You can see that when the command "ls" is executed the second time, the hits value is increased.

7. Path Variable
After all the above steps are over or not available, shell searches for the given command in the directories specified in the PATH variable. 

[jk@rhel1 ~]$ echo $PATH
/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/jk/bin
[jk@rhel1 ~]$


Wednesday, October 8, 2014

Disk Metadata : Superblock, Directory and Inodes

Metadata
Filesystem blocks are user for 2 purposes : To store User data and Metadata
  • User data - stores actual data contained in files
  • Metadata - stores file system structural information such as superblock, inodes, directories
Metadata describes the structure of the file system. Most common metadata structure are superblock, inode and directories.

Superblocks
Every FS has a superblock which contains info about filesystems such as :
  • File system type
  • Size
  • Status
  • Information about other metadata structures
    • For filesystems with 1k blocksizes, a backup superblock can be found at block 8193
    • For filesystems with 2k blocksizes, at block 16384
    • For 4k blocksizes, at block 32768.
List backup superblocks:
# dumpe2fs /dev/hda3 | grep -i superblock

If Superblock  is corrupted, restore with backup :
# e2fsck -f -b 8193 /dev/sda3
 
Inode
 
 An inode is a data structure on a Linux Unix FS which stores stores basic information about a regular file, directory, or other file system objects.