BEHIND THE SCENES: WHAT REALLY HAPPENS WHEN EXECUTING "ls -l *.c" IN THE SHELL?

Entering lines of commands in the terminal everyday must make every software engineer believes that they are familiar with the concept of the Shell. But knowing ‘How to use the Shell’ is only the tip of the iceberg of the Shell concept. In this article, we will dive deeper into the Shell to discover ‘How Shell actually works’ via a familiar command ‘ls -l *.c’.

Before we answer the above question, let’s revise our knowledge first.

What is a “Terminal”?

It is a program that opens as a black screen window allowing user to interact with the Shell.

To starting the Terminal on Linux, we can use shortcut Ctrl+Alt+T to open the window. The first thing we see is a shell prompt that contains username and the name of the machine followed by the ‘$’ sign.

For example, in the above picture, the pop-up terminal shows the username “nhiyn14” and the name of machine “nhiyn14-VirtualBox”. We can also tell that we are currently in the directory /holberton/Simple_Shell. And the shell prompt $ is ready to receive command.

Now we can start entering commands to the Shell.

What is “the Shell”?

The Shell is a program that stands between the User and the Kernel Operation system. The Shell receives the commands from the keyboard, analyses and processes the inserted command and gives them to Kernel to execute the command.

Most modern Linux systems employ the Shell program called bash, which is an enhanced version of sh – the original Unix Shell program.

What is “Kernel”?

A Kernel is the heart and core of an Operating System, hence, has control over everything in the system. The Kernel allows communication between the software and hardware. The Kernel remains in the memory until the Operating System is shut down.

When a Shell process makes a request to the Kernel, then it is called System Call.

What is “ls -l”? What is “*.c”?

ls -l *.c        

• ls is a command that lists the files and directories of the current directory by default.
• -l is an option which is available within the ls command, use to list the content in the long listing format.
• * is a special character which is called wildcard selecting all filenames.
• .c is a source code file written in C programming language.

A combination of all the above elements, execution of “ls -l *.c” command will return a list of all the source code .c file in long listing format of the current directory.

For example, in the above picture, “ls” only lists all the name of all the files in the current directory. While “ls -l *.c” lists all .c file permissions, the number of links, owner name, owner group, file size, time of last modification, and the file name.

We covered the tip of the iceberg. Now, we are ready to dive deeper into the Shell.

WHAT REALLY HAPPENING INSIDE THE SHELL AFTER ENTERING ls -l *.c?

Step 1: getline() reads user command input and stores in buffer

The Shell uses getline() function to the command inserted in Shell prompt, create a buffer of a given size and then store the command in the buffer. getline() function is included in the stdio.h library:

getline(&buffer, &size, stdin)        

• &buffer is the address of the first character position where the input string will be stored.
• &size is the address of the variable that holds the size of the input buffer.
• stdin is the command that user entered in Shell prompt.

Upon success, getline() returns the number of characters read, including the delimiter character, but not the terminate NULL byte ('\0'). Upon failure or end-of-line condition, getline() returns -1.

For example, if user type “ls -l *.c” in stdin, the command is “ls-l *.c\0”. getline() retrieves the command line and put “ls-l *.c” into buffer.

Step 2: strtok() splits the command into tokens

After Shell received the buffer containing the entire command line, strtok() function splits the command line into tokens to determine the command and its arguments. getline() function is included in the string.h.

token = strtok(buffer, " \t\n\r")        

• buffer is the buffer holding the command line retrieved from stdin
• " \t\n\r" " are the delimiter helping strtok to recognize where to split the command and create a new token.

Continue from the example above, when reading the buffer, strtok() recognizes the “ “ in between “ls -l *.c” and splits it into 3 tokens of “ls”, “-l” and “*.c”.

Step 3: check first token and replace the alias with actual command if needed

After getting all the tokens, the Shell starts a check to determine whether the first token (the main command) match any defined alias in its systems. if the first token is an alias, it will be replaced by its original value.

For example, an alias “list” has the value of the command “ls”. When user types “a” in the Terminal, the Shell will check the first token of “a”, recognize “a” is an alias and change it back to “ls”.

Step 4: check first token and execute if it’s a built-in command

After checking for alias, the Shell checks whether the first token match any built-in commands. Built-in commands come from within the Shell, hence, the Shell can execute the built-in commands directly without invoking another program.

The following link shows the list of Shell built-in commands.

Continue from the example above, the Shell will check “ls” against the list of built-in functions. Since “ls” does not match any of the built-in commands, the Shell will continue to step 5.

Step 5: folk() syscall to create a child process to execute our command in it

After checking for built-ins, The Shell now employs system call fork() to create a child process where the Shell will execute the command in. fork() is defined in the header file sys/types.h and unistd.h.

Following the successful forking to create a child process:

In the child process (when folk() returns 0),

•  The Shell will concatenate the first token with different directories in PATH variable. The Shell then check all the directories to find out if such path exists. If path exists, the Shell calls for execve(), alternatively move to step 6.
•  execve() system call is used to replace the current program in the child process with another program that match the tokens and execute the new program.

In the parent process (when folk() returns is greater than 0), wait() system call is used to tell parent process to wait for the child process to finish. The syscall is included in sys/wait.h header file.

Continue from the example above,

Inside the child process, the Shell concatenates the token “ls” with different directories in PATH. One of the directories in PATH is “/bin”. The check continues until The Shell find a match of “/bin/ls” existing.

The Shell then call execve() to execute command “ls” and argument options “-l” and “*.c”.

Since parent and child process is running at the same time, the Shell uses wait() to tell the parent to do nothing while the child executes another program.

Step 6: exit() terminates child process, return to parent process

The Shell uses exit() system call at the end of the child process to terminate the child and redirect the program back to the parent process. exit() syscall is included in stdlib.h header file.

The purpose of step 5 and 6 is to help the Shell to go back to the Shell prompt '$' after executing the input command.

Step 7: Exit the Shell

Now back to the Shell prompt, we have three option to exit the Shell

Option 1: exit

           exit is one of the Shell built-in functions.

Option 2: Ctrl + C

           Sending the interrupt (terminate) signal SIGINT to the current process. Once the process gets that signal, it’s terminating itself and returns the user to the shell prompt

Option 3: Ctrl + D

           Signalling this is end-of-file and that no more command will be received in stdin. Hence, the Shell tell the current program to exit().

The flow chart below shows the basic of how the Shell works:

For further reading:

For further reading:

execve(2) - Linux man page https://linux.die.net/man/2/execve

exit(3) — Linux manual page  https://man7.org/linux/man-pages/man3/exit.3.html

ls(1) — Linux manual page https://man7.org/linux/man-pages/man1/ls.1.html

Manipulating Files http://linuxcommand.org/lc3_lts0050.php

Shell Builtin Commands https://www.gnu.org/software/bash/manual/html_node/Shell-Builtin-Commands.html#:~:text=Builtin%20commands%20are%20contained%20within,directly%2C%20without%20invoking%20another%20program.

The getline() Function https://c-for-dummies.com/blog/?p=1112

wait(2) — Linux manual page  https://man7.org/linux/man-pages/man2/wait.2.html

What happens when you Ctrl-c in the terminal https://medium.com/@aantipov/what-happens-when-you-ctrl-c-in-the-terminal-36b093443e06#:~:text=Turned%20out%20the%20way%20Ctrl,user%20to%20the%20shell%20prompt.

What is "the Shell"? https://linuxcommand.org/lc3_lts0010.php

What is Kernel in Operating System and what are the various types of Kernel? https://afteracademy.com/blog/what-is-kernel-in-operating-system-and-what-are-the-various-types-of-kernel