Handling Files Under Unix.pptx

Handling Files under Unix
Mrs.Harsha V Patil, MIT ACSC Alandi , Pune

Introduction:
A file is one in which data can be stored.It is a sequence of bits,bytes or lines and
stored on a storage device like a disk.Inunix everything is stored in terms of files.It
could be a program,an executable code,a file,a set of instructions,a database,a
directory or a subdirectory. A unix file is a tool or an application that defines the
st.ructure of the file and its format.
Unix Filenames:
 Unix is case sensitive! So a file named my data. Txt is different from mydata.txt
which is again different from mydata.txt.
 Unix filenames contain only letters , numbers and the_(underscore)&.(dot)
characters . All other characters should be avoided. The /(slash) characters is
especially important/since it is used to designate subdirectories.

 It is also possible to have additional dots in the filename.
 The part of the name that follows the dot is often used to designate the type of
file.
Files that end in.TXT are text files.
Files that end in.C are source code in the ‘C’ language.
Files that end in.HTML files for the web.
 But this is just a convention and not a rule enforced by the operating system .
this is a good and sensible convention which you should follow.
 Most UNIX system allow a maximum of 14 characters as the length of a
filename. However it depends on the UNIX variant used.

 SPECIAL TOOLS UTILITIES
Unix (shell) has some special features that are useful for interaction and
programming. Before discussing further, we should know about standard files.
Standard files are the special files which are the streams of characters seen as
input or output by the commands. There are three files representing three
streams, each associated with a default device.
 Standard input(stdin)
 Standard output(stdout)
 Standard error(stderr)
Standard input: is a file or a stream representing input. Keyboard is the
Standard input file .But we can use redirection with the < symbol or pipeline
with the symbol | to specify other files as input files.
Standard output: is a file or a stream representing output. Terminal screen
(monitor) is the Standard output file .But we can use redirection with the >, >>
symbol s or pipeline with the symbol | to specify other files as output files.
Standard error: it is a file or stream representing error messages that emerge
from the command or the shell. This is also connected to display screen.

Each standard files are represented by a number called file descriptor , they are
 0 standard input
 1 standard output
 2 standard error
I/O Redirection: Redirection is a feature that reassigns the standard input and
output files. For input redirection (input source) < operator is used, so that input
is not from keyboard, but from another file. And for output redirection > or>>
operators are used , which outputs to other files but not to the terminal.
Eg: $ banner < file1 [ it takes input from file1 and displays it on monitor]
$ echo “ good morning” > file2 [ sends output to file2]
Note : 1. > operator is used for overwriting
2. >> operator is used for appending

Piping : it is a feature available in Unix where two or more commands can be
combined together. Here output of first command is treated as input for second
command. The symbol used for pipe is | . A sequence of commands using one or
more pipes is called a pipeline. The advantage of pipe is that, a command line
containing many commands can be executed as a background process.
For example: $ cat stdlist | sort
[ here output of the command cat is directly send as input to sort command]
Filters: Filters are UNIX commands that take their inputs from the standard
input file, process it and send it to the standard output file. A filter is actually a
program that can read directly from files whose names are provided as arguments,
process it or filer it and sends the result to the standard output, unless a
redirection or a pipeline symbol is used.
Some commonly used filter commands are :

 cat
 head
 tail
 cut
 paste
 uniq
 tr
 wc
 sort
 Grep
NOTE: All filter commands does not change the contents of files but changes will
appear only while displaying. The changes can be stored permanently by using
redirection facility

cat: This command is a multipurpose filter command used for creating file,
displaying contents of file, copy file, append to file and to concatenate two or
more files. This command is already discussed in earlier chapter.
Options:
-n : displays each line with its line number
-b : same as -n but it ignores blank lines
-u : the output is not blubbered
-s : silent about non existent files
head: This command is used to display the first few lines of a file. The user can
specify the number of lines to be displayed by using the n argument. The default
number of lines that are displayed is 10.
Syntax: head [-options] filename

Selected options:
 -cn : print the first n characters of the file
 -n : print the first n lines the default is 10
 $ head dolls [ displays first 10 lines of dolls file ]
 $ head -5 dolls [displays first 5 lines from dolls file ]
 $ head -c5 dolls [displays first 5 characters (including white space) from dolls
file ]
tail: This command is used to display the last few lines from a file. The user can
specify the number of lines to be displayed by using the n argument. The default
number of lines that are displayed is 10.

Syntax: tail [-options] filename
Selected options:
 -cn : print the last n characters of the file
 -n : print the last lines the default is 10
 +n : prints the lines from nth line to end of file
 $ tail dolls [ displays last 10 lines of dolls file ]
 $ tail -5 dolls [displays last 5 lines from dolls file ]
 $ tail –c5 dolls [ displays last 5 characters (includes whitespace)]
usage
 ls | tail -15 [ displays last 15 lines of file listing]
 head abc | tail -5 [ prints lines 6 through 10 of file abc]

cut: This command is used to cut the file vertically i.e to copy (display) the
specified columns to the standard output file. This command can be used to cut
out parts of a file .it takes filenames as arguments. It can cut columns or fields in a
file. However it does not delete the selected parts of the file.
Options
 -b list : to specify the bite position
 -c list : to extract the characters as specified position in the list . a dash (–) is
used
 to indicate the range and comma(,) is used to separate items in the list.
 -f list : to extract the fields as specified position in the list. a dash (–) is used to
 indicate the range and comma(,) is used to separate items in the list.
 -dc : the character following the d is field delimiter and this option is used only
 with -f option
 $ cut -c 10-15 stdlist
 $ cut -d”:” –f 1,3 stdlist

paste: This command is used to combine two or more files vertically. The lines of
the input files must be in corresponding order before the paste utility can be used.
If there is no corresponding field the command puts the delimiter and leaves it
blank.
Options
 -dc : to specify the delimiter character used in combined file
 $ paste file1 file2
 $ paste -d”:” file1 file2
Note: paste expands table’s (file’s) width by increasing the number of columns.
But cat expands a table’s length by increasing the number of rows.
uniq: This command is used to remove the repeated lines from a file. The file
must be in sorted order before using this command.

Options
 -c: precedes each line in o/p with count of number of times the line occurred in
i/p
 file
 -d: selects only one copy of the duplicate entries
 -u:selects lines which are not repeated
 -i: ignore case
 $ uniq names
 $ uniq -c names
 $ uniq -u names

tr: This command is used to translate characters. This command manipulates
individual characters in a line. It takes input only from standard input.
syntax: tr [options] [expression 1] [expression 2] < input file
Options
 -c: it complement the set of characters in the expression
 -d: deletes specified range of characters
 -s: sequeezes multiple occurrences of a character into a single character
 $ tr “[a-z]” “[A-Z]” < fruits
 $tr -d ‘:’ <file1
 $ tr -s ‘ ‘ < file1
 $ tr -c ‘a-zA-Z’ ‘*’ <file2 [it matches all the characters that are not found in the
first string and replaces the matched characters with character in the second
string , here all spaces.dots and newline character are replaced with hyphens]

wc: this command is used to count the number of lines ,words and characters in
one or more files. It gives 4 column output, where the first column indicates
number of lines in the file, second column indicates number of words, the third
indicates number of characters and the last column indicates the file name.
Options
 -c: it prints the counts of characters
 -w: it prints the counts of words
 -l : it prints the counts of lines
 $ wc fruits
 $ wc -c fruits
 $wc -w fruits
 $wc -l fruits
sort: this command is used to sort the contents of a file. Using this utility one or
more files can be sorted in alphabetic or numeric order, and also it can reorder the
file based on one or more of the fields.

Options
 -d : sorts according to dictionary and ignores punctuation.
 -f:ignores caps while sorting
 -m : merges two or more given sorted files tnto single sorted file
 -n : sorts according to numeric order
 -r : sorts in reverse order
 -u : removes duplicates and displays unique values
 -o : specifies output file name
 -t : specifies the delimiter used in the file
 -k fieldno : sorts the data according the field number specified by the fieldno
 $ sort file1
 $ sort -o sfile file1
 $ sort -r file1

grep: this command is used to searches its input for a pattern and displays lines
containing the pattern (Globally search a Regular Expression and Print it ) .
The syntax is: grep [options] [pattern] [file name]…..the pattern can be a string
with single or multiple words or special characters. It is advised to enclose the
pattern with single codes.
Options
 -e : pattern : to specify multiple search pattern.
 -i : prepares case while searching.
 -n: used to display records (lines) along with line number
 -v: prints those lines which do not match the pattern
 -c: displays only the count of lines which contains pattern
 -l: displays only the names of files containing the pattern
 $ grep “bca” stdlist
 $ grep -i “bca” stdlist
 $ grep -n “BCA” stdlist
grep family contains three commands namely grep, egrep(extended grep) and
fgrep(fixed grep)

egrep : [Extended Global Regular Expressions Print] is a pattern searching
command which belongs to the family of grep functions. It works the same way as
grep -E does. It treats the pattern as an extended regular expression and prints out
the lines that match the pattern. If there are several files with the matching
pattern, it also displays the file names for each line.
 The egrep command used mainly due to the fact that it is faster than the grep
command.
 The egrep command treats the meta-characters as they are and do not require
to be escaped as is the case with grep. This allows reducing the overhead of
replacing these characters while pattern matching making egrep faster than
grep or fgrep.
Syntax:
 egrep [options] '<regular expression>' <filename>

Some common options are:
 -c for counting the number of successful matches and not printing the actual
matches
 -i to make the search case insensitive, -n to print the line number before each
match
 printout
 -v to take the complement of the regular expression (i.e. return the lines
 which don't match),
 -l to print the filenames of files with lines which match the expression.
 $ egrep ‘bca|bcom’ studfile
 $ egrep –i ‘bca|bcom’ studfile
 $ egrep –f pfile studfile where pfile contains pattern

fgrep : This command searches for fixed-character strings in a file or files.
"Fixedcharacter"means the string is interpreted literally — metacharacters do not
exist, and therefore regular expressions cannot be used.
fgrep is useful when you need to search for strings which contain lots of regular
expression metacharacters, such as "$", "^", etc. By specifying that your search
string contains fixed characters, you don't need to escape each of them with a
backslash.
 Running fgrep is the same as running grep with the -F option.
 $ egrep ‘bca
 Bcom
 bsc’ studfile
 $ egrep –i ‘bca
 bcom’ studfile

Difference between egrep and fgrep
1. Both egrep and fgrep are derived from the base grep command. The “egrep”
stands for “extended grep” while the fgrep stands for “fixed-string grep.”
2.An egrep command is used to search for multiple patterns inside a file or other
kind of data repository while frgrep is used to look for strings.
3.The term “egrep” is commonly expressed as “grep-E” while “fgrep” is encoded as
“grep-F.”
4.The egrep command allows the use of extended regular expressions while grep
only searches for the matching word or term that the user specified in the
command. The frep doesn’t recognize or understand regular or extended regular
expression.
5.Compared to the other search commands, the search process for fgrep is very
fast since it is only concerned with the provided search word.
6.The egrep command usually uses operators in order to yield a more progressive
or specific search research. A plus sign and the question mark deal with single,
regular expressions or search terms. On the other hand, vertical bars and
parentheses are used for multiple, regular expressions with opposing functions.
The vertical bar separates the expressions while the parentheses operator groups
them.

tar
The Linux ‘tar’ stands for tape archive, is used to create Archive and extract the
Archive files. tar command in Linux is one of the important command which
provides archiving functionality in Linux. We can use Linux tar command to
create compressed or uncompressed Archive files and also maintain and modify
them.
Syntax:
tar [options] filename
Options:
 -c : Creates Archive
 -x : Extract the archive
 -f : creates archive with given filename
 -t : displays or lists files in archived file
 -u : archives and adds to an existing archive file
 -v : Displays Verbose Information
 -A : Concatenates the archive files

 -z : zip, tells tar command that create tar file using gzip
 -j : filter archive tar file using tbzip
 -W : Verify a archive file
 -r : update or add file or directory in already existed .tar file An Archive file is a
file that is composed of one or more files along with metadata. Archive files are
used to collect multiple data files together into a single file for easier portability
and storage, or simply to compress files to use less storage space.
Examples:
 Creating an uncompressed tar Archive using option -cvf : This command
creates a tar
file called file.tar which is the Archive of all .c files in current directory.
$ tar cvf file.tar *.c
 Extracting files from Archive using option -xvf : This command extracts
files from Archives.
$ tar xvf file.tar
 gzip compression on the tar Archive, using option -z : This command
creates a tar file called file.tar.gz which is the Archive of .c files.
$ tar cvzf file.tar.gz *.c

 Extracting a gzip tar Archive *.tar.gz using option -xvzf : This command
extracts files from tar archived file.tar.gz files.
$ tar xvzf file.tar.gz
 Creating compressed tar archive file in Linux using option -j : This
command compresses and creates archive file less than the size of the gzip.
Both compress and decompress takes more time then gzip.
$ tar cvfj file.tar.tbz example.cpp
• untar single tar file or specified directory in Linux : This command will
Untar a file in current directory or in a specified directory using -C option.
$ tar xvfj file.tar
 Untar multiple .tar, .tar.gz, .tar.tbz file in Linux : This command will
extract or untar multiple files from the tar, tar.gz and tar.bz2 archive file. For
example the above command will extract “fileA” “
$ tar xvf file.tar "fileA" "fileB"

File access permission (FAP)
The files that users create will not be accessible to other users. However UNIX
allows to explicitly make their files available to others. Every file of folder in UNIX
has acces permission.
There are three types of permission:
 Read access
 Write access
 Execute access
Read permission allows users to view the contents of a file and to copy it
Write permission allows users to modify the files
Execute permission allows users to run the file, if it is executable.
Types of users
 The owner of the file(user)
 The users who belong to the same group as the file(group)
 Everyone else (other)

File permissions notation
 Textual representation like “-rwxr- -r- -“
 Numeric(octal) representation like “644”
Textual representation
It is used in UNIX long directory listing.it is consists of 10 character. The first
character shows the file type. Next 9 characters are permission, consisting of 3
group: Owner, group , others.
 Each group consists of three symbols : r w x (in this order)
 If some permission is denied, then a dash “-“is used instead.
File type user permission group permission others permission
File Type User Permission Group Permission Other Permission
- r w x r - - r - -

Numeric representation:
 If a numeric representation is used (like in chmod command) then it is in the
 octal format(base of 8)
 Digits involved are 0 to 7. Every octal digit combines read, write and execute
 permission together, respective access rights for owner, group and other are
 the last three digits of the numeric file permissions representation.
Octal digits Text equivalent Binary value Meaning
0 --- 000 All type of access are denied
1 --r 001 Execute access is allowed only
2 -w- 010 Write access is allowed only
3 -wx 011 Write and execute access are allowed
4 r-- 100 Read access allowed only
5 r-w 101 Read and execute access are allowed
6 rw- 110 Read and write access are allowed
7 rwx 111 Everything as allowed

Chmod Command in Linux/Unix with Examples:
 Unix chmod command is used to change the access permissions of files and
directories. It stands for change mode. It can not change the permission of
symbolic links. Even, it ignores the symbolic links come across recursive
directory traversal.
 In the Unix file system, each file is associated with a particular owner and have
permission access for different users. The user classes may be:
1.owner
2.group member
3.others (Everybody else)
The file permissions in Linux are the following three types:
1.read (r)
2.write (w)
3.execute (x)

 Syntax:chmod <options> <permissions> <file name> The primary component
of the chmod permission:
For instance, rwxr-x---
All groups of three characters specify permissions for all classes:
 rwx: The leftmost three characters specify permissions for the file owner (i.e.,
the User class).
 r-x: The three middle characters specify permissions for the group owning the
file (i.e., the Group class).
 ---: The three rightmost characters specify permissions for the Other class.
Users who aren't the file owner and group members can't access the file.
The permission statement is represented in indicators such as u+x, u-x. Where
'u' stands for 'user,' '+' stands for add, '-' stands for remove, 'x' stands for
executable (which).
The user value can be:
 u: the owner of the file
 g: group member
 o: others
 a: all
 The permission types can be r, w, and x.

 “chmod” in Unix [The “mode” helps in setting new permissions that have to
be applied to files or directories.
 This mode can be specified in several ways, we will discuss two modes:
Symbolic and Octal mode.
1) Symbolic mode:
 If we talk about symbolic mode, we can say that it is the most common method
used for specifying fir permissions. In this we have to make a combination
of letters and operators to set or tell what to do with permissions.
 The following operators can be used with the symbolic mode:
Operators Definition
`+` Add permissions
`-` Remove permissions
`=`
Set the permissions to the
specified values

 The following letters that can be used in symbolic mode:
Letters Definition
`r` Read permission
`w` Write permission
`x` Execute permission
The following Reference that are used:
Reference Class
u Owner
g Group
o Others
a All (owner,groups,others)

Examples of Using the Symbolic mode:
Read, write and execute permissions to the file owner:
chmod u+rwx [file_name]
Remove write permission for the group and others:
chmod go-w [file_name]
Read and write for Owner, and Read-only for the group and other:
chmod u+rw,go+r [file_name]

2) Octal mode/Numeric Mode
 It is also a method for specifying permissions. In this method we specify
permission using three-digit number. Where..
 First digit specify the permission for Owner.
 Second digit specify the permission for Group.
 Third digit specify the permission for Others. The digits
 NOTE: The digits are calculated by adding the values of the individual
permissions.
Value Permission
4 Read Permission
2 Write Permission
1 Execute Permission

Examples of Using the Octal mode:
 Suppose if we to give read and write permission to the file Owner. Read, write
and executable permission to the Group. Read-only permission to the Other.
They our command would be.
 chmod 674 [file_name]Here.
 6 represent permission of file Owner which are (rw).
 7 represent permission of Group which are (rwx).
 4 represent permission of Other which is (r).

Absolute and Relative Pathnames in UNIX
A path is a unique location to a file or a folder in a file system of an OS.A path
to a file is a combination of / and alpha-numeric characters.
1.Absolute Path-name:
An absolute path is defined as the specifying the location of a file or directory
from the root directory(/).
To write an absolute path-name:
 Start at the root directory ( / ) and work down.
 Write a slash ( / ) after every directory name (last one is optional)
For Example :
$cat abc.sql
will work only if the fie “abc.sql” exists in your current directory. However,
if this file is not present in your working directory and is present
somewhere else say in /home/kt , then this command will work only if you
will use it like shown below:
cat /home/kt/abc.sql

 In the above example, if the first character of a pathname is /, the file’s location
must be determined with respect to root. When you have more than one / in a
pathname, for each such /, you have to descend one level in the file system like
in the above kt is one level below home, and thus two levels below root.
2.Relative path
 Relative path is defined as the path related to the present working
directly(pwd). It starts at your current directory and never starts with a / .
 To be more specific let’s take a look on the below figure in which if we are
looking for photos then absolute path for it will be provided
as /home/jono/photos but assuming that we are already present in jono
directory then the relative path for the same can be written as
simple photos.

Using . and .. in Relative Path-names:
 UNIX offers a shortcut in the relative pathname– that uses either the current
or parent directory as reference and specifies the path relative to it. A relative
path-name uses one of these cryptic symbols:
.(a single dot) - this represents the current directory.
..(two dots) - this represents the parent directory.
Now, what this actually means is that if we are currently in directory
/home/kt/abc and now you can use .. as an argument to cd to move to the
parent directory /home/kt as :
$pwd
/home/kt/abc
$cd .. ***moves one level up***
$pwd
/home/kt

Example of Absolute and Relative Path:
1.Changing directory with relative path concept :$pwd
/home/kt
$cd abc
$pwd
/home/kt/abc
2.Changing directory with absolute path concept:
$pwd
/home/kt
$cd
/home/kt/abc
$pwd
/home/kt/abc

Head :-displaying first few lines.
 The head command display the first few lines of one or more file .this
 command is used to verify the contents of a file. Without any option. By
 default it display the first 10 lines of the file
Syntax: head [option] file name selected
Options:
 -cn: print the first n characters of the file.
 -n: print the first n lines. the default is 10.
Tail : displaying last few lines
The tail command displays the last few lines of a file .by default, it displays
the last 10 lines of the file. It connot be used for multiple files
 Tail bcalist displays the last 10 lines of the file.
 Tail -5 bcalist displays last 5 lines of the bcalist.
 Tail +8 bcalist display all the lines starting from line number 8 upto end of file.

WC:
Wc command is a filter used to count the number of lines , words and character of
one or more files
 It take one one or more filename as its arguments
 It gives the output in 4 columns:,
1 is number of lines
2 is number of words
3 number of characters and the lost column indicates the filename
Wc options:
 -c : prints the number of bytes.
 -l : prints the line count.
 -L : prints the length of the longest line.
 -m: prints the number of character .
 -w: counts words delimited by white space character or new line character.

TR: translate command
 Tr commands is used to translate character .it is a filter that manipulates
individual character in a line.
Syntax : tr[options ][expression1][expression]<input file
 Tr command take two arguments .each arguments may be a character or a
string of the character.
 The behaviour of tr com
Example:
 $cat list1
 A friend in need is a friend indeed.
 $tr ‘frie’ FRI’<LIST1
 AFRIInd InnIId Is a FRIIndInddIId.
 $tr ‘fri’ ‘FRIE’ < list1
 A FRIend In need Is a FRIend Indeed.mand can be explained with an

Cut command :
 Cut command splits files vertically .this command can be used to extract the
required fields or columns from the file
Example:
$cut –c 1-3 stdlist
001
010
007
008
003
Cut options :
 -b list : the list after –b specifies the byte positions
 -c list : the list following -c specifies the character positions such as –c 1-7 which
passes the first 7 characters of each line .
 -f list : the list following -f is a list of fields , separated in the file by a delimiter
character (-d)

Sort :
 The sort command is used to sort the information in a text file in ascending or
descending order .it is also used to merge sorted files it takes zero, one or more
filename as arguments
Sort option :
Sort command has several options .some of the most command options are given
below.
 -d : sorts according to dictionary and ignores the punctuation.
 -f : ignores caps while sorting .
 -n : sorts according to numeric order.
 -r : sorts in reverse order.
 -n : removes duplicates and display unique values.
 -0 outfile : place the sorted output in the file out file.

Thanks !

Handling Files Under Unix.pptx

More Related Content

Similar to Handling Files Under Unix.pptx (20)

More from Harsha Patil (20)

Recently uploaded (20)

Handling Files Under Unix.pptx