file_system.rst



File System

File system

On unix there two important questions to ask before to do anything:
who am I ?
where am I ?
somewhere on the file system !


What is the file system

Programs store data in files on disks.
A file system is the way the files are stored on a disk.
In Unix, the files are organized in a tree structure.


what is a "tree structure"?
do you already know a tree structure in biology?


file system tree structure


file system is like a phylogenic tree: it's a hierarchical structure.
ancestors are replaced by directories and species by files.
so it contains directories that contains other directories and/or files.

where am I?
in a directory.

Note
The filesystem structure may vary according the Linux distribution


/ This is the root directory.


/bin : This directory contains executable programs which are needed in single user mode and to bring the system up or repair it.


/boot : Contains static files for the boot loader. This directory only holds the files which are needed during the boot process.


/dev : Special or device files, which refer to physical devices such as hard disk, keyboard, monitor, mouse and modem etc


/etc : Contains configuration files which are local to the machine. Some larger software packages, like Apache, can have their own subdirectories below /etc i.e. /etc/httpd. Some important subdirectories in /etc:


/etc/skel : When a new user account is created, files from this directory are usually copied into the userâ€™s home directory.

/etc/X11 : Configuration files for the X11 window system .

/etc/sysconfig : Important configuration file used by SysV script stored in /etc/init.d and /etc.rcX directories

/etc/cron.* : cron daemon configuration files which is used to execute scheduled commands


/home : Your sweet home to store data and other files. However in large installation yhe structure of /home directory depends on local administration decisions.


/lib : This directory should hold those shared libraries that are necessary to boot the system and to run the commands in the root filesystem.


/lib64 : 64 bit shared libraries that are necessary to boot the system and to run the commands in the root filesystem.


/lib32 : 32 bit shared libraries that are necessary to boot the system and to run the commands in the root filesystem.


/mnt : This directory contains mount points for temporarily mounted filesystems


/opt : This directory should contain add-on packages such as install download firefox or static files


/proc : This is a mount point for the proc filesystem, which provides information about running processes and the kernel.


/root : This directory is usually the home directory for the root user.


/sbin : Like /bin, this directory holds commands needed to boot the system, but which are usually not executed by normal users, root / admin user specific commands goes here.


/tmp : This directory contains temporary files which may be deleted with no notice, such as by a regular job or at system boot up.


/usr : This directory is usually mounted from a separate partition. It should hold only sharable, read-only data, so that it can be mounted by various machines run ning Linux
(useful for diskless client or multiuser Linux network such as university network). Programs, libraries, documentation etc. for all user-related programs.


/var : This directory contains files which may change in size, such as spool and log files.


/lost+found : Every partition has a lost+found in its upper directory. Files that were saved during failures are here, for e.g ext2/ext3 fsck recovery.


https://www.howtogeek.com/117435/htg-explains-the-linux-directory-structure-explained/


what directory ?
pwd (for print working directory)
/home/unix

This the coordinates of your current position.
But what does it mean?


To navigate, coordinates are not enough.
You need a map!


where am I on the map?
This is my map!


So now, I know where I am.
But now, I'd like to explore the map.


let's go for exploration

I need to know how to move.
I need know how to specify my destination.


how to move?


cd  (changing directory)
cd location


how to specify a location?


give absolute position of your destination
give relative path


absolute path


absolute path to file abc_mouse.fa

exercise:
use command cd to explore the file systems.


relative path


so we need to upgrade our location vocabulary:

where a I am : .

one level up : ..


exercise: use command cd to explore the file system using relative path.

shortcuts

I want to go home => cd or cd ~

I want to go in the home of foobar  cd ~foobar

I want to go back to my previous location => cd -

use tab to display the path completion.


Exploration

Each machine have it's own map.
we don't have a map for all machines.
So we have to explore.

what are there in a this directory?


ls (list directory contents)


ls


ls path (list directory contents display it in lexicographic order)

some useful options:


ls -l : use a long listing format


ls -a : do not ignore entries starting with . (ls -al)


ls -d : list directory entries instead of contents


ls -t : sort by modification time


ls -r : reverse order while sorting


ls -lrt : what does it mean?


Exercise:


download the following archive :download:`unix training material <../_static/unix/unix_training.tar.bz2>` on you home.


copy the url


go on your home cd


wget <url>


run the command tar -xjf unix_training.tar.bz2

We will explain these commands later in the course)


Go in ~/unix_training/DataBio , explore the subtree.


special characters
When a user want to execute a command on several files with similar names,
or if he doesn't know the exact names, he can use jockers:


* replace any characters (0 or more characters)

? replace one character (1 character)

[...] specify a set of possible characters

[x-y] allow characters from x to z in lexicographic order

\ escape any characters


jockers exercises

cd ~/unix_training/DataBio
ls -l Sequences/Nucleique/*.fasta
ls -l Sequences/Proteique/*.fasta
ls -l Sequences/*/
ls -l Sequences/*/q?1.*
How to get the listing of all files like ABCD2_MOUSE.fa or ABCD3_HUMAN.sp but NOT abcd_result.txt?
Do you see abcd1_human1.fa?
Does we can list ABCD2_MOUSE.fa and abcd1_human1.fa easily with one ls command?


create directories


mkdir path create a directory.

mkdir -p path create intermediate directories if necessary.

exercise:

go in your home:
create a directory
go in, create a directory sandbox, and in this directory 2 directories data, results


copy file and directory

cp file1 file2
cp file1 (file2 ...) directory
cp -r dir1 dir2

you can use option -p to preserve rights (owner, group, last acces and modification date)
exercise:
copy hmm files (ending with .hmm) in ~/unix_training/DataBio/HMM/ in the data directory you just made.

mv/rename file/directories


mv src dest (src and dest are paths)

you can specify several src like mv src1 src2 src2 ... dest

mv warning
The mv command has not the same effect depending if src and dest are directories or files.

Exercise:
mv files blast2 report in ~/unix_training/DataBio/ in sandbox/results

remove file


rm file1 file2 file ... remove files (Not directories)


rmdir dir1 dir2 ... remove empty directories


empty mean no files even files starting with dot, no directories.


Warning
by default the user is not asked before to delete a file or not.
You are a big girl/guy, you know what you do.
if you are unsure you can add -i option. In this case you will be prompt
before deleting an item.


Exercise:
remove all sandbox/results subtree

link

In unix as other systems we can create shortcut for files or directory.
We call them links. There is several kind of links, here we will see only simbolic links.
when you list a directory (ls -l) they appear like that: ::


lrwxrwxrwx 1 bneron  bneron      27 Oct 15 13:34 file1 -> /home/bneron/very_big_fasta
The command establishing this link is:

ln -s /home/bneron/very_big_fasta file1


ln -s source_file  link_name

We can create a link toward a directory

Exercise
Create a link in ~/sandbox to unix_training/DataBio/Sequences/ directory