Skip to content
Snippets Groups Projects
Commit 762bdd0a authored by Bertrand  NÉRON's avatar Bertrand NÉRON
Browse files

fix rst syntax

remove hieroglyph specific class
transform in classical sphinx
parent ce5b8ed5
No related branches found
No related tags found
No related merge requests found
Pipeline #87426 passed
...@@ -90,6 +90,16 @@ man vs info vs help ...@@ -90,6 +90,16 @@ man vs info vs help
set the date and time set the date and time
===================== =====================
Having a right time is important, lot of tools used time to know if they have to do something ...
We will set time of the machine to the UTC time
Then the correction to the localtime will be perform by Linux
To set the time
#. restart your PC and press escape key during boot phase
#. select enter set up
#. set the time
#. save and exit
sudoers sudoers
...@@ -119,6 +129,7 @@ create new user ...@@ -119,6 +129,7 @@ create new user
only root can perform this command only root can perform this command
create new group create new group
================ ================
......
...@@ -125,10 +125,11 @@ Redirecting Standard Error ...@@ -125,10 +125,11 @@ Redirecting Standard Error
* how to redirect stdout and stderr in same file? * how to redirect stdout and stderr in same file?
.. rst-class:: build ``squizz -c FASTA qn1.gb > qn1.fa 2>&1``
* ``squizz -c FASTA qn1.gb > qn1.fa 2>&1`` .. warning::
* .. warning:: 2>&1 there is no spaces between >and &1
2>&1 there is no spaces between >and &1
The chain command The chain command
================= =================
...@@ -164,14 +165,14 @@ How do I run several commands in Sequence or all at once? ...@@ -164,14 +165,14 @@ How do I run several commands in Sequence or all at once?
| and store the result in ``QR1.FA`` file | and store the result in ``QR1.FA`` file
| all in one command line | all in one command line
.. rst-class:: build
* squizz -c FASTA ~/DataBio/Sequences/Nucleique/qr1.gde > qr1.fa ; tr 'acgt' 'ACGT' < qr1.fa > QR1.FA ``squizz -c FASTA ~/DataBio/Sequences/Nucleique/qr1.gde > qr1.fa ; tr 'acgt' 'ACGT' < qr1.fa > QR1.FA``
limitation limitation
========== ==========
squizz -c FASTA qr1.gd > qr1.fa ; tr 'acgt' 'ACGT' < qr1.fa > QR1.FA ``squizz -c FASTA qr1.gd > qr1.fa ; tr 'acgt' 'ACGT' < qr1.fa > QR1.FA``
| Here there is no consequences. | Here there is no consequences.
| But if my command is more complex like ``squizz -c FASTA qr1.gd > qr1.fa ; blastall -p blastp -d sp -i qr1.fa`` | But if my command is more complex like ``squizz -c FASTA qr1.gd > qr1.fa ; blastall -p blastp -d sp -i qr1.fa``
...@@ -204,3 +205,36 @@ command2 will be executed if and only if command1 returned zero exit status. ...@@ -204,3 +205,36 @@ command2 will be executed if and only if command1 returned zero exit status.
The exit status depends how the developer implement programs. The exit status depends how the developer implement programs.
Sometimes the developer does not follow the convention. Sometimes the developer does not follow the convention.
So it's not possible to chain correctly those programs. So it's not possible to chain correctly those programs.
xargs
=====
build and execute command lines from standard input
xargs executes the command with any initial-arguments
followed by items read from standard input.
Do not forget to add -l to process standard input line by line
::
find . -name '*.fasta' | xargs grep ">"
* **-I** allows you to insert the line from the standard input
somewhere else than in the end of the command
::
find . -name '*.fasta' | xargs -I fic cat fic >>allmysequences
tee
===
* tee copies its input stream to the standard output and the files
specified in argument
.. figure:: ../_static/unix/images/688px-Tee.png
:class: align-center
:width: 500px
http://en.wikipedia.org/wiki/Tee_%28command%29#mediaviewer/File:Tee.svg
...@@ -15,14 +15,6 @@ File system ...@@ -15,14 +15,6 @@ File system
* who am I ? * who am I ?
* where am I ? * where am I ?
.. rst-class:: build
* who am I ?
* | We will see this later on this course.
| For now you are the user "unix"
* where am I ?
* somewhere on the file system ! * somewhere on the file system !
...@@ -38,8 +30,6 @@ what is a "tree structure"? ...@@ -38,8 +30,6 @@ what is a "tree structure"?
do you already know a tree structure in biology? do you already know a tree structure in biology?
.. rst-class:: build
.. figure:: ../_static/unix/images/F1_large.jpg .. figure:: ../_static/unix/images/F1_large.jpg
:class: align-center :class: align-center
:width: 500px :width: 500px
...@@ -56,27 +46,24 @@ file sytem tree structure ...@@ -56,27 +46,24 @@ file sytem tree structure
* ancestors are replaced by directories and species by files. * ancestors are replaced by directories and species by files.
* so it contains directories that contains other directories and/or files. * so it contains directories that contains other directories and/or files.
* where am I? where am I?
.. rst-class:: build
* in a directory. in a directory.
what directory ? what directory ?
================ ================
* ``pwd`` (for **p**\ rint **w**\ orking **d**\ irectory) ``pwd`` (for **p**\ rint **w**\ orking **d**\ irectory)
.. rst-class:: build
* /home/unix /home/unix
* This the coordinates of your current position. | This the coordinates of your current position.
* But what does it mean? | But what does it mean?
* To navigate, coordinates are not enough. | To navigate, coordinates are not enough.
* You need a map! | You need a map!
where am I on the map? where am I on the map?
...@@ -88,17 +75,16 @@ This is my map! ...@@ -88,17 +75,16 @@ This is my map!
:class: align-center :class: align-center
:width: 600px :width: 600px
* So now, I know where I'am. | So now, I know where I'am.
* But now, I'd like to explore the map. | But now, I'd like to explore the map.
let's go for exploration let's go for exploration
======================== ========================
* I need to know how to move. | I need to know how to move.
* I need know how to specify my destination. | I need know how to specify my destination.
.. rst-class:: build
* how to move? * how to move?
...@@ -154,9 +140,8 @@ Exploration ...@@ -154,9 +140,8 @@ Exploration
what are there in a this directory? what are there in a this directory?
.. rst-class:: build
* ``ls`` (list directory contents) | ``ls`` (list directory contents)
ls ls
== ==
...@@ -206,7 +191,7 @@ how to get the listing of all files like abcd2_mouse.fa or abcd3_human.sp but ** ...@@ -206,7 +191,7 @@ how to get the listing of all files like abcd2_mouse.fa or abcd3_human.sp but **
abcd_result.txt? abcd_result.txt?
create direcotries create directories
================== ==================
* **mkdir path** create a directory. * **mkdir path** create a directory.
...@@ -304,222 +289,3 @@ The command establishing this link is: :: ...@@ -304,222 +289,3 @@ The command establishing this link is: ::
* ln -s source_file link_name * ln -s source_file link_name
We can create a link toward a directory We can create a link toward a directory
Permissions
===========
* Unix is a multi-users system. So some users have some rights to do something and some other not.
* The system allow to protect your data form an other user.
* The system allow some users to work in groups.
all this features are managed by the **permissions** (or right access)
3 classes of permissions
========================
::
$ ls -l
total 1176
drwxr-xr-x 4 bneron CIB 4096 Mar 4 2014 Alignment
-rw-r--r-- 1 bneron CIB 220196 Mar 4 2014 CIP-55-138_A.b_A06_037.ab1
drwxr-xr-x 3 bneron bneron 4096 Jun 4 14:51 HMM
.. figure:: ../_static/unix/images/permissions.png
:class: align-center
:width: 650px
| **U**: user
| **G**: group
| **O**: others
| **r**: read (4)
| **w**: write (2)
| **x**: execute (1)
rwx what does it really mean?
=============================
+---+---+------------------------+-------------------------------------------------+
| | | file | directory |
+===+===+========================+=================================================+
| r | 4 | read | allow to list |
+---+---+------------------------+-------------------------------------------------+
| w | 2 | write | allow to create and delete file and directories |
+---+---+------------------------+-------------------------------------------------+
| x | 1 | the file is executable | allow to enter and pass trhough the directory |
+---+---+------------------------+-------------------------------------------------+
who am I?
=========
Permisions are based on your identity and the groups to wich you belong.
* **whoami** => give you the login you logged with
* **id <login>** => give the id and the groups to which the login blong (login is optional) ::
bneron@bic-t2a:~$id
uid=2896(bneron) gid=3044(CIB) groups=3044(CIB),87(biok),110(sis),547(mobyle),990(gensoft),3160(vpn-ssl-users),20000(bioweb)
* uid is my login
* gid is my primary group
* groups is the list of all groups to which I belongs (max 16 groups).
change permisions
=================
**chmod [ugo] [+-] [rwx] <target>**
::
chmod [ugo] [+-] [rwx] <target>
drwxr-xr-x 4 bneron CIB 4096 Mar 4 2014 Alignment
chmod g+w Alignment
drwxrwxr-x 4 bneron CIB 4096 Mar 4 2014 Alignment
chmod o-rx Alignment
drwxrwx--- 4 bneron CIB 4096 Mar 4 2014 Alignment
chmod g-wo+x Alignment
chmod: invalid mode: 'g-wo+x'
chmod go +rx Alignment
drwxrwxr-x 4 bneron CIB 4096 Mar 4 2014 Alignment
you can use -R option to apply permision to a directory and recursively to its contents.
change permisions syntax 2
==========================
**chmod num_value <target>**
| num value is composed of 3 digits respectively for
| user, group, other
| each value is the sum of the octal symbols
| for instance for w => 4, read rw => 4+2 = 6 ...
::
-rw-r--r-- 1 bneron sis 0 Oct 16 13:59 file1
chmod 755 file1
-rwxr-xr-x 1 bneron sis 0 Oct 16 13:59 file1
chmod 640 file1
-rw-r----- 1 bneron sis 0 Oct 16 13:59 file1
octal notation
==============
+----------------------+----------+-----------------------+
| Symbolic | Octal | English |
| Notation | Notation | |
+======================+==========+=======================+
| \-\-\-\-\-\-\-\-\-\- | 000 | no permissions |
+----------------------+----------+-----------------------+
| ---x--x--x | 111 | execute |
+----------------------+----------+-----------------------+
| --w--w--w- | 222 | write |
+----------------------+----------+-----------------------+
| --wx-wx-wx | 333 | write & execute |
+----------------------+----------+-----------------------+
| -r--r--r-- | 444 | read |
+----------------------+----------+-----------------------+
| -r-xr-xr-x | 555 | read & execute |
+----------------------+----------+-----------------------+
| -rw-rw-rw- | 666 | read & write |
+----------------------+----------+-----------------------+
| -rwxrwxrwx | 777 | read, write & execute |
+----------------------+----------+-----------------------+
**exercises:**
| create files and directories,
| change their permisions.
permissions by default
======================
To controls the file/dir's permissions at their creation.
* **umask** display the mask in octal notation: 0022
* **umask -S** display the mask in symbolic notation: u=rwx,g=rx,o=rx
| umask is the complement to 7 to the permissions applied
| umask = 0022
| **directories**: permissions are 777
| 7-0=\ **7**, 7-0=\ **7**, 7-2=\ **5**, 7-2=\ **5**
| directories will be created with 755 rights (rwxr-xr-x)
| **files** is same as dir except, never **x** by default
| 7-0=\ **7**, 7-0=\ **7**, 7-2=\ **5**, 7-2=\ **5** then remove **x** if necessary
| files will be created with 644 rights (rw-r--r--)
octal codes meaning
===================
+----------------+---------------------+----------------+
| Octal digit in | | |
| umask command | Allows on directory | allows on file |
+================+=====================+================+
| 0 | read, write,execute | read, write |
+----------------+---------------------+----------------+
| 1 | read and write | read, write |
+----------------+---------------------+----------------+
| 2 | read and execute | read |
+----------------+---------------------+----------------+
| 3 | read only | read |
+----------------+---------------------+----------------+
| 4 | write and execute | write |
+----------------+---------------------+----------------+
| 5 | write only | write |
+----------------+---------------------+----------------+
| 6 | execute only | no permissions |
+----------------+---------------------+----------------+
| 7 | no permissions | no permissions |
+----------------+---------------------+----------------+
setting the umask
=================
* **umask octal_value**
* **umask -S symbolic value**
::
umask 001
umask -S
u=rwx,g=rx,o=rx
umask -S o+w
u=rwx,g=rx,o=rwx
exercise:
| create dir and files (use touch) with different umask
| check their permisions with ``ls -l``
.. role:: red
:red:`To set x on file you need to use chmod.`
change owner/group
==================
chown command (for **ch**\ ange **ow** ner)
* **chown owner file/dir**
* **chown owner.group file/dir**
:red:`the ownership of a file may only be altered by a super-user`
* **chgrp group file/dir**
.. warning::
* Only a member of a group can change a file's group
* And only to one of which they are a member.
* **-R** option recurse through subdirectories.
...@@ -171,6 +171,7 @@ there is various solutions to escape special characters. ...@@ -171,6 +171,7 @@ there is various solutions to escape special characters.
* enclose file name in **'** (single-quotes) or **"** (double-quotes) * enclose file name in **'** (single-quotes) or **"** (double-quotes)
.. warning:: .. warning::
backslashes, single-quotes and double-quotes present different levels of protection backslashes, single-quotes and double-quotes present different levels of protection
:: ::
......
...@@ -36,7 +36,7 @@ exercise ...@@ -36,7 +36,7 @@ exercise
syntax :: syntax ::
# use the bashrc # use the bashrc
source ${HOOME}/.bashrc source ${HOME}/.bashrc
#. make a copy of the *.bashrc* file #. make a copy of the *.bashrc* file
#. use vim to edit the *.bashrc* file and add the alias. #. use vim to edit the *.bashrc* file and add the alias.
......
...@@ -89,24 +89,21 @@ grep with Regular Expressions ...@@ -89,24 +89,21 @@ grep with Regular Expressions
* more information on regular expression in *man grep* * more information on regular expression in *man grep*
sed
===
Regular expression vs shell special characters. Regular expression vs shell special characters.
=============================================== ===============================================
**warning** .. warning::
* remember some characters have a special meaning for the shell, eg *>*, *>>*, *2>*, *\**, *?*, *|*, ... * remember some characters have a special meaning for the shell, eg *>*, *>>*, *2>*, *\**, *?*, *|*, ...
* remember shell evaluation is executed **before** command execution * remember shell evaluation is executed **before** command execution
example: what do you expect in the following **wrong** examples: example: what do you expect in the following **wrong** examples:
* grep > file * grep > file
* grep * file * grep * file
regular expression and shell share some metacharacters that do not have the same meaning, when you want to search for those characters you have to protect them. regular expression and shell share some metacharacters that do not have the same meaning,
when you want to search for those characters you have to protect them.
**hint:** **always** protect your search pattern by enclosing it in double quotes *"* **hint:** **always** protect your search pattern by enclosing it in double quotes *"*
......
...@@ -5,6 +5,8 @@ ...@@ -5,6 +5,8 @@
replace pattern replace pattern
*************** ***************
.. role:: red
tr tr
== ==
...@@ -52,6 +54,11 @@ or the program dos2unix ...@@ -52,6 +54,11 @@ or the program dos2unix
dos2unix transforms the file *inplace*. dos2unix transforms the file *inplace*.
exercice
--------
transform sequence in lower case in upper case
sed. sed.
==== ====
...@@ -83,7 +90,7 @@ replace pattern ...@@ -83,7 +90,7 @@ replace pattern
* **sed 's<SEP>pattern<SEP>replace' file** NB <SEP> may be any single character. usual <SEP> ones are */*, *|*, *,*, *#*. * **sed 's<SEP>pattern<SEP>replace' file** NB <SEP> may be any single character. usual <SEP> ones are */*, *|*, *,*, *#*.
* ``sed 's/chromosome/chr/' arrayAnnot.txt`` * ``sed 's/chromosome/chr/' arrayAnnot.txt``
replaces **only the 1st** occurence of *chromosome* by 'chr' in **each line** of *arrayAnnot.txt* replaces **only the 1st** occurence of *chromosome* by 'chr' in :red:`each line` of *arrayAnnot.txt*
* ``sed 's/chromosome/chr/' arrayAnnot.txt`` * ``sed 's/chromosome/chr/' arrayAnnot.txt``
does the same. does the same.
* ``sed 's/chromosome/chr/g' arrayAnnot.txt`` * ``sed 's/chromosome/chr/g' arrayAnnot.txt``
...@@ -95,7 +102,7 @@ expression ...@@ -95,7 +102,7 @@ expression
---------- ----------
* ``\w`` match word (alphanumeric and _) * ``\w`` match word (alphanumeric and _)
* [0-9] digits * ``\d`` digits
* keep only the field bank|accession number|entry_name (*sp|P80874|GS69_BACSU*) * keep only the field bank|accession number|entry_name (*sp|P80874|GS69_BACSU*)
...@@ -148,6 +155,36 @@ The "-n" option will not print anything unless an explicit request to print is f ...@@ -148,6 +155,36 @@ The "-n" option will not print anything unless an explicit request to print is f
**sed -n -e'/pattern/p'** will print only line containing pattern **sed -n -e'/pattern/p'** will print only line containing pattern
special characters
------------------
* useful metacharacters:
* **\.** any character
* **\[\]** bracket expressions: defines a set of characters
* *[abc]* : matches character *"a"* or *"b"* or *"c"*
* *[a-z]* : matches on character in the range from *"a"* to *"z"*, ie *"a"* or *"b"* ... or *"z"*
* *\\w* : matches word (alphanumeric and \_)
* *\\s* : matches blank characters (spaces, tabulation)
* Anchoring.
* **\^** represent start of the string.
* **\$** represent end of the string.
* Repetition.
* **\?** match at most once the preceding element. ie **A?** represent zero or **1** A => "" or "A"
* **\+** match one or more times the preceding element. ie **A+** represent **1** A to any number of successive A => "A" or "AA" or "AAA" etc ...
* **\*** match zero or more times the preceding element. ie **a\*** represent **0** to any number of successive A => "" or "A" or "AA" or "AAA" etc ...
* **{n}** match exactly **n** times the preceding element. ie **A{5}** represent exactly **5** successives A => "AAAAA"
* **{m,n}** match *m* to *n* times the preceding element. ie **A{2,5}** represent from **2** to **5** successives A => "AA" or "AAA" or "AAAA" or "AAAAA"
* Logical OR.
* **(one|two)** match the strings "one" or "two"
* Recording
* **\\(pattern\\)** : will record the pattern that matched
* Negation
* **!** : negate the pattern it precedes
more information on regular expression in *man sed*
Addresses Addresses
--------- ---------
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment