Collection_Data_Types.rst 12 KB
Newer Older
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
1
2
3
4
5
.. _Collection_Data_types:

*********************
Collection Data Types
*********************
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
6

7
Exercises
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
8
9
=========

10
Exercise
11
12
--------

13
14
| Draw the representation in memory of the following expressions.
| what is the data type of each object?
15
16
17
18
19
20
21
22

::   

   x = [1, 2, 3, 4]
   y = x[1]
   y = 3.14
   x[1] = 'foo'
   
23
24
25
26
27
.. figure:: _static/figs/list_1.png
   :width: 400px
   :alt: set
   :figclass: align-center
   
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
::

   x = [1, 2, 3, 4]
   x += [5, 6]

.. figure:: _static/figs/augmented_assignment_list.png  
   :width: 400px
   :alt: set
   :figclass: align-center 

::

   >>> x = [1, 2, 3, 4]
   >>> id(x)
   139950507563632
   >>> x += [5,6]
   >>> id(x)
   139950507563632
   
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
47
48
49
50
51
With mutable object like ``list`` when we mutate the object the state of the object is modified.
But the reference to the object is still unchanged.
So in this exemple we have two ways to access to the list [1,2] if we modify the state of the list itself.
but not the references to this object, then the 2 variables x and y still reference the list containing
[1,2,3,4]. 
52

53
54
55
56
57
58
59
compare with the exercise on string and integers:

Since list are mutable, when ``+=`` is used the original list object is modified, so no rebinding of *x* is necessary.
We can observe this using *id()* which give the memory adress of an object. This adress does not change after the
``+=`` operation.

.. note::
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
60
   even the results is the same there is a subtelty to use augmented operator.
61
62
63
64
   in ``a operator= b`` python looks up ``a`` ’s value only once, so it is potentially faster
   than the ``a = a operator b``.

Exercise
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
--------

wihout using python shell, what is the results of the following statements:  
 
.. note:: 
   sum is a function which return the sum of each elements of a list.
      
::
 
   x = [1, 2, 3, 4]
   x[3] = -4 # what is the value of x now ?
   y = sum(x)/len(x) #what is the value of y ? why ?
   
   y = 0

because sum(x) is an integer, len(x) is also an integer so in python2.x the result is an integer, 
all the digits after the periods are discarded.
In python3 we will obtain the expected result (see :ref:``) 
   
   
85
Exercise
86
87
88
89
--------

How to compute safely the average of a list? ::

Bertrand  NÉRON's avatar
Bertrand NÉRON committed
90
   float(sum(l)) / float(len(l))
91

92
Exercise
93
94
--------

95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
Draw the representation in memory of the following expressions. ::

   x = [1, ['a','b','c'], 3, 4]
   y = x[1]
   y[2] = 'z'
   # what is the value of x ?
   
.. figure:: _static/figs/list_2-1.png
   :width: 400px
   :alt: set
   :figclass: align-center
   

.. container:: clearer

    .. image :: _static/figs/spacer.png
       
 When we execute *y = x[1]*, we create ``y`` wich reference the list ``['a', 'b', 'c']``.
 This list has 2 references on it: ``y`` and ``x[1]`` .
   
   
.. figure:: _static/figs/list_2-2.png
   :width: 400px
   :alt: set
   :figclass: align-center
 
   
.. container:: clearer

    .. image :: _static/figs/spacer.png
       
   
 This object is a list so it is a mutable object.
 So we can access **and** modify it by the two ways ``y`` or ``x[1]`` ::
 
   x = [1, ['a','b','z'], 3, 4]
    
132
133
134
135
136
137
138
139
140
Exercise
--------

from the list l = [1, 2, 3, 4, 5, 6, 7, 8, 9] generate 2 lists l1 containing all odd values, and l2 all even values.::

   l = [1, 2, 3, 4, 5, 6, 7, 8, 9]
   l1 = l[::2]
   l2 = l[1::2]

141
    
142
Exercise
143
144
--------
   
145
generate a list containing all codons. ::
146
147
   
   bases = 'acgt'
148
   codons = []
149
150
151
      for a in bases:
         for b in bases:
            for c in bases:
152
153
154
               codon = a + b + c
               codons.append(codon)
               
155
Exercise
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
156
157
158
159
160
161
162
163
164
165
166
167
168
--------

From a list return a new list without any duplicate, regardless of the order of items. 
For example: ::

   >>> l = [5,2,3,2,2,3,5,1]
   >>> uniqify(l)
   >>> [1,2,3,5] #is one of the solutions 

solution ::

   >>> list(set(l))

169

170
Exercise
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
--------

let the following enzymes collection: ::
 
   import collections
   RestrictEnzyme = collections.namedtuple("RestrictEnzyme", "name comment sequence cut end")

   ecor1 = RestrictEnzyme("EcoRI", "Ecoli restriction enzime I", "gaattc", 1, "sticky")
   ecor5 = RestrictEnzyme("EcoRV", "Ecoli restriction enzime V", "gatatc", 3, "blunt")
   bamh1 = RestrictEnzyme("BamHI", "type II restriction endonuclease from Bacillus amyloliquefaciens ", "ggatcc", 1, "sticky")
   hind3 = RestrictEnzyme("HindIII", "type II site-specific nuclease from Haemophilus influenzae", "aagctt", 1 , "sticky")
   taq1 = RestrictEnzyme("TaqI", "Thermus aquaticus", "tcga", 1 , "sticky")
   not1 = RestrictEnzyme("NotI", "Nocardia otitidis", "gcggccgc", 2 , "sticky")
   sau3a1 = RestrictEnzyme("Sau3aI", "Staphylococcus aureus", "gatc", 0 , "sticky")
   hae3 = RestrictEnzyme("HaeIII", "Haemophilus aegyptius", "ggcc", 2 , "blunt")
   sma1 =  RestrictEnzyme("SmaI", "Serratia marcescens", "cccggg", 3 , "blunt")

and the 2 dna fragments: ::

   dna_1 = """tcgcgcaacgtcgcctacatctcaagattcagcgccgagatccccgggggttgagcgatccccgtcagttggcgtgaattcag
   cagcagcgcaccccgggcgtagaattccagttgcagataatagctgatttagttaacttggatcacagaagcttccaga
   ccaccgtatggatcccaacgcactgttacggatccaattcgtacgtttggggtgatttgattcccgctgcctgccagg"""

   dna_2 = """gagcatgagcggaattctgcatagcgcaagaatgcggccgcttagagcgatgctgccctaaactctatgcagcgggcgtgagg
   attcagtggcttcagaattcctcccgggagaagctgaatagtgaaacgattgaggtgttgtggtgaaccgagtaag
   agcagcttaaatcggagagaattccatttactggccagggtaagagttttggtaaatatatagtgatatctggcttg"""

| which enzymes cut the dna_1 ?
|                  the dna_2 ?
|                  the dna_1 but not the dna_2?

::

   dna_1 = dna_1.replace('\n', '')
   dans_2 = dna_2.replace('\n', '')
206
207
208

algorithm 1 ::

209
210
211
   enzymes = [ecor1, ecor5, bamh1, hind3, taq1, not1, sau3a1, hae3, sma1]
   digest_1 = []
   for enz in enzymes:
212
      if enz.sequence in dna_1:
213
214
         digest_1.append(enz)

215
216
217
algorithm 2 we can determine the position of the site :: 
   
   enzymes = [ecor1, ecor5, bamh1, hind3, taq1, not1, sau3a1, hae3, sma1]
218
219
220
   digest_1 = []
   for enz in enzymes:
      pos = dna_1.find(enz.sequence)
221
      if pos != -1:
222
223
         digest_1.append(enz)

224
225
226
227

with these algorithms we find if an enzyme cut the dna but we cannot find all cuts in the dna for an enzyme. ::

   enzymes = [ecor1, ecor5, bamh1, hind3, taq1, not1, sau3a1, hae3, sma1]
228
229
   digest_1 = []
   for enz in enzymes:
230
231
232
233
234
      print enz.name, dna_1.count(enz.sequence)

the latter algorithm display the number of occurence of each enzyme, But we cannot determine the position of every sites.
We will see howt to do this later.

235

Bertrand  NÉRON's avatar
Bertrand NÉRON committed
236

237
Exercise
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
--------
From a list return a new list without any duplicate, but keeping the order of items. 
For example: ::

   >>> l = [5,2,3,2,2,3,5,1]
   >>> uniqify_with_order(l)
   >>> [5,2,3,1]  

solution ::

   >>> uniq = []
   >>> for item in l:
   >>>   if item not in uniq:
   >>>      uniq.append(item)


254
Exercise
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
--------

list and count occurences of every 3mers in the following sequence ::

   s = """gtcagaccttcctcctcagaagctcacagaaaaacacgctttctgaaagattccacactcaatgccaaaatataccacag
   gaaaattttgcaaggctcacggatttccagtgcaccactggctaaccaagtaggagcacctcttctactgccatgaaagg
   aaaccttcaaaccctaccactgagccattaactaccatcctgtttaagatctgaaaaacatgaagactgtattgctcctg
   atttgtcttctaggatctgctttcaccactccaaccgatccattgaactaccaatttggggcccatggacagaaaactgc
   agagaagcataaatatactcattctgaaatgccagaggaagagaacacagggtttgtaaacaaaggtgatgtgctgtctg
   gccacaggaccataaaagcagaggtaccggtactggatacacagaaggatgagccctgggcttccagaagacaaggacaa
   ggtgatggtgagcatcaaacaaaaaacagcctgaggagcattaacttccttactctgcacagtaatccagggttggcttc
   tgataaccaggaaagcaactctggcagcagcagggaacagcacagctctgagcaccaccagcccaggaggcacaggaaac
   acggcaacatggctggccagtgggctctgagaggagaaagtccagtggatgctcttggtctggttcgtgagcgcaacaca"""

and finally print the results one 3mer and it's occurence per line. 

write first the pseudocode, then implement it.

bonus:
print the kmer by incresing occurences.

solution ::

   s = s.replace('\n', '')
   kmers = {}
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
280
281
282
283
   # range exclude the last value range(3) -> 0, 1 ,2
   # so we nned to go to len(s) minus trimer + 1 to include the 
   # last base 
   for i in range(len(s) - 3 +1):
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
284
285
286
287
288
289
      kmer = s[i:i+3]
      kmers[kmer] = kmers.get(kmer, 0) + 1

   for kmer, occurence in kmers.items():
      print kmer, " = ", occurence

290
291
292
293
294
295
we can use also a defaultdict: ::

   import collections
   
   s = s.replace('\n', '')
   kmers = collection.defaultdict(int)
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
296
   for i in range(len(s) - 2):
297
298
299
      kmer = s[i:i+3]
      kmers[kmer] += 1

Bertrand  NÉRON's avatar
Bertrand NÉRON committed
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
solution bonus ::

   list_of_kmers = kmers.items()  
   from operator import itemgetter
   list_of_kmers.sort(key=itemgetter(1)) 
   for kmer, occurence in list_of_kmers:
      print kmer, " = ", occurence

 solution bonus ::

   list_of_kmers = kmers.items()      
   list_of_kmers.sort(key = lambda kmer: kmer[1])
   for kmer, occurence in list_of_kmers:
      print kmer, " = ", occurence   
      
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
  
Exercise
--------

compute the reversed complement of the following sequence: ::

   seq = 'acggcaacatggctggccagtgggctctgagaggagaaagtccagtggatgctcttggtctggttcgtgagcgcaacaca'

   base_comp = { 'a' : 't', 
                 'c' : 'g',
                 'g' : 'c',
                 't' : 'a'}
   complement = ''
   for base in seq:
      complement += base_comp[base]

   reverse_comp = complement[::-1]
   print reverse_comp
   tgtgttgcgctcacgaaccagaccaagagcatccactggactttctcctctcagagcccactggccagccatgttgccgt
  
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
other solution
""""""""""""""

python provide an interresting method for our problem. 
The ``translate`` method work on string and need a parameter which is a object
that can do the correspondance between characters in old string a the new one.
``maketrans`` is a function in module ``string`` that allow us to build this object.
``maketrans`` take 2 arguments, two strings, the first string contains the characters
to change, the second string the corresponding characters in the new string.
Thus the two strings **must** have the same lenght. The correspondance between
the characters to change and their new values is made in funtion of thier position.
the first character of the first string will be replaced by the first character of the second string,
the second character of the first string will be replaced by the second character of the second string, on so on.   
So we can write the reverse complement without loop.

::

   from string import maketrans

   #first we reverse the sequence
   rev_comp = seq[::-1]
   #then we complement it
   nucleotide = 'ATCG'
   complement = 'TAGC'
   rosetta_stone = maketrans(nucleotide, complement)
   rev_comp = rev_comp.translate(rosetta_stone)

Bertrand  NÉRON's avatar
Bertrand NÉRON committed
362
      
363
Exercise
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
--------

given the following dict : ::

   d = {1 : 'a', 2 : 'b', 3 : 'c' , 4 : 'd'}
   
We want obtain a new dict with the keys and the values inverted so we will obtain: ::

   inverted_d  {'a': 1, 'c': 3, 'b': 2, 'd': 4}

solution ::

   inverted_d = {}
   for key in d.keys():
       inverted_d[d[key]] = key
       
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
380
381
382
383
384
385
solution ::

   inverted_d = {}
   for key, value in d.items():
       inverted_d[value] = key
              
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
386
387
solution ::

388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
   inverted_d = {v : k for k, v in d.items()}
   
Exercise
--------

We assume that we have a phylogenic tree of mammals represented as nested lists. ::
    
   mammals = ['Bovine', ['Gibbon', ['Orang Utan', ['Gorilla', ['Chimp', 'Human']]]], 'Mouse' ]
   
We want to work on the subtree of apes (Gibbon, Orang Utan, Gorilla, Chimp, Human) 

* extract the this subtree in a new tree
* then insert 'Bonobo' at the same level of Chimp we want to obtanin something like this :[chimp, bonobo], Human]
   
what's append on mammals? explain the result. ::

   import copy
     
   mammals = ['Bovine', ['Gibbon', ['Orang Utan', ['Gorilla', ['Chimp', 'Human']]]], 'Mouse' ]
   apes = copy.copy(mammals[1])
   apes [1][1][1] = [['Chimp', 'Bonobo'], 'Human']
   print mammals
   ['Bovine', ['Gibbon', ['Orang Utan', ['Gorilla', ['Chimp', 'Human']]]], 'Mouse' ]

what we should do to work with apes without modify mammals?
   
when we extract apes form mammals we did a shallow copy of mammals. tha mean we create a new list but each item in mammals 
are not copy. when we modify apes we mutate an element of apes which was also referenced in mammals so mammals is modified to.
This is what we call a side effect. To avoid that we should use deepcopy from module copy.
to create apes we should write: ::

   apes = copy.deepcopy(mammals[1]) 
   
deepcopy not only copy the list but make also a copy of each items of list recursively.
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
422