Collection_Data_Types.rst 11.7 KB
Newer Older
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
1
2
3
4
5
.. _Collection_Data_types:

*********************
Collection Data Types
*********************
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
6

7
Exercises
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
8
9
=========

10
Exercise
11
12
--------

13
14
| Draw the representation in memory of the following expressions.
| what is the data type of each object?
15
16
17
18
19
20
21
22

::   

   x = [1, 2, 3, 4]
   y = x[1]
   y = 3.14
   x[1] = 'foo'
   
23
24
25
26
27
.. figure:: _static/figs/list_1.png
   :width: 400px
   :alt: set
   :figclass: align-center
   
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
::

   x = [1, 2, 3, 4]
   x += [5, 6]

.. figure:: _static/figs/augmented_assignment_list.png  
   :width: 400px
   :alt: set
   :figclass: align-center 

::

   >>> x = [1, 2, 3, 4]
   >>> id(x)
   139950507563632
   >>> x += [5,6]
   >>> id(x)
   139950507563632
   
47

48
49
50
51
52
53
54
55
56
57
58
59
compare with the exercise on string and integers:

Since list are mutable, when ``+=`` is used the original list object is modified, so no rebinding of *x* is necessary.
We can observe this using *id()* which give the memory adress of an object. This adress does not change after the
``+=`` operation.

.. note::
   even the results is the same ther is a subtelty to use augmented operator.
   in ``a operator= b`` python looks up ``a`` ’s value only once, so it is potentially faster
   than the ``a = a operator b``.

Exercise
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
--------

wihout using python shell, what is the results of the following statements:  
 
.. note:: 
   sum is a function which return the sum of each elements of a list.
      
::
 
   x = [1, 2, 3, 4]
   x[3] = -4 # what is the value of x now ?
   y = sum(x)/len(x) #what is the value of y ? why ?
   
   y = 0

because sum(x) is an integer, len(x) is also an integer so in python2.x the result is an integer, 
all the digits after the periods are discarded.
In python3 we will obtain the expected result (see :ref:``) 
   
   
80
Exercise
81
82
83
84
--------

How to compute safely the average of a list? ::

Bertrand  NÉRON's avatar
Bertrand NÉRON committed
85
   float(sum(l)) / float(len(l))
86

87
Exercise
88
89
--------

90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
Draw the representation in memory of the following expressions. ::

   x = [1, ['a','b','c'], 3, 4]
   y = x[1]
   y[2] = 'z'
   # what is the value of x ?
   
.. figure:: _static/figs/list_2-1.png
   :width: 400px
   :alt: set
   :figclass: align-center
   

.. container:: clearer

    .. image :: _static/figs/spacer.png
       
 When we execute *y = x[1]*, we create ``y`` wich reference the list ``['a', 'b', 'c']``.
 This list has 2 references on it: ``y`` and ``x[1]`` .
   
   
.. figure:: _static/figs/list_2-2.png
   :width: 400px
   :alt: set
   :figclass: align-center
 
   
.. container:: clearer

    .. image :: _static/figs/spacer.png
       
   
 This object is a list so it is a mutable object.
 So we can access **and** modify it by the two ways ``y`` or ``x[1]`` ::
 
   x = [1, ['a','b','z'], 3, 4]
    
127
128
129
130
131
132
133
134
135
Exercise
--------

from the list l = [1, 2, 3, 4, 5, 6, 7, 8, 9] generate 2 lists l1 containing all odd values, and l2 all even values.::

   l = [1, 2, 3, 4, 5, 6, 7, 8, 9]
   l1 = l[::2]
   l2 = l[1::2]

136
    
137
Exercise
138
139
--------
   
140
generate a list containing all codons. ::
141
142
   
   bases = 'acgt'
143
   codons = []
144
145
146
      for a in bases:
         for b in bases:
            for c in bases:
147
148
149
               codon = a + b + c
               codons.append(codon)
               
150
Exercise
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
151
152
153
154
155
156
157
158
159
160
161
162
163
--------

From a list return a new list without any duplicate, regardless of the order of items. 
For example: ::

   >>> l = [5,2,3,2,2,3,5,1]
   >>> uniqify(l)
   >>> [1,2,3,5] #is one of the solutions 

solution ::

   >>> list(set(l))

164

165
Exercise
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
--------

let the following enzymes collection: ::
 
   import collections
   RestrictEnzyme = collections.namedtuple("RestrictEnzyme", "name comment sequence cut end")

   ecor1 = RestrictEnzyme("EcoRI", "Ecoli restriction enzime I", "gaattc", 1, "sticky")
   ecor5 = RestrictEnzyme("EcoRV", "Ecoli restriction enzime V", "gatatc", 3, "blunt")
   bamh1 = RestrictEnzyme("BamHI", "type II restriction endonuclease from Bacillus amyloliquefaciens ", "ggatcc", 1, "sticky")
   hind3 = RestrictEnzyme("HindIII", "type II site-specific nuclease from Haemophilus influenzae", "aagctt", 1 , "sticky")
   taq1 = RestrictEnzyme("TaqI", "Thermus aquaticus", "tcga", 1 , "sticky")
   not1 = RestrictEnzyme("NotI", "Nocardia otitidis", "gcggccgc", 2 , "sticky")
   sau3a1 = RestrictEnzyme("Sau3aI", "Staphylococcus aureus", "gatc", 0 , "sticky")
   hae3 = RestrictEnzyme("HaeIII", "Haemophilus aegyptius", "ggcc", 2 , "blunt")
   sma1 =  RestrictEnzyme("SmaI", "Serratia marcescens", "cccggg", 3 , "blunt")

and the 2 dna fragments: ::

   dna_1 = """tcgcgcaacgtcgcctacatctcaagattcagcgccgagatccccgggggttgagcgatccccgtcagttggcgtgaattcag
   cagcagcgcaccccgggcgtagaattccagttgcagataatagctgatttagttaacttggatcacagaagcttccaga
   ccaccgtatggatcccaacgcactgttacggatccaattcgtacgtttggggtgatttgattcccgctgcctgccagg"""

   dna_2 = """gagcatgagcggaattctgcatagcgcaagaatgcggccgcttagagcgatgctgccctaaactctatgcagcgggcgtgagg
   attcagtggcttcagaattcctcccgggagaagctgaatagtgaaacgattgaggtgttgtggtgaaccgagtaag
   agcagcttaaatcggagagaattccatttactggccagggtaagagttttggtaaatatatagtgatatctggcttg"""

| which enzymes cut the dna_1 ?
|                  the dna_2 ?
|                  the dna_1 but not the dna_2?

::

   dna_1 = dna_1.replace('\n', '')
   dans_2 = dna_2.replace('\n', '')
201
202
203

algorithm 1 ::

204
205
206
   enzymes = [ecor1, ecor5, bamh1, hind3, taq1, not1, sau3a1, hae3, sma1]
   digest_1 = []
   for enz in enzymes:
207
      if enz.sequence in dna_1:
208
209
         digest_1.append(enz)

210
211
212
algorithm 2 we can determine the position of the site :: 
   
   enzymes = [ecor1, ecor5, bamh1, hind3, taq1, not1, sau3a1, hae3, sma1]
213
214
215
   digest_1 = []
   for enz in enzymes:
      pos = dna_1.find(enz.sequence)
216
      if pos != -1:
217
218
         digest_1.append(enz)

219
220
221
222

with these algorithms we find if an enzyme cut the dna but we cannot find all cuts in the dna for an enzyme. ::

   enzymes = [ecor1, ecor5, bamh1, hind3, taq1, not1, sau3a1, hae3, sma1]
223
224
   digest_1 = []
   for enz in enzymes:
225
226
227
228
229
      print enz.name, dna_1.count(enz.sequence)

the latter algorithm display the number of occurence of each enzyme, But we cannot determine the position of every sites.
We will see howt to do this later.

230

Bertrand  NÉRON's avatar
Bertrand NÉRON committed
231

232
Exercise
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
--------
From a list return a new list without any duplicate, but keeping the order of items. 
For example: ::

   >>> l = [5,2,3,2,2,3,5,1]
   >>> uniqify_with_order(l)
   >>> [5,2,3,1]  

solution ::

   >>> uniq = []
   >>> for item in l:
   >>>   if item not in uniq:
   >>>      uniq.append(item)


249
Exercise
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
--------

list and count occurences of every 3mers in the following sequence ::

   s = """gtcagaccttcctcctcagaagctcacagaaaaacacgctttctgaaagattccacactcaatgccaaaatataccacag
   gaaaattttgcaaggctcacggatttccagtgcaccactggctaaccaagtaggagcacctcttctactgccatgaaagg
   aaaccttcaaaccctaccactgagccattaactaccatcctgtttaagatctgaaaaacatgaagactgtattgctcctg
   atttgtcttctaggatctgctttcaccactccaaccgatccattgaactaccaatttggggcccatggacagaaaactgc
   agagaagcataaatatactcattctgaaatgccagaggaagagaacacagggtttgtaaacaaaggtgatgtgctgtctg
   gccacaggaccataaaagcagaggtaccggtactggatacacagaaggatgagccctgggcttccagaagacaaggacaa
   ggtgatggtgagcatcaaacaaaaaacagcctgaggagcattaacttccttactctgcacagtaatccagggttggcttc
   tgataaccaggaaagcaactctggcagcagcagggaacagcacagctctgagcaccaccagcccaggaggcacaggaaac
   acggcaacatggctggccagtgggctctgagaggagaaagtccagtggatgctcttggtctggttcgtgagcgcaacaca"""

and finally print the results one 3mer and it's occurence per line. 

write first the pseudocode, then implement it.

bonus:
print the kmer by incresing occurences.

solution ::

   s = s.replace('\n', '')
   kmers = {}
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
275
276
277
278
   # range exclude the last value range(3) -> 0, 1 ,2
   # so we nned to go to len(s) minus trimer + 1 to include the 
   # last base 
   for i in range(len(s) - 3 +1):
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
279
280
281
282
283
284
      kmer = s[i:i+3]
      kmers[kmer] = kmers.get(kmer, 0) + 1

   for kmer, occurence in kmers.items():
      print kmer, " = ", occurence

285
286
287
288
289
290
we can use also a defaultdict: ::

   import collections
   
   s = s.replace('\n', '')
   kmers = collection.defaultdict(int)
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
291
   for i in range(len(s) - 2):
292
293
294
      kmer = s[i:i+3]
      kmers[kmer] += 1

Bertrand  NÉRON's avatar
Bertrand NÉRON committed
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
solution bonus ::

   list_of_kmers = kmers.items()  
   from operator import itemgetter
   list_of_kmers.sort(key=itemgetter(1)) 
   for kmer, occurence in list_of_kmers:
      print kmer, " = ", occurence

 solution bonus ::

   list_of_kmers = kmers.items()      
   list_of_kmers.sort(key = lambda kmer: kmer[1])
   for kmer, occurence in list_of_kmers:
      print kmer, " = ", occurence   
      
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
  
Exercise
--------

compute the reversed complement of the following sequence: ::

   seq = 'acggcaacatggctggccagtgggctctgagaggagaaagtccagtggatgctcttggtctggttcgtgagcgcaacaca'

   base_comp = { 'a' : 't', 
                 'c' : 'g',
                 'g' : 'c',
                 't' : 'a'}
   complement = ''
   for base in seq:
      complement += base_comp[base]

   reverse_comp = complement[::-1]
   print reverse_comp
   tgtgttgcgctcacgaaccagaccaagagcatccactggactttctcctctcagagcccactggccagccatgttgccgt
  
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
other solution
""""""""""""""

python provide an interresting method for our problem. 
The ``translate`` method work on string and need a parameter which is a object
that can do the correspondance between characters in old string a the new one.
``maketrans`` is a function in module ``string`` that allow us to build this object.
``maketrans`` take 2 arguments, two strings, the first string contains the characters
to change, the second string the corresponding characters in the new string.
Thus the two strings **must** have the same lenght. The correspondance between
the characters to change and their new values is made in funtion of thier position.
the first character of the first string will be replaced by the first character of the second string,
the second character of the first string will be replaced by the second character of the second string, on so on.   
So we can write the reverse complement without loop.

::

   from string import maketrans

   #first we reverse the sequence
   rev_comp = seq[::-1]
   #then we complement it
   nucleotide = 'ATCG'
   complement = 'TAGC'
   rosetta_stone = maketrans(nucleotide, complement)
   rev_comp = rev_comp.translate(rosetta_stone)

Bertrand  NÉRON's avatar
Bertrand NÉRON committed
357
      
358
Exercise
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
--------

given the following dict : ::

   d = {1 : 'a', 2 : 'b', 3 : 'c' , 4 : 'd'}
   
We want obtain a new dict with the keys and the values inverted so we will obtain: ::

   inverted_d  {'a': 1, 'c': 3, 'b': 2, 'd': 4}

solution ::

   inverted_d = {}
   for key in d.keys():
       inverted_d[d[key]] = key
       
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
375
376
377
378
379
380
solution ::

   inverted_d = {}
   for key, value in d.items():
       inverted_d[value] = key
              
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
381
382
solution ::

383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
   inverted_d = {v : k for k, v in d.items()}
   
Exercise
--------

We assume that we have a phylogenic tree of mammals represented as nested lists. ::
    
   mammals = ['Bovine', ['Gibbon', ['Orang Utan', ['Gorilla', ['Chimp', 'Human']]]], 'Mouse' ]
   
We want to work on the subtree of apes (Gibbon, Orang Utan, Gorilla, Chimp, Human) 

* extract the this subtree in a new tree
* then insert 'Bonobo' at the same level of Chimp we want to obtanin something like this :[chimp, bonobo], Human]
   
what's append on mammals? explain the result. ::

   import copy
     
   mammals = ['Bovine', ['Gibbon', ['Orang Utan', ['Gorilla', ['Chimp', 'Human']]]], 'Mouse' ]
   apes = copy.copy(mammals[1])
   apes [1][1][1] = [['Chimp', 'Bonobo'], 'Human']
   print mammals
   ['Bovine', ['Gibbon', ['Orang Utan', ['Gorilla', ['Chimp', 'Human']]]], 'Mouse' ]

what we should do to work with apes without modify mammals?
   
when we extract apes form mammals we did a shallow copy of mammals. tha mean we create a new list but each item in mammals 
are not copy. when we modify apes we mutate an element of apes which was also referenced in mammals so mammals is modified to.
This is what we call a side effect. To avoid that we should use deepcopy from module copy.
to create apes we should write: ::

   apes = copy.deepcopy(mammals[1]) 
   
deepcopy not only copy the list but make also a copy of each items of list recursively.
Bertrand  NÉRON's avatar
Bertrand NÉRON committed
417