Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
Rayan CHIKHI
serratus-batch-dl
Commits
1e21f696
Commit
1e21f696
authored
Jun 16, 2020
by
Rayan CHIKHI
Browse files
fixing some mistakes
parent
1bac8ece
Changes
2
Hide whitespace changes
Inline
Side-by-side
src/batch_processor.py
View file @
1e21f696
...
...
@@ -23,15 +23,19 @@ def process_file(accession, region):
print
(
"region - "
+
region
)
startTime
=
datetime
.
now
()
os
.
system
(
' '
.
join
([
"df"
,
"-T"
]))
os
.
system
(
' '
.
join
([
"cat"
,
"/dfT.txt"
]))
os
.
system
(
' '
.
join
([
"lsblk"
]))
os
.
system
(
' '
.
join
([
"cat"
,
"/lsblk.txt"
]))
# go to /tmp (that's where local storage / nvme is)
# go to /data instead (EBS)
os
.
chdir
(
"/mnt/serratus-data"
)
os
.
system
(
' '
.
join
([
"pwd"
]))
# check free space
os
.
system
(
' '
.
join
([
"df"
,
"-T"
]))
os
.
system
(
' '
.
join
([
"df"
,
"-h"
,
"."
]))
os
.
system
(
' '
.
join
([
"lsblk"
]))
# download reads from accession
os
.
system
(
'mkdir -p out/'
)
...
...
template/template.yaml
View file @
1e21f696
...
...
@@ -112,13 +112,13 @@ Resources:
Vcpus
:
4
Memory
:
8000
# seems that 4000 wasn't enough for parallel-fastq-dump, 4 threads
MountPoints
:
-
ContainerPath
:
"
/serratus-data"
-
ContainerPath
:
"
/
mnt/
serratus-data"
ReadOnly
:
false
SourceVolume
:
serratus-data
Volumes
:
-
Name
:
serratus-data
Host
:
SourcePath
:
"
/serratus-data"
SourcePath
:
"
/
mnt/
serratus-data"
RetryStrategy
:
Attempts
:
1
RayanSerratusDlBatchProcessingJobQueue
:
...
...
@@ -182,7 +182,9 @@ Resources:
mkdir /mnt/serratus-data
echo -e "/dev/xvdh\t/mnt/serratus-data\text4\tdefaults\t0\t0" >> /etc/fstab
mount -a
chmod 777 /serratus-data
lsblk > /lsblk.txt
df -T > /dfT.txt
chmod 777 /mnt/serratus-data
service docker restart
--==MYBOUNDARY==--
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment