Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
Rayan CHIKHI
serratus-batch-dl
Commits
5ebe3309
Commit
5ebe3309
authored
Jun 20, 2020
by
Rayan
Browse files
attempt to download missing files
parent
85d5f0b9
Changes
3
Hide whitespace changes
Inline
Side-by-side
src/Dockerfile
View file @
5ebe3309
...
...
@@ -19,7 +19,7 @@ ARG AWS_SECRET_ACCESS_KEY
#ENV AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
#SRA toolkit (from serratus-dl)
ENV
SRATOOLKITVERSION='2.10.
4
'
ENV
SRATOOLKITVERSION='2.10.
7
'
RUN
wget
--quiet
https://ftp-trace.ncbi.nlm.nih.gov/sra/sdk/
${
SRATOOLKITVERSION
}
/sratoolkit.
${
SRATOOLKITVERSION
}
-centos_linux64
.tar.gz
&&
\
tar
xzf sratoolkit.
${
SRATOOLKITVERSION
}
-centos_linux64
.tar.gz
&&
\
rm
-f
sratoolkit.
${
SRATOOLKITVERSION
}
-centos_linux64
.tar.gz
&&
\
...
...
@@ -45,6 +45,8 @@ RUN chmod +x parallel-fastq-dump
RUN
wget
--quiet
http://opengene.org/fastp/fastp
RUN
chmod
+x fastp
RUN
mkdir
-p
/mnt/serratus-data
# for local testing
RUN
pwd
RUN
df
-T
RUN
ls
...
...
src/batch_processor.py
View file @
5ebe3309
...
...
@@ -80,7 +80,7 @@ def process_file(accession, region):
# 'is larger than maximum allowed: skipped' solved by:
# https://wiki.rc.hms.harvard.edu/display/O2/Aspera+to+download+NCBI+SRA+data
#os.system('prefetch --max-size 35G '+accession)
os
.
system
(
'prefetch '
+
accession
)
os
.
system
(
'prefetch
--max-size 35G
'
+
accession
)
prefetch_time
=
datetime
.
now
()
-
prefetch_start
sdb_log
(
sdb
,
accession
,
'prefetch_time'
,
int
(
prefetch_time
.
seconds
))
os
.
system
(
'mkdir -p tmp/'
)
...
...
src/get-aws-profile.sh
100644 → 100755
View file @
5ebe3309
File mode changed from 100644 to 100755
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment