Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
L
LarvaTagger.jl
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package Registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Nyx
LarvaTagger.jl
Commits
bb80ddd5
Commit
bb80ddd5
authored
6 months ago
by
François LAURENT
Browse files
Options
Downloads
Patches
Plain Diff
fix: type error in cli argument handling
parent
cfef5d4f
No related branches found
No related tags found
1 merge request
!22
Set of commits to be tagger v0.19
Pipeline
#144187
passed
6 months ago
Stage: test
Changes
2
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
src/Taggers.jl
+1
-1
1 addition, 1 deletion
src/Taggers.jl
test/deploy_and_test.sh
+19
-56
19 additions, 56 deletions
test/deploy_and_test.sh
with
20 additions
and
57 deletions
src/Taggers.jl
+
1
−
1
View file @
bb80ddd5
...
@@ -227,7 +227,7 @@ function parsekwargs!(args, kwargs)
...
@@ -227,7 +227,7 @@ function parsekwargs!(args, kwargs)
end
end
function
run
(
tagger
,
switch
,
kwargs
,
extra
...
)
function
run
(
tagger
,
switch
,
kwargs
,
extra
...
)
args
=
[
"--model-instance"
,
tagger
.
model_instance
]
args
=
Any
[
"--model-instance"
,
tagger
.
model_instance
]
for
(
p
,
v
)
in
pairs
(
extra
)
for
(
p
,
v
)
in
pairs
(
extra
)
if
!
isnothing
(
v
)
if
!
isnothing
(
v
)
push!
(
args
,
p
)
push!
(
args
,
p
)
...
...
This diff is collapsed.
Click to expand it.
test/deploy_and_test.sh
+
19
−
56
View file @
bb80ddd5
...
@@ -10,22 +10,21 @@
...
@@ -10,22 +10,21 @@
# * LOCAL_SCENARII: can be set to 1 to overwrite LarvaTagger.jl's test/scenarii.sh file
# * LOCAL_SCENARII: can be set to 1 to overwrite LarvaTagger.jl's test/scenarii.sh file
# with the scenarii.sh file that comes along the present script (same directory)
# with the scenarii.sh file that comes along the present script (same directory)
# * LAUNCHER: default is "srun" if Slurm is available, otherwise nothing
# * LAUNCHER: default is "srun" if Slurm is available, otherwise nothing
# * JULIA_PROJECT: where to find PyCall (do not modify)
# Example on Maestro (GPU-accelerated):
# Example on Maestro (GPU-accelerated):
# SLURM_OPTS='-p dbc_pmo -q dbc -c 4' JULIA_THREADS=4 ./deploy_and_test.sh
# SLURM_OPTS='-p dbc_pmo -q dbc -c 4' JULIA_THREADS=4 ./deploy_and_test.sh
# or (CPU-only):
# or (CPU-only):
# SLURM_OPTS='-p dedicated -q fast -c 16' JULIA_THREADS=16 ./deploy_and_test.sh
# SLURM_OPTS='-p dedicated -q fast -c 16' JULIA_THREADS=16 ./deploy_and_test.sh
if
[
-n
"
$1
"
]
&&
[
"
$1
"
=
"-f"
]
||
[
"
$1
"
=
"--force"
]
;
then
if
!
[
-f
scripts/install.sh
]
;
then
rm
-rf
LarvaTagger
echo
"Call
`
basename
$0
`
from the project's root directory:"
elif
[
-d
LarvaTagger
]
;
then
echo
" test/deploy_and_test.sh"
echo
"
\`
$0
\`
needs to make a local LarvaTagger directory"
echo
"but this directory already exists."
echo
"Run
\`
$0
--force
\`
or pick another location."
exit
1
exit
1
fi
fi
scripts/install.sh
--uninstall
scripts/install.sh
--with-backend
--experimental
#############
#############
## Maestro ##
## Maestro ##
#############
#############
...
@@ -33,13 +32,6 @@ fi
...
@@ -33,13 +32,6 @@ fi
# SSL verification can be disabled for git clone to work:
# SSL verification can be disabled for git clone to work:
# export GIT_SSL_NO_VERIFY=true
# export GIT_SSL_NO_VERIFY=true
if
!
julia
-v
&> /dev/null
;
then
# do NOT use juliaup to install Julia
module load Python &> /dev/null
# Maestro
python3
-m
pip
install
jill
python3
-m
jill
install
-v
1.10.0
-c
fi
# h5diff (for tests) on Maestro
# h5diff (for tests) on Maestro
if
!
h5diff
-V
&> /dev/null
;
then
if
!
h5diff
-V
&> /dev/null
;
then
module load hdf5 &> /dev/null
module load hdf5 &> /dev/null
...
@@ -60,68 +52,39 @@ fi
...
@@ -60,68 +52,39 @@ fi
#############
#############
CURDIR
=
$(
pwd
)
CURDIR
=
$(
pwd
)
LTROOT
=
$HOME
/.local/share/larvatagger/LarvaTagger.jl
mkdir
-p
LarvaTagger
&&
cd
$_
# shunit2 is for tests specifically
# shunit2 is for tests specifically
if
[
-z
"
$(
which shunit2
)
"
]
;
then
if
[
-z
"
$(
which shunit2
)
"
]
;
then
git clone
-c
advice.detachedHead
=
false
-b
v2.1.8 https://github.com/kward/shunit2
(
cd
"
$LTROOT
/test"
&&
\
git clone
-c
advice.detachedHead
=
false
-b
v2.1.8 https://github.com/kward/shunit2
)
fi
fi
## comment out the below line to build a development environment
if
[
-f
LarvaTagger_test_data.tgz
]
;
then
#CLONE_OPTS=" --depth 1 --no-tags --single-branch -b dev"
#git clone$CLONE_OPTS https://gitlab.pasteur.fr/nyx/planarlarvae.jl PlanarLarvae
#git clone$CLONE_OPTS https://gitlab.pasteur.fr/nyx/LarvaTagger.jl
BRANCH
=
dev
wget
-O-
https://gitlab.pasteur.fr/nyx/planarlarvae.jl/-/archive/
${
BRANCH
}
/planarlarvae.jl-
${
BRANCH
}
.tar.gz |
tar
zxv
&&
mv
planarlarvae.jl-
${
BRANCH
}
PlanarLarvae
wget
-O-
https://gitlab.pasteur.fr/nyx/larvatagger.jl/-/archive/
${
BRANCH
}
/larvatagger.jl-
${
BRANCH
}
.tar.gz |
tar
zxv
&&
mv
larvatagger.jl-
${
BRANCH
}
LarvaTagger.jl
ln
-s
$(
realpath
LarvaTagger.jl
)
LarvaTagger
#git clone$CLONE_OPTS https://gitlab.pasteur.fr/nyx/TaggingBackends
#git clone$CLONE_OPTS https://gitlab.pasteur.fr/nyx/MaggotUBA-adapter
wget
-O-
https://gitlab.pasteur.fr/nyx/TaggingBackends/-/archive/
${
BRANCH
}
/TaggingBackends-
${
BRANCH
}
.tar.gz |
tar
zxv
&&
mv
TaggingBackends-
${
BRANCH
}
TaggingBackends
wget
-O-
https://gitlab.pasteur.fr/nyx/MaggotUBA-adapter/-/archive/
${
BRANCH
}
/MaggotUBA-adapter-
${
BRANCH
}
.tar.gz |
tar
zxv
&&
mv
MaggotUBA-adapter-
${
BRANCH
}
MaggotUBA-adapter
ln
-s
$(
realpath
MaggotUBA-adapter
)
MaggotUBA
# the poetry install step may not be necessary
(
cd
TaggingBackends
&&
julia
--project
=
.
-e
'using Pkg; Pkg.instantiate(); Pkg.develop(path="../PlanarLarvae")'
&&
poetry
env
use python3.8
&&
poetry
install
)
# the step below may not be necessary
(
cd
TaggingBackends
&&
JULIA_PROJECT
=
$(
pwd
)
poetry run python
-c
'import julia; julia.install()'
)
(
cd
MaggotUBA
&&
poetry
env
use python3.8
&&
(
cat
requirements.txt | xargs
-I
% sh
-c
'poetry add "%"'
)
&&
poetry
install
-v
&&
poetry remove taggingbackends
&&
poetry add ../TaggingBackends
&&
scripts/make_models.jl default
)
cd
LarvaTagger
||
exit
1
julia
--project
=
.
-e
'using Pkg; Pkg.instantiate(); Pkg.develop(path="../PlanarLarvae")'
# the remainder of this script is test-specific
if
[
-f
../../LarvaTagger_test_data.tgz
]
;
then
# To generate the test data, first run the tests with environment variable KEEP_MODEL_FILES=1
# To generate the test data, first run the tests with environment variable KEEP_MODEL_FILES=1
# copy the LarvaTagger/MaggotUBA/models/test_train_* directories into LarvaTagger/LarvaTagger.jl/test/data
# copy the LarvaTagger/MaggotUBA/models/test_train_* directories into LarvaTagger/LarvaTagger.jl/test/data
# and build the archive from the LarvaTagger/LarvaTagger.jl directory.
# and build the archive from the LarvaTagger/LarvaTagger.jl directory.
# For example:
# For example:
# KEEP_MODEL_FILES=1 ./deploy_and_test.sh
# KEEP_MODEL_FILES=1 ./deploy_and_test.sh
# cd LarvaTagger/LarvaTagger.jl
# cd
~/.local/share/larvatagger/
LarvaTagger/LarvaTagger.jl
# rm -rf test/data/test_train_*
# rm -rf test/data/test_train_*
# cp -Rp ../MaggotUBA/models/test_train_* test/data/
# cp -Rp ../MaggotUBA/models/test_train_* test/data/
# tar zcvf LarvaTagger_test_data.tgz test/data/*
# tar zcvf LarvaTagger_test_data.tgz test/data/*
# mv LarvaTagger_test_data.tgz ../../
(
cd
"
$LTROOT
"
&&
tar
zxvf LarvaTagger_test_data.tgz
)
tar
zxvf ../../LarvaTagger_test_data.tgz
else
else
# Not recommended; reproducibility is not guarantee across hosts or architectures yet
# Not recommended; reproducibility is not guarantee across hosts or architectures yet
#wget -O- https://dl.pasteur.fr/fop/8MvgygD4/LarvaTagger_test_data.tgz | tar zxv
#wget -O- https://dl.pasteur.fr/fop/8MvgygD4/LarvaTagger_test_data.tgz | tar zxv
wget
-O-
https://dl.pasteur.fr/fop/WdnyHdKR/240208_LarvaTagger_test_data.tgz |
tar
zxv
(
cd
"
$LTROOT
"
&&
\
wget
-O-
https://dl.pasteur.fr/fop/t8CtkCpY/240208_LarvaTagger_test_data.tgz |
tar
zxv
)
fi
fi
if
[
"
$LOCAL_SCENARII
"
=
"1"
]
;
then
if
[
"
$LOCAL_SCENARII
"
=
"1"
]
;
then
if
[
-f
"
$CURDIR
/scenarii.sh"
]
;
then
cp
"
$CURDIR
/scenarii.sh"
test
/
if
[
-f
"test/scenarii.sh"
]
;
then
elif
[
-f
"
$CURDIR
/test/scenarii.sh"
]
;
then
cp
"
$CURDIR
/test/scenarii.sh"
test
/
cp
"test/scenarii.sh"
"
$LTROOT
/test/"
else
echo
"Cannot find a local scenarii.sh file"
else
echo
"Cannot find a local scenarii.sh file"
fi
fi
fi
fi
if
[
-z
"
$(
which shunit2
)
"
]
;
then
ln
-s
"
$CURDIR
/LarvaTagger/shunit2/shunit2"
test
/
fi
echo
"
${
launcher
}
test/scenarii.sh"
echo
"
${
launcher
}
test/scenarii.sh"
${
launcher
}
test
/scenarii.sh
(
cd
"
$LTROOT
"
&&
${
launcher
}
test
/scenarii.sh
)
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment