Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
BIGSdb
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
2
Issues
2
List
Boards
Labels
Service Desk
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Operations
Operations
Incidents
Environments
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Emmanuel QUEVILLON
BIGSdb
Commits
a232271e
Commit
a232271e
authored
Jan 19, 2011
by
Keith Jolley
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Work started on auto-refreshing job viewer.
parent
dec50cf9
Changes
7
Hide whitespace changes
Inline
Side-by-side
Showing
7 changed files
with
180 additions
and
49 deletions
+180
-49
bigsdb.pl
bigsdb.pl
+1
-0
lib/BIGSdb/Application.pm
lib/BIGSdb/Application.pm
+6
-2
lib/BIGSdb/JobViewerPage.pm
lib/BIGSdb/JobViewerPage.pm
+81
-0
lib/BIGSdb/OfflineJobManager.pm
lib/BIGSdb/OfflineJobManager.pm
+62
-35
lib/BIGSdb/Page.pm
lib/BIGSdb/Page.pm
+7
-2
lib/BIGSdb/Plugins/XmfaExport.pm
lib/BIGSdb/Plugins/XmfaExport.pm
+11
-9
sql/jobs.sql
sql/jobs.sql
+12
-1
No files found.
bigsdb.pl
View file @
a232271e
...
...
@@ -76,6 +76,7 @@ use BIGSdb::ExtractedSequencePage;
use
BIGSdb::
AlleleQueryPage
;
use
BIGSdb::
LocusInfoPage
;
use
BIGSdb::
OfflineJobManager
;
use
BIGSdb::
JobViewerPage
;
my
$r
=
shift
;
#Apache request object (used for mod_perl)
Log::
Log4perl
->
init_once
(
CONFIG_DIR
.
'
/logging.conf
'
);
...
...
lib/BIGSdb/Application.pm
View file @
a232271e
...
...
@@ -54,7 +54,9 @@ sub new {
$self
->
_setup_prefstore
();
$self
->
_initiate_authdb
if
$self
->
{'
system
'}
->
{'
authentication
'}
eq
'
builtin
';
$self
->
_initiate_jobmanager
(
$config_dir
,
$plugin_dir
,
$dbase_config_dir
)
if
$self
->
{'
cgi
'}
->
param
('
page
')
eq
'
plugin
'
&&
$self
->
{'
config
'}
->
{'
jobs_db
'};
if
(
$self
->
{'
cgi
'}
->
param
('
page
')
eq
'
plugin
'
||
$self
->
{'
cgi
'}
->
param
('
page
')
eq
'
job
'
)
&&
$self
->
{'
config
'}
->
{'
jobs_db
'};
$self
->
_initiate_plugins
(
$plugin_dir
);
}
}
...
...
@@ -334,7 +336,8 @@ sub print_page {
'
fieldValues
'
=>
'
FieldHelpPage
',
'
extractedSequence
'
=>
'
ExtractedSequencePage
',
'
alleleQuery
'
=>
'
AlleleQueryPage
',
'
locusInfo
'
=>
'
LocusInfoPage
'
'
locusInfo
'
=>
'
LocusInfoPage
',
'
job
'
=>
'
JobViewerPage
'
);
my
$page
;
my
%page_attributes
=
(
...
...
@@ -351,6 +354,7 @@ sub print_page {
'
dataConnector
'
=>
$self
->
{'
dataConnector
'},
'
pluginManager
'
=>
$self
->
{'
pluginManager
'},
'
mod_perl_request
'
=>
$self
->
{'
mod_perl_request
'},
'
jobManager
'
=>
$self
->
{'
jobManager
'},
);
my
$continue
=
1
;
my
$auth_cookies_ref
;
...
...
lib/BIGSdb/JobViewerPage.pm
0 → 100644
View file @
a232271e
#Written by Keith Jolley
#Copyright (c) 2010, University of Oxford
#E-mail: keith.jolley@zoo.ox.ac.uk
#
#This file is part of Bacterial Isolate Genome Sequence Database (BIGSdb).
#
#BIGSdb is free software: you can redistribute it and/or modify
#it under the terms of the GNU General Public License as published by
#the Free Software Foundation, either version 3 of the License, or
#(at your option) any later version.
#
#BIGSdb is distributed in the hope that it will be useful,
#but WITHOUT ANY WARRANTY; without even the implied warranty of
#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#GNU General Public License for more details.
#
#You should have received a copy of the GNU General Public License
#along with BIGSdb. If not, see <http://www.gnu.org/licenses/>.
package
BIGSdb::
JobViewerPage
;
use
strict
;
use
base
qw(BIGSdb::Page)
;
use
Log::
Log4perl
qw(get_logger)
;
my
$logger
=
get_logger
('
BIGSdb.Page
');
sub
initiate
{
my
(
$self
)
=
@_
;
$self
->
{'
jQuery
'}
=
1
;
$self
->
{'
noCache
'}
=
1
;
$self
->
{'
refresh
'}
=
30
;
}
sub
print_content
{
my
(
$self
)
=
@_
;
my
$q
=
$self
->
{'
cgi
'};
print
"
<h1>Job status viewer</h1>
";
my
$id
=
$q
->
param
('
id
');
if
(
$id
!~
/BIGSdb_\d+/
){
print
"
<div class=
\"
box
\"
id=
\"
statusbad
\"
>
\n
";
print
"
<p>The submitted job id is invalid.</p>
\n
";
print
"
</div>
";
return
;
}
my
(
$job
,
$params
,
$output
)
=
$self
->
{'
jobManager
'}
->
get_job
(
$id
);
if
(
ref
$job
ne
'
HASH
'
||
!
$job
->
{'
id
'}){
print
"
<div class=
\"
box
\"
id=
\"
statusbad
\"
>
\n
";
print
"
<p>The submitted job does not exist.</p>
\n
";
print
"
</div>
\n
";
return
;
}
(
my
$submit_time
=
$job
->
{'
submit_time
'})
=~
s/\.\d+$//
;
#remove fractions of second
(
my
$start_time
=
$job
->
{'
start_time
'})
=~
s/\.\d+$//
;
(
my
$stop_time
=
$job
->
{'
stop_time
'})
=~
s/\.\d+$//
;
print
<<
"
HTML
";
<
div
class
=
"
box
"
id
=
"
resultstable
"
>
<
h2
>
Status
</
h2
>
<
table
class
=
"
resultstable
"
>
<
tr class="td1"><th style="text-align:right">Job id: </th><td style="text-align:left">$id</td></tr>
<tr class="td2"><th style="text-align:right">Submit time: </th><td style="text-align:left">$submit_time</td></tr>
<tr class
=
"
td1
"
><
th
style
=
"
text-align:right
"
>
Status:
<
/th><td style="text-align:left">$job->{'status'}</
td
></
tr>
<tr class="td2"><th style="text-align:right">
Start
time
:
<
/th><td style="text-align:left">$start_time</
td
></
tr>
<tr class="td1"><th style="text-align:right">
%
complete:
<
/th><td style="text-align:left">$job->{'percent_complete'}</
td
></
tr>
<tr class="td2"><th style="text-align:right">
Stop
time
:
<
/th><td style="text-align:left">$stop_time</
td
></
tr>
</table>
<h2>
Output
</
h2
>
HTML
if
(
!
(
$job
->
{'
message_html
'}
||
ref
$output
eq
'
HASH
')){
print
"
<p>No output yet.</p>
\n
";
}
else
{
print
"
$job
->{'message_html'}
";
}
print
"
<p>This page will reload in
$self
->{'refresh'} seconds. You can refresh it any time, or bookmark it and close your browser if you wish.</p>
\n
";
print
"
</div>
\n
";
}
sub
get_title
{
my
(
$self
)
=
@_
;
my
$desc
=
$self
->
{'
system
'}
->
{'
description
'}
||
'
BIGSdb
';
return
"
Job status viewer -
$desc
";
}
1
;
\ No newline at end of file
lib/BIGSdb/OfflineJobManager.pm
View file @
a232271e
...
...
@@ -21,10 +21,10 @@ use strict;
use
base
qw(BIGSdb::Application)
;
use
Error
qw(:try)
;
use
Log::
Log4perl
qw(get_logger)
;
my
$logger
=
get_logger
('
BIGSdb.Job
');
sub
new
{
#The job manager uses its own Dataconnector since it may be called by a stand-alone script.
my
(
$class
,
$config_dir
,
$plugin_dir
,
$dbase_config_dir
,
$host
,
$port
,
$user
,
$password
)
=
@_
;
my
$self
=
{};
...
...
@@ -75,6 +75,7 @@ sub _db_connect {
}
sub
add_job
{
#Required params:
#dbase_config: name of db configuration direction in /etc/dbases
#ip_address: connecting address
...
...
@@ -93,17 +94,17 @@ sub add_job {
}
}
$params
->
{'
priority
'}
=
5
if
!
$params
->
{'
priority
'};
my
$id
=
BIGSdb::Utils::
get_random
();
my
$id
=
BIGSdb::Utils::
get_random
();
my
$cgi_params
=
$params
->
{'
parameters
'};
if
(
ref
$cgi_params
ne
'
HASH
')
{
if
(
ref
$cgi_params
ne
'
HASH
'
)
{
$logger
->
error
("
CGI parameters not passed as a ref
");
throw
BIGSdb::
DataException
("
CGI parameters not passed as a ref
");
}
eval
{
$self
->
{'
db
'}
->
do
(
"
INSERT INTO jobs (id,dbase_config,username,email,ip_address,submit_time,module,status,percent_complete,priority) VALUES (?,?,?,?,?,?,?,?,?,?)
",
undef
,
$id
,
undef
,
$id
,
$params
->
{'
dbase_config
'},
$params
->
{'
username
'},
$params
->
{'
email
'},
...
...
@@ -115,13 +116,36 @@ sub add_job {
$params
->
{'
priority
'}
);
my
$param_sql
=
$self
->
{'
db
'}
->
prepare
("
INSERT INTO params (job_id,key,value) VALUES (?,?,?)
");
$"
=
'
||
';
foreach
(
keys
%$cgi_params
)
{
my
@values
=
split
(
"
\
0
",
$cgi_params
->
{
$_
}
);
$param_sql
->
execute
(
$id
,
$_
,"
@values
"
);
$"
=
'
||
';
foreach
(
keys
%$cgi_params
)
{
my
@values
=
split
(
"
\
0
",
$cgi_params
->
{
$_
}
);
$param_sql
->
execute
(
$id
,
$_
,
"
@values
"
);
}
};
if
(
$@
){
if
(
$@
)
{
$logger
->
error
(
$@
);
$self
->
{'
db
'}
->
rollback
;
}
else
{
$self
->
{'
db
'}
->
commit
;
}
return
$id
;
}
sub
update_job_output
{
my
(
$self
,
$job_id
,
$output_hash
)
=
@_
;
if
(
ref
$output_hash
ne
'
HASH
'
)
{
$logger
->
error
("
status hash not passed as a ref
");
throw
BIGSdb::
DataException
("
status hash not passed as a ref
");
}
eval
{
$self
->
{'
db
'}
->
do
(
"
UPDATE output SET filename=?, description=? WHERE job_id=?
",
undef
,
$output_hash
->
{'
filename
'},
$output_hash
->
{'
description
'},
$job_id
);
};
if
(
$@
)
{
$logger
->
error
(
$@
);
$self
->
{'
db
'}
->
rollback
;
}
else
{
...
...
@@ -130,18 +154,19 @@ sub add_job {
}
sub
update_job_status
{
my
(
$self
,
$job_id
,
$status_hash
)
=
@_
;
if
(
ref
$status_hash
ne
'
HASH
')
{
my
(
$self
,
$job_id
,
$status_hash
)
=
@_
;
if
(
ref
$status_hash
ne
'
HASH
'
)
{
$logger
->
error
("
status hash not passed as a ref
");
throw
BIGSdb::
DataException
("
status hash not passed as a ref
");
}
eval
{
foreach
(
keys
%$status_hash
){
$self
->
{'
db
'}
->
do
("
UPDATE jobs SET
$_
=? WHERE id=?
",
undef
,
$status_hash
->
{
$_
},
$job_id
);
$logger
->
debug
("
$job_id
$_
:
$status_hash
->{
$_
}
")
foreach
(
keys
%$status_hash
)
{
$self
->
{'
db
'}
->
do
(
"
UPDATE jobs SET
$_
=? WHERE id=?
",
undef
,
$status_hash
->
{
$_
},
$job_id
);
$logger
->
debug
("
$job_id
$_
:
$status_hash
->{
$_
}
");
}
};
if
(
$@
){
if
(
$@
)
{
$logger
->
error
(
$@
);
$self
->
{'
db
'}
->
rollback
;
}
else
{
...
...
@@ -150,54 +175,56 @@ sub update_job_status {
}
sub
get_job
{
my
(
$self
,
$job_id
)
=
@_
;
my
(
$self
,
$job_id
)
=
@_
;
my
$sql
=
$self
->
{'
db
'}
->
prepare
("
SELECT * FROM jobs WHERE id=?
");
eval
{
$sql
->
execute
(
$job_id
);
};
if
(
$@
){
eval
{
$sql
->
execute
(
$job_id
);
};
if
(
$@
)
{
$logger
->
error
(
$@
);
return
;
}
my
$job
=
$sql
->
fetchrow_hashref
;
$sql
=
$self
->
{'
db
'}
->
prepare
("
SELECT key,value FROM params WHERE job_id=?
");
my
$params
;
eval
{
$sql
->
execute
(
$job
->
{'
id
'});
};
if
(
$@
){
my
(
$params
,
$output
);
eval
{
$sql
->
execute
(
$job
->
{'
id
'}
);
};
if
(
$@
)
{
$logger
->
error
(
$@
);
return
;
}
while
(
my
(
$key
,
$value
)
=
$sql
->
fetchrow_array
)
{
while
(
my
(
$key
,
$value
)
=
$sql
->
fetchrow_array
)
{
$params
->
{
$key
}
=
$value
;
}
return
(
$job
,
$params
);
$sql
=
$self
->
{'
db
'}
->
prepare
("
SELECT filename,description FROM output WHERE job_id=?
");
eval
{
$sql
->
execute
(
$job
->
{'
id
'}
);
};
if
(
$@
)
{
$logger
->
error
(
$@
);
return
;
}
while
(
my
(
$filename
,
$desc
)
=
$sql
->
fetchrow_array
)
{
$output
->
{
$desc
}
=
$filename
;
}
return
(
$job
,
$params
,
$output
);
}
sub
get_next_job_id
{
my
(
$self
)
=
@_
;
my
$sql
=
$self
->
{'
db
'}
->
prepare
("
SELECT id FROM jobs WHERE status='submitted' ORDER BY priority asc,submit_time asc LIMIT 1
");
eval
{
$sql
->
execute
;
};
if
(
$@
){
eval
{
$sql
->
execute
;
};
if
(
$@
)
{
$logger
->
error
(
$@
);
return
;
}
my
(
$job
)
=
$sql
->
fetchrow_array
;
return
$job
;
return
$job
;
}
sub
get_load_average
{
my
$uptime
=
`
uptime
`;
my
$load
;
if
(
$uptime
=~
/load average:\s+([\d\.]+)/
)
{
if
(
$uptime
=~
/load average:\s+([\d\.]+)/
)
{
$load
=
$
1
;
}
else
{
throw
BIGSdb::
DataException
("
Can't determine load average
");
}
return
$load
;
}
1
;
lib/BIGSdb/Page.pm
View file @
a232271e
...
...
@@ -190,16 +190,21 @@ sub print {
close
$fh
;
}
}
my
$http_equiv
;
if
(
$self
->
{'
refresh
'}){
$http_equiv
=
"
<meta http-equiv=
\"
refresh
\"
content=
\"
$self
->{'refresh'}
\"
/>
";
}
if
(
%shortcut_icon
)
{
print
$q
->
start_html
(
-
title
=>
$title
,
-
meta
=>
{
%meta_content
},
-
style
=>
{
-
src
=>
$stylesheet
},
-
head
=>
CGI
->
Link
(
{
%shortcut_icon
}
)
,
-
head
=>
[
CGI
->
Link
(
{
%shortcut_icon
}
),
$http_equiv
]
,
-
script
=>
\
@javascript
);
}
else
{
print
$q
->
start_html
(
-
title
=>
$title
,
-
meta
=>
{
%meta_content
},
-
style
=>
{
-
src
=>
$stylesheet
},
-
script
=>
\
@javascript
);
print
$q
->
start_html
(
-
title
=>
$title
,
-
meta
=>
{
%meta_content
},
-
style
=>
{
-
src
=>
$stylesheet
},
-
script
=>
\
@javascript
,
-
head
=>
$http_equiv
);
}
$self
->
_print_header
();
$self
->
_print_login_details
if
$self
->
{'
system
'}
->
{'
read_access
'}
ne
'
public
'
||
$self
->
{'
curate
'};
...
...
lib/BIGSdb/Plugins/XmfaExport.pm
View file @
a232271e
...
...
@@ -111,19 +111,11 @@ sub run {
if
(
!
@fields_selected
)
{
print
"
<div class=
\"
box
\"
id=
\"
statusbad
\"
><p>No fields have been selected!</p></div>
\n
";
}
else
{
print
<<"HTML";
<div class="box" id="resultstable">
<p>The output file has been submitted to the job queue.</p>
<p>Please be aware that this job may take a long time depending on the number of sequences to align.</p>
</div>
HTML
# my $filename = ( BIGSdb::Utils::get_random() ) . '.txt';
# my $full_path = "$self->{'config'}->{'tmp_dir'}/$filename";
my
$params
=
$q
->
Vars
;
$params
->
{'
pk
'}
=
$pk
;
(
my
$list
=
$q
->
param
('
list
'))
=~
s/[\r\n]+/\|\|/g
;
$params
->
{'
list
'}
=
$list
;
$self
->
{'
jobManager
'}
->
add_job
(
my
$job_id
=
$self
->
{'
jobManager
'}
->
add_job
(
{
'
dbase_config
'
=>
$self
->
{'
instance
'},
'
ip_address
'
=>
$q
->
remote_host
,
...
...
@@ -132,6 +124,16 @@ HTML
'
parameters
'
=>
$params
}
);
print
<<"HTML";
<div class="box" id="resultstable">
<p>This analysis has been submitted to the job queue.</p>
<p>Please be aware that this job may take a long time depending on the number of sequences to align
and how busy the server is. Alignment of hundreds of sequences can take many hours!</p>
<p><a href="$self->{'script_name'}?db=$self->{'instance'}&page=job&id=$job_id">
Follow the progress of this job and view the output.</a></p>
</div>
HTML
# $| = 1;
# my ( $problem_ids, $no_output ) = $self->_write_xmfa( $list, \@fields_selected, $full_path, $pk );
...
...
sql/jobs.sql
View file @
a232271e
...
...
@@ -10,7 +10,7 @@ stop_time timestamp,
module
text
NOT
NULL
,
status
text
NOT
NULL
,
percent_complete
int
,
message
text
,
message
_html
text
,
priority
int
NOT
NULL
,
PRIMARY
KEY
(
id
)
);
...
...
@@ -29,3 +29,14 @@ ON UPDATE CASCADE
GRANT
SELECT
,
UPDATE
,
INSERT
,
DELETE
ON
params
TO
apache
,
bigsdb
;
CREATE
TABLE
output
(
job_id
text
NOT
NULL
,
filename
text
NOT
NULL
,
description
text
NOT
NULL
,
PRIMARY
KEY
(
job_id
),
CONSTRAINT
o_job_id
FOREIGN
KEY
(
job_id
)
REFERENCES
jobs
ON
DELETE
CASCADE
ON
UPDATE
CASCADE
);
GRANT
SELECT
,
UPDATE
,
INSERT
,
DELETE
ON
output
TO
apache
,
bigsdb
;
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment