Question: Job was reported by drmaa as terminal but job state in SLURM is: PENDING, returning to monitor queue
0
gravatar for chadmatsalla
11 months ago by
chadmatsalla • 40
chadmatsalla • 40 wrote:

I have a brand new install of slurm with two nodes. I just managed to get slurm working.

Galaxy web reports no jobs.

Galaxy's main logs are filling with:

galaxy.jobs.runners.slurm WARNING 2017-12-06 15:50:59,820 (91/40) Job was reported by drmaa as terminal but job state in SLURM is: PENDING, returning to monitor queue galaxy.jobs.runners.drmaa DEBUG 2017-12-06 15:51:00,830 (91/40) state change: job finished, but failed

I can't seem to find out what to do about this.

 root@slurm-controller:~# squeue
         JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)

root@slurm-controller:~# sacct -s PD,R
   JobID    JobName  Partition    Account  AllocCPUS      State ExitCode
------------ ---------- ---------- ---------- ---------- ---------- --------
40           g91_uploa+    bioinfo                     1    PENDING      0:0

Is this a slurm problem? A Galaxy problem?

Thanks!

Chad Matsalla

ADD COMMENT • link • modified 11 months ago • written 11 months ago by chadmatsalla • 40
0
gravatar for Nate Coraor
11 months ago by
Nate Coraor ♦ 3.2k
United States
Nate Coraor ♦ 3.2k wrote:

I'm not sure why the drmaa library is considering PENDING jobs as terminal. What version of slurm-drmaa are you using?

ADD COMMENT • link written 11 months ago by Nate Coraor ♦ 3.2k
0
gravatar for chadmatsalla
11 months ago by
chadmatsalla • 40
chadmatsalla • 40 wrote:

Thanks for your help.

To fix this I used the well-worn strategy of "turning it off and turning it back on". I restarted the entire slurm apparatus and the entire galaxy apparatus.

The messages have stopped but sacct still shows that job as pending. I'm not really sure how to deal with that given that it's now showing in squeue.

ADD COMMENT • link written 11 months ago by chadmatsalla • 40
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 169 users visited in the last hour