Job is waiting to run -- Have the clusters been particularly busy this weekend?

Question: Job is waiting to run -- Have the clusters been particularly busy this weekend?

4.6 years ago by

lb465 • 0

United Kingdom

lb465 • 0 wrote:

Jobs that I tried to run over the weekend or this morning have all held at 'Job is waiting to run'. Is this just because the server has been particularly busy or is there some other problem?

galaxy • 1.2k views

ADD COMMENT • link •

modified 4.6 years ago by Jennifer Hillman Jackson ♦ 25k • written 4.6 years ago by lb465 • 0

The queue for large jobs seems very bust atm. Please be patient.

ADD REPLY • link written 4.6 years ago by Martin Čech ♦♦ 4.9k

4.6 years ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

Are you working on http://usegalaxy.org (Main)? Logged into your account? And are not over your data quota (check under "User -> Preferences)?

All of of this is a "yes", then it would be normal to have jobs that use more resources (in general the tools found under groups staring with an "NGS:") queue into a waiting state while they wait for their turn to execute (this weekend or ever). Other jobs such as Text Manipulation may start right away (if not dependent on an upstream job that is queued). Different job types are sent to different types of nodes on our cluster. That all said, the NGS queue has been very busy, as Martin notes.

If you are not using a registered account, then only one job at a time will be active. A job holding a place in the queue counts as an active job. And if you are on another server, how the processing is done can vary.

https://wiki.galaxyproject.org/Main
https://wiki.galaxyproject.org/Support#Dataset_status_and_how_jobs_execute

It is not clear if you tried to deleted/restart jobs from over the weekend or not, but this is best avoided and will extended wait times.

Best, Jen, Galaxy team

ADD COMMENT • link written 4.6 years ago by Jennifer Hillman Jackson ♦ 25k

I have a similar problem. I'm using a registered account on usegalaxy.org. I have 4 jobs queued for about a day and still hasn't started running yet, 2 of them are trim leading/trailing characters and the other 2 are fastq groomers. I tried local instance, but I notice that my local instance is missing a lot of tools that usegalaxy.org has, such as 'Filter FASTQ reads by quality score and length'. Where can I find the scripts/codes of such tool so I can just run it on python?

ADD REPLY • link written 4.5 years ago by mushroom.tree • 0

Hi - When the queue is really busy, some jobs may queue up to a day, sometimes longer if we are really busy (but this is not always the case. All of the jobs you mentioned are in the NGS group, so all go to the busiest cluster. Jobs are processed in the order submitted, plus one at a time per user, to ensure fair access. If you happen to have jobs queued in another history, or deleted jobs that were in progress, those can slow you down. The best strategy is to queue the jobs you need and then let them run to completion. I know I saw this quite often but it really is true :)

Your other question should be in a new post next time please. The Galaxy distribution does not contain all tools by default, This is to keep it light-weight. Add new tools from the Tool Shed. These wikis guide you through the process.

Step 1: move to a local production instance set-up or consider a cloud Galaxy
http://getgalaxy.org
http://usegalaxy.org/cloud

Step 2: Add tools from the Tool Shed
http://usegalaxy.org/toolshed

You can configure where you want the underlying tools to be, but "tool-data" is a common choice. See the "Admin" section of our wiki for more details.

Thanks! Jen, Galaxy team

ADD REPLY • link written 4.5 years ago by Jennifer Hillman Jackson ♦ 25k

... i installed my local instance using hg clone, and 'hg update default', so i believe i have the latest version?

ADD REPLY • link written 4.5 years ago by mushroom.tree • 0

No, if you want the latest distribution, use "hg update stable". See this section of the Get Galaxy wiki:
https://wiki.galaxyproject.org/Admin/GetGalaxy#Get_the_latest_copy_from_the_repository

News Brief's have links to more resources, as do the "Admin" pages in the side bar of our wiki.

Other Public Servers may be an option. Which permit public login and how much quota/what tools are available changes through time, so you'll need to review to see if there is a good fit: https://wiki.galaxyproject.org/PublicGalaxyServers

Good luck! Jen, Galaxy team

ADD REPLY • link written 4.5 years ago by Jennifer Hillman Jackson ♦ 25k

Similar posts • Search »