Question: Hello Newbie Questions About Galaxy And Clustering
0
gravatar for leconte
5.7 years ago by
leconte • 20
leconte • 20 wrote:
hello I have installed an galaxy instance for purpose testing on a 8 CPU server without problem all run perfectly. I have made some galaxy xml wrapper to understant how galaxy calls program and other little things. Now I want to install a Galaxy server with a cluster configuration. Before I have tested HTCondor but I have no Idea how I can mix Galaxy and HTCondor. The perfect thing was somebody have already tested this type of condor. But I can satisfy myself with any other cluster type. What I don't understant in fact ( maybe my worst problem ) where lie all parts of this puzzle. I explain. I know that I need Galaxy. I know too I need a cluster ( in my case HTCondor ) I understand I need something between these 2 parts ... I think something like PBS or SGE but it's not totaly clear for me I'm search on galaxy mailling list archive ... but I just found 2 mails in 2008 :( has anybody made this conf or have a configuration ? greetings PS : sorry for my poor english :)
galaxy • 997 views
ADD COMMENT • link • modified 5.6 years ago • written 5.7 years ago by leconte • 20
0
gravatar for leconte
5.6 years ago by
leconte • 20
leconte • 20 wrote:
hello, My subject is at the border between Galaxy and Condor I'found method to use Galaxy with condor cluster solution. In fact I have wrote a script when Galaxy call clustalw2 , it calls in fact a script called clustalw2. All run exactly as I want excepting ... but Galaxy get in error and said there was a problem I turn my script to debug mode and it puts followings output + prog=/usr/bin/clustalw2 + err=0 + infile=-infile=/home/galaxy/galaxy- dist/database/files/000/dataset_11.dat + outfile=-outfile=/home/galaxy/galaxy- dist/database/job_working_directory/000/46/galaxy_dataset_80.dat + outorder=-OUTORDER=ALIGNED + seqnos=-SEQNOS=OFF + typeseq=-TYPE=DNA + var6= + var7= + var8= + var9= + echo ++ echo -infile=/home/galaxy/galaxy- dist/database/files/000/dataset_11.dat ++ sed -e s/-infile=// + infile2=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat ++ sed -e 's/galaxy_dataset_\(.*\)\.dat//' ++ sed -e s/-outfile=// ++ echo -outfile=/home/galaxy/galaxy- dist/database/job_working_directory/000/46/galaxy_dataset_80.dat + WorkingDir=/home/galaxy/galaxy- dist/database/job_working_directory/000/46/ + mkdir -p /home/galaxy/galaxy- dist/database/job_working_directory/000/46/ + mkdir -p /home/galaxy/galaxy- dist/database/job_working_directory/000/46/ + mkdir -p /home/galaxy/galaxy- dist/database/job_working_directory/000/46/ + [[ -n /usr/bin/clustalw2==clustalw2 ]] + echo 'universe = vanilla' + echo 'executable = /usr/bin/mpirun ' + echo 'arguments = "-np 10 -hostfile /home/galaxy/hostfile /usr/bin/clustalw-mpi -infile=/home/galaxy /galaxy-dist/database/files/000/dataset_11.dat -outfile=/home/galaxy /galaxy- dist/database/job_working_directory/000/46/galaxy_dataset_80.dat -OUTORDER=ALIGNED -SEQNOS=OFF -TYPE=DNA "' + echo 'Getenv = True machine_count = 1 transfer_input_files = /home/galaxy/hostfile,/usr/bin/clustalw-mpi, /home/galaxy/galaxy-dist/database/files/000/dataset_11.dat should_transfer_files = yes Log = /home/galaxy/logs/normal.$(cluster).$(process).log Output = /home/galaxy/logs/normal.$(cluster).$(process).out Error = /home/galaxy/logs/normal.$(cluster).$(process).error notification = never queue 1' + /usr/bin/condor_submit /home/galaxy/submit Submitting job(s). 1 job(s) submitted to cluster 179. I suspect the method I use, doesn't return properly information to galaxy ... but I don't know what I can do ... Any Idea ...
ADD COMMENT • link written 5.6 years ago by leconte • 20
Hello, I've moved this over to the galaxy-dev list since it concerns a local installation. You'll want to use the 'condor' job running plugin, which is provided with the Galaxy distribution. Have a look at the job_conf.xml.sample_advanced file to see how this would be configured. A very basic job_conf.xml that should submit all jobs to HTCondor: <job_conf> <plugins workers="4"> <plugin id="condor" type="runner" load="galaxy.jobs.runners.condor:CondorJobRunner"/> </plugins> <destinations default="condor"> <destination id="condor" runner="condor"/> </destinations> </job_conf> --nate
ADD REPLY • link written 5.6 years ago by Nate Coraor ♦ 3.2k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 182 users visited in the last hour