Question: How to download a batch consisting of 50 bam files, using CURL or wget
1
gravatar for ppurkayastha2010
8 months ago by
ppurkayastha201030 wrote:

How could I download a batch consists of 50 bam files, using CURL or wget.

I tried the following solution,

https://biostar.usegalaxy.org/p/26527/

but still is not working :(

wget -O --no-check-certificate 'https://usegalaxy.org/api/dataset_collections/d103ecff2e1c7b62/download'
--2018-04-04 01:59:36--  https://usegalaxy.org/api/dataset_collections/d103ecff2e1c7b62/download
Resolving usegalaxy.org usegalaxy.org)... 129.114.60.56, 129.114.60.60
Connecting to usegalaxy.org usegalaxy.org)|129.114.60.56|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2018-04-04 01:59:37 ERROR 403: Forbidden.

I tried doing all the listed options in the following link for big data

https://galaxyproject.org/support/download-data/#download-tip-big-data

but still it is giving me the same error.

curl linux galaxy wget • 442 views
ADD COMMENTlink modified 7 months ago by Martin Čech ♦♦ 4.9k • written 8 months ago by ppurkayastha201030
2
gravatar for Martin Čech
7 months ago by
Martin Čech ♦♦ 4.9k
United States
Martin Čech ♦♦ 4.9k wrote:

For dataset collections and datasets within collections you have to supply your API key with the request. You can add it to the end of the collection download url, the command would look something like this:

$ wget https://usegalaxy.org/api/dataset_collections/d20ad3e1ccd4595de/download?key=MYSECRETAPIKEY (note the ?key= before the API key itself)

I updated our docs at https://galaxyproject.org/support/download-data to reflect this requirement.

ADD COMMENTlink written 7 months ago by Martin Čech ♦♦ 4.9k

Thanks a lot :-) I could download the files now..

ADD REPLYlink written 7 months ago by ppurkayastha201030

but this is not allowing to download more than a GB data.

Showing an error

Read error at byte 1079804844 (Success).Retrying.

ADD REPLYlink modified 7 months ago • written 7 months ago by ppurkayastha201030

Related post closed out as a duplicate with more details: https://biostar.usegalaxy.org/p/27392/

Quote:

I am trying to download a data-set consisting of 50 bam files, I use the following command to download

$ wget https://usegalaxy.org/api/dataset_collections/d20ad3e1ccd4595de/download?key=MYSECRETAPIKEY

but I am unable to download the data beyond 1 GB.

I tried to download using Curl, but I still get the following error

curl -O 'https://usegalaxy.org/api/dataset_collections/e3f40e4987/download?key=MYSECRETKEY'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1029M    0 1029M    0     0   256k      0 --:--:--  1:08:26 --:--:--  259k
curl: (18) transfer closed with outstanding read data remaining

Please help

ADD REPLYlink written 7 months ago by Jennifer Hillman Jackson25k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 175 users visited in the last hour