Toolbox Aid

August 9th, 2024 11:27 AM Mr. Q Categories: Command

Command: wget

Used to download files from the web. wget supports HTTP, HTTPS, and FTP protocols, and can handle file retrieval in various ways.

---

Sample Command and Output:

$ wget http://example.com/file.txt
--2024-08-09 12:34:56--  http://example.com/file.txt
Resolving example.com (example.com)... 93.184.216.34
Connecting to example.com (example.com)|93.184.216.34|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12345 (12K) [text/plain]
Saving to: ‘file.txt’

file.txt          100%[===================>]  12.06K  --.-KB/s    in 0.01s   

2024-08-09 12:34:56 (1.20 MB/s) - ‘file.txt’ saved [12345/12345]

Description:

• wget http://example.com/file.txt: Downloads file.txt from http://example.com to the current directory. The output shows the download progress and completion status.

---

Sample Command and Output:

$ wget -O newfile.txt http://example.com/file.txt
--2024-08-09 12:35:12--  http://example.com/file.txt
Saving to: ‘newfile.txt’

newfile.txt         100%[===================>]  12.06K  --.-KB/s    in 0.01s   

2024-08-09 12:35:12 (1.20 MB/s) - ‘newfile.txt’ saved [12345/12345]

Description:

• wget -O newfile.txt http://example.com/file.txt: Downloads file.txt from http://example.com and saves it as newfile.txt. The -O option specifies the output file name.

---

Sample Command and Output:

$ wget -r http://example.com/directory/
--2024-08-09 12:36:23--  http://example.com/directory/
Resolving example.com (example.com)... 93.184.216.34
Connecting to example.com (example.com)|93.184.216.34|:80... connected.
HTTP request sent, awaiting response... 200 OK
...
Downloading http://example.com/directory/file1.txt
Downloading http://example.com/directory/subdir/file2.txt
...

Description:

• wget -r http://example.com/directory/: Recursively downloads files from the specified directory on the web server. The -r option enables recursive retrieval, downloading all files and directories within the specified URL.

---

Sample Command and Output:

$ wget -c http://example.com/largefile.zip
--2024-08-09 12:37:45--  http://example.com/largefile.zip
Resuming transfer from byte 123456
...

Description:

• wget -c http://example.com/largefile.zip: Resumes a partially downloaded file from where it left off. The -c option is useful for continuing interrupted downloads.

---

Additional Arguments:

• -q: Suppresses output messages, making the command operate in quiet mode.
• –limit-rate=200k: Limits the download speed to 200 KB/s.

Sample Command and Output:

$ wget -q --limit-rate=200k http://example.com/file.txt

Description:

• wget -q --limit-rate=200k http://example.com/file.txt: Downloads file.txt while limiting the download speed to 200 KB/s and suppressing output messages. The -q option quiets the command output, and --limit-rate restricts the speed.

---

Rediredt output log to file for headless operations

-o /volume1/wget.log

---

Direct wget files to specific path

-P Downloaded files will be placed on path /volume1/{site URL}

-P /volume1/

---

If the site you are downloading has throttling, this option will help on failures

--waitretry=1