install wget
apt-get install wget
wget mac robot
wget -e robots=off -r -np https://www.w3.org/History/19921103-hypertext/hypertext/
wget
The simplest way to use wget is to provide it with the location of a file to download over HTTP. For example, to download the file http://website.com/files/file.zip, this command:
$ wget http://website.com/files/file.zip
...would download the file into the working directory.
There are many options that allow you to use wget in different ways, for different purposes.
wget -
wget --quiet http://www.mysite.com/sitemap.xml --output-document - | egrep -o "https?://[^<]+" | wget -i -
Copyright © 2021 Codeinu
Forgot your account's password or having trouble logging into your Account? Don't worry, we'll help you to get back your account. Enter your email address and we'll send you a recovery link to reset your password. If you are experiencing problems resetting your password contact us