Технологическое извлечение с помощью URL веб-сайта

Можно изучить дистрибутивы Kali или Parrot для информационных инструментов сбора.

• nikto один из них, что я попробовал прежде и даю частичную информацию. Это доступно, репозиторий Ubuntu также.

  ~$ whatis nikto
  nikto (1)            - Scan web server for known vulnerabilities
  
  ~$ sudo apt-get install nikto
  ~$ sudo nikto -update
  ~$ nikto -Tuning b -h www.wikipedia.org
  - Nikto v2.1.5
  ---------------------------------------------------------------------------
  + Target IP:          91.198.174.192
  + Target Hostname:    www.wikipedia.org
  + Target Port:        80
  + Start Time:         2016-11-14 09:22:30 (GMT1)
  ---------------------------------------------------------------------------
  + Server: Varnish
  + IP address found in the 'x-client-ip' header. The IP is "105.107.105.185".
  + The anti-clickjacking X-Frame-Options header is not present.
  + Uncommon header 'x-client-ip' found, with contents: 105.107.105.185
  + Uncommon header 'x-cache' found, with contents: cp3041 int
  + Uncommon header 'x-varnish' found, with contents: 827655138
  + Uncommon header 'x-cache-status' found, with contents: int
  + Root page / redirects to: https://www.wikipedia.org/
  + No CGI Directories found (use '-C all' to force check all possible dirs)
  + Server banner has changed from 'Varnish' to 'mw1187.eqiad.wmnet' which may suggest a WAF, load balancer or proxy is in place
  + Cookie GeoIP created without the httponly flag
  + Retrieved via header: 1.1 varnish-v4, 1.1 varnish-v4, 1.1 varnish-v4
  + Retrieved x-powered-by header: HHVM/3.3.0-static
  + Server leaks inodes via ETags, header found with file /, fields: 0xW/3b2 0x5369720eefb07 
  + Uncommon header 'x-analytics' found, with contents: nocookies=1
  + Uncommon header 'backend-timing' found, with contents: D=236 t=1478774110870502
  + 269 items checked: 0 error(s) and 12 item(s) reported on remote host
  + End Time:           2016-11-14 09:23:21 (GMT1) (51 seconds)
  ---------------------------------------------------------------------------
  + 1 host(s) tested
  

• whatweb другой инструмент. Это не исправило ошибку (недопустимая многобайтовая ошибка Escape) в Ubuntu.

  1. Открытый файл кодирования библиотеки автоматического обнаружения для редактирования

     sudo nano /usr/lib/ruby/vendor_ruby/rchardet/universaldetector.rb
     

  2. Добавить # encoding: US-ASCII

  
  

  Даже с этим обходным решением выше, вывод не является чистым как в Kali.

  ~$ whatis whatweb
  whatweb (1)          - Web scanner to identify what websites are running.
  
  ~$ whatweb www.wikipedia.org
  /usr/share/whatweb/lib/tld.rb:85: warning: key "2nd_level_registration" is duplicated and overwritten on line 85
  /usr/share/whatweb/lib/tld.rb:93: warning: key "2nd_level_registration" is duplicated and overwritten on line 93
  /usr/share/whatweb/lib/tld.rb:95: warning: key "2nd_level_registration" is duplicated and overwritten on line 95
  /usr/share/whatweb/plugins/wordpress.rb:436: warning: key "2.7-beta1" is duplicated and overwritten on line 453
  /usr/share/whatweb/lib/extend-http.rb:102:in `connect': Object#timeout is deprecated, use Timeout.timeout instead.
  http://www.wikipedia.org [301] Cookies[WMF-Last-Access], Country[NETHERLANDS][NL], HTTPServer[Varnish], HttpOnly[WMF-Last-Access], IP[91.198.174.192], RedirectLocation[https://www.wikipedia.org/], UncommonHeaders[x-varnish,x-cache-status,x-client-ip], Varnish
  /usr/share/whatweb/lib/extend-http.rb:102:in `connect': Object#timeout is deprecated, use Timeout.timeout instead.
  /usr/share/whatweb/lib/extend-http.rb:140:in `connect': Object#timeout is deprecated, use Timeout.timeout instead.
  https://www.wikipedia.org/ [200] Cookies[GeoIP,WMF-Last-Access], Country[NETHERLANDS][NL], Email[Wikipedia-logo-v2@1.5x.png,Wikipedia-logo-v2@2x.png,Wikipedia_wordmark@1.5x.png,Wikipedia_wordmark@2x.png,sprite-bookshelf_icons@1.5x.png,sprite-bookshelf_icons@2x.png,sprite-project-logos@1.5x.png,sprite-project-logos@2x.png], HTML5, HTTPServer[mw1253.eqiad.wmnet], HttpOnly[WMF-Last-Access], IP[91.198.174.192], probably MediaWiki, Script, Title[Wikipedia], UncommonHeaders[backend-timing,x-varnish,x-cache-status,strict-transport-security,x-analytics,x-client-ip], Varnish, Via-Proxy[1.1 varnish-v4, 1.1 varnish-v4, 1.1 varnish-v4], X-Powered-By[HHVM/3.3.0-static]
  

  Вывод от Kali:

  ~# whatweb https://www.wikipedia.org
  https://www.wikipedia.org [200 OK] Cookies[GeoIP,WMF-Last-Access], Country[NETHERLANDS][NL], Email[Wikipedia-logo-v2@1.5x.png,Wikipedia-logo-v2@2x.png,Wikipedia_wordmark@1.5x.png,Wikipedia_wordmark@2x.png,sprite-bookshelf_icons@1.5x.png,sprite-bookshelf_icons@2x.png,sprite-project-logos@1.5x.png,sprite-project-logos@2x.png], HTML5, HTTPServer[mw1253.eqiad.wmnet], HttpOnly[WMF-Last-Access], IP[91.198.174.192], probably MediaWiki, Script, Strict-Transport-Security[max-age=31536000; includeSubDomains; preload], Title[Wikipedia], UncommonHeaders[backend-timing,x-varnish,x-cache-status,x-analytics,x-client-ip], Varnish, Via-Proxy[1.1 varnish-v4, 1.1 varnish-v4, 1.1 varnish-v4], X-Powered-By[HHVM/3.3.0-static]