how to find out dead links

[Loïc Grenié]

2009/11/14 Eugeneapolinary Ju <eugeneapolinary81 at yahoo.com>:
> wget -r -p -U Firefox "http://www.somesite.com/" 2>&1 | grep 404 > 404.txt
>
>
> why come 404.txt is 0 Byte? how to put the STDOUT to a file with wget?

   Have you tried

wget -r -p -U Firefox "http://www.somesite.com/"

   There is no 404 message (at least here). To be more precise, there is
  no 404 message because there is no web server that can output the
  404 message. A web page can fail for (at least) three different reasons:

1) non-existing address
2) crashed or overloaded web server
3) non-existing page

   In case 3) you'll get a 404 message, in case 1) you'll get basically
  nothing and in case 2) it depends... This means that you "grep 404"
  will get only case 3).

      Hope this helps somwhat,

           Loïc