Directory traversal using ROBOTS.txt

Hi Friends first of all let me tell u What is Directory Transversal
Directory Transversal allows you to change what directory you are located in by typing in the URL bar. This allows you to access pages you normally can't on an insecure webpage.
Now what is robots.txt-

"Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol."
It works likes this: a robot wants to vists a Web site URL, say http://www.example.com/welcome.html. Before it does so, it firsts checks for http://www.example.com/robots.txt, and finds:

User-agent: *
Disallow: /

The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
==>Now next step is Finding disallowed pages

This is quite simple. Go to the main page and type in:



Http://www.[hostname].ext/robots.txt

In this textfile, you will see something that looks like this:

User-agent: *

Allow: /searchhistory/

Disallow: /news?output=xhtml&

Allow: /news?output=xhtml

Disallow: /search

Disallow: /groups

Disallow: /images

Disallow: /catalogs


>>User-Agent can be something else then* this means all. This can be a type of webbrowser. That would block the user agent.

Now our last step is to access pages since we can traverse those directories which are allowed but what about those which are disallowed for traversal. here we can try a trick that may help you to get into the restricted directory:


Type a directory that comes after CODE:

Disallow:

Chances are, you will get denied.

To get access you could see if you can mod your cookie, but that's not what I'm talking about.

Now type in the same directory as before, but add /Anyrandomletters&symbols

It should look like this:

[host]/disalloweddirectory/Anyrandomletters&symbols



This will give you an error saying not found. Good.

Now for the Directory Transversal part:

[host]/disalloweddirectory/anyrandomletters&symbols/../



You should have noticed the /../ That is the Directory Transversal part.

What this does is send you back one directory, and if the website is insecure, then you can get access to the Disallowed area.

This doesn't work with just about every popular site, but works with privately owned sites which are nor properly coded.
Thnx :-) Enjoy hacking :-)