User-agent: * Disallow: /includes Disallow: /stylesheets Disallow: /uploads # it may be better to specify this on a per-page basis in the future # The basic idea is that if you include a tag like: in your HTML document, that document won't be indexed. # If you do: the links in that document will not be parsed by the robot.