User-agent: * Disallow: /documents/
Crawl-Delay: 20