PBQ 7: Web Application Reconnaissance - Robots.txt Analysis

A penetration tester is performing reconnaissance for a web application assessment. Upon investigation, the tester reviews the robots.txt file at http://example.com/robots.txt.

Robots.txt Contents:
1  User-agent: *
2  Disallow: /search
3  Allow: /search/about
4  User-agent: acunetix
5  crawl-delay: 10
6  Allow: /search/static
7  User-agent: Baidu
8  crawl-delay: 12
9  Disallow: /Home
10 User-agent: Slurp
11 crawl-delay: 20
12 Allow: /sdch
13 User-agent: Comptia
14 Allow: /admin
15 Allow: /wp-admin
16 crawl-delay: 15
17 Allow: /groups
18 Allow: /?hl=
19 Allow: /wp-login.php

Your Task:
1. Identify paths that should be removed from robots.txt (revealing sensitive directories)
2. Select appropriate tool for WordPress security testing