The following modules are useful for a penetration tester testing a web server for vulnerabilities:

• dir_listing: This module will connect to the target web server and determine whether directory browsing is enabled on it.
• dir_scanner: Using this module, you can scan the target for any interesting web directories. You can provide the module with a custom created dictionary or use the default one.
• enum_wayback: This is an interesting module that queries the Internet Archive website and looks out for web pages in the target domain. Old web pages that might have been unlinked may still be accessible and can be found using the Internet Archive website. You can also identify the changes that the website has undergone throughout the years.
• files_dir: This module can be used to scan the server for data leakage vulnerabilities by locating backups of configuration files and source code files.
• http_login: If the web page has a login page that works over HTTP, you can try to brute force it using the Metasploit dictionary.
• robots_txt: Robot files can contain some unexplored URLs, and you can query them using this module to find the URLs that are not indexed by a search engine.
• webdav_scanner: This module can be used to find out if WebDAV is enabled on the server, which basically turns the web server into a file server.