Wireshark, definitely. We use it extensively in our lab for testing firmware changes and problem reports on the International Space Station LAN. work great for catching individual packets and analyzing them layer by layer, protocol by protocol, tracing back and forth traffic of protocol handshaking, and just plain overall bandwidth. However, it will only record the traffic on the wire(s) it is connected to. To see other parts of the system, you either need to run multiple copies of Wireshark, or find something else that puts an agent on other machines to watch traffic in other parts of the net.
Our main switch is just a small embedded PPC, and does not have the RAM or Flash to run Wireshark, and probably not the speed also. But some switch/routers might be able to and then you could see traffic on any of the lines connected to it.