Wireshark, definitely. We use it extensively in our lab
for testing firmware changes and problem reports on the
International Space Station LAN. work great for catching
individual packets and analyzing them layer by layer,
protocol by protocol, tracing back and forth traffic of
protocol handshaking, and just plain overall bandwidth.
However, it will only record the traffic on the wire(s) it
is connected to. To see other parts of the system, you
either need to run multiple copies of Wireshark, or find
something else that puts an agent on other machines to watch
traffic in other parts of the net.
Our main switch is just a small embedded PPC, and does not
have the RAM or Flash to run Wireshark, and probably not the
speed also. But some switch/routers might be able to and then
you could see traffic on any of the lines connected to it.