When you ssh into another box from Mac OS X, X11 automatically starts up because X11 forwarding is enabled by default. I have no idea why it’s a default as I suspect the vast majority of Mac OS X users never use X11 forwarding. It’s annoying to need to quit it every time. The fix however is easy. Edit your ssh config located here:
By default you’ll see a line like this in a series of otherwise commented out configs:
Change that “yes” to a “no” and save the file. No more quitting X11 every time you ssh into another box.
In the event you need to use X11 forwarding from time to time, you can enable it when connecting by doing:
ssh hostname.tld -o ForwardX11=yes
It’s nice to get rid of those little annoyances every once in a while.