The problem with USB DACs is that it's a real pita to switch from headphones to speakers. My sound card allows the headphone amp to function from the front audio ports of my case, I have a quality case so there is no sound degradation from using the front panel. So I have both my headphones and speakers connected at all times and can switch between them through software.
There are some sound cards I believe who come with dedicated headphone port at the back which is really the best solution from a wire management point of view.