I believe you could just as well wire them in parallel and omit the switch. The PC output is usually prepared to directly drive headphones, so two line level inputs in parallel can't anywhere near overload it. The end result may be quieter than now, but you can probably compensate by just cranking up the PC volume.
BTW always a good idea to unplug all equipment from mains before making this kind of connections. If the PC is not connected to an earthed mains socket, the chassis (= ground) will float somewhere between 0 and 230VAC, and it's easy to burn something with it - especially with RCA plugs and such that make the connection on signal before ground! The TV antenna socket can also bring in unwanted ground potentials from the neighbours' equipment...