You might be having a problem with the starter itself. The drive might be getting hung up in the flywheel or the plunger on the solenoid might be sticking. Both of those things would allow the starter to keep running even if you release the key, as the main contacts for the starter motor would remain closed.
I would proceed by removing the small wire (the one that activates the solenoid) from the starter solenoid, hooking up a voltmeter or test light from that to ground, and then work the starter/ignition switch. If the voltage rises and drops in conjuction with the start position of the ignition switch, then the problem is in the starter and/or it's engagement with the flywheel, not the ignition switch.
Edit: I just more carefully re-read your post, and you said that if you physically turned the key back, that the starter would shut off. My above diagnosis might not be correct then. However, I would try it anyway. Also, you won't get as frustrated having an engine cranking killing the battery while you're trying to figure out what's wrong. Does the starter engage when you just turn to the ignition position? Or does it only start to crank when you turn to the start position and then refuse to turn off when the key springs back?