I wish they'd made some kind of plug-in arrangement because I do use mine a lot, but when I don't need them they're always somehow in the way.
If you do remove the brackets and/or the little sockets that hold the visor in place (near the center of the windshield), you CAN put them back with sheet-metal screws. Use stainless steel - I used some aircraft hardware I had sitting around, just be real careful not to strip the hole out too much and consider using silicon to help seal up the hole (and lock the screw in place).
One of the problems I've seen is the pop-rivets will sometimes start to give and the bracket will get some slop in it. The only way to tighten it is to rerevit, shim it, or go with screws.