I always replace the split olive after two uses as a ridge is formed at the edge when you clamp them down, and that, I think, is where the actual sealing, if any, takes place. I do believe the split gets closed up, but I don't understand the physics of it.
Here's a conundrum: There should be a gasket at the forward plate along with the shim pack used to remove end play so, presumably, you have to allow for the compressed thickness of the gasket when calculating the necessary shim pack (IIRC, last time I did this the gasket measured 0.005" thick). But, the gasket only seals the border of the end plate, as oil can (has to) get to the end of the worm to lube the forward bearing, so oil can still seep past the shims. I usually put some sealant on the shims, and most of it gets squeezed out but I think it helps. Also, IIRC the 4 bolts on the end plate go into open holes, and oil can get past a split washer. What I've been doing for situations like this--including the bolts that secure the bellhousing to the gearbox, whose holes are also open--is to put a fresh copper washer under a grade 8 washer, and use blue threadlocker on the bolt, instead of a split washer (and torque appropriately).
Steve, did you install a new peg on the rocker shaft?