If you induce vacuum on an advance, you can check it with the engine off and at idle. First with the engine off, disconnect the vacuum advance line from the carb and block it to prevent leaks. Next hook up your gauge and pull a few inches of vacuum and just let it sit for a few minutes. If it doesn't drop or go back to zero, your diaphragm is not leaking.
Release the vacuum from the advance.
Next, start the car and and let it warm up, or better yet, do this after the car is already at operating temperature. Pull a few inches of vacuum. The engine speed will increase slightly, or with a retard unit, it will decrease slightly.
This will also test that the advance unit is working with the distributor. It does not confirm that everything is set perfectly, only that it is working.
Good luck.