First, has the trim aluminum actually marred the surface of the door panel? In other words, is the panel vinyl surface warn and deformed? If this is the case, aluminum particles may be imbedded within the vinyl’s surface and, after their removal, you will still have the mar.
I would suggest, and assume, you already tried a standard vinyl cleaner. If that didn’t work and if it were my problem, I would use an aluminum polish, such as Mothers Aluminum Polish, and apply with light rubbing to loosen the deposits on/in the door panel. I would then polish the aluminum trim to reduce the potential for future deposits by first applying the Mothers followed by a light application of car wax.
Since vinyl is usually color-through, you would have to rub pretty hard to go through the color. In any case, I would suggest you take your time and use a light touch.
Hope this helps,
Ray (64BJ8P1)