TBH I’ve never masked rock directly. I’m in the habit of layering materials in the order they make sense in the real world. So rock generally serves as a base. Once all the layers are done, rock mask can be had by subtracting all layer masks from pure white.
What’s your exact problem with your current method?
P.S - Take a look at this thread for workflow ideas. I was curious about this topic as well some time back.