On the BACK of the drums is a lever mechanism that the ebrake cable pulls on to set the brake drum tension.

That tends to get rusty and can get either progressively stiffer to operate and/or have "catches" where it sticks then grabs, etc...more inconsistently. So, go to the diff side of the brakes and look at the lever in back, and the plunger inside the drum that it pulls on (Open the drum).

If you manually pull the lever it should operate that plunger. If not, you need to work it free with rust dissolver, etc...such as PB Blaster (NOT WD-40, etc).

Don't get the de-rust solvents on the rubber parts though, as it dissolves them too, and then you have other issues, and so forth.

You can apply brake grease (I like the synthetic versions) to the moving parts to extend the time befor you have to do it again.