Strictly speaking, you can have DC currents that vary with time as well; just think full-wave rectifiers that effectively take the absolute value of the AC signal, but as the current doesn't change direction, it's DC. Or, the electricity generated from a generator; varies with time, but it is DC. As far as I know, only currents from a battery are effectively constant.
As for which formula to use, the instantaneous power P = V^2/R where V is the voltage at that time.
We want averages though, so we need to average V^2/R. R is a constant, so we need the average of V^2. For an AC circuit, V = Vpeak * sin(2pi*t/T) where T is the period. You could probably prove, as an exercise, that the average value of V^2 = Vpeak^2 * sin^2(2pi*t/T) over an entire period is half of the peak voltage squared by integrating. Square-rooting this "average" yields the RMS value of Vpeak/sqrt 2.