I am a computer scientist (with numerical methods and HPC background) who pretends to know weather modeling with WRF. I have run into the following issue so many times that I thought I'd post a query about it. Often, as is the case right now, I'm doing this on a huge domain (2000x2000x45) on a busy supercomputer, so I don't have a lot of trial and error time. Sometimes I can "play" with a smaller domain at 1km resolution in the rugged region, just to gain insight, and then successfully apply my findings to the larger domain.
I'm often trying to run cases at 1km resolution (using 30s topography) in rugged regions like Montana and Alaska (more Alaska these days). I'll often initialize with FNL or NARR and, very frequently my run stops at whatever value I set for radt (or cudt, if I've forgotten to set cu_physics to 0. Although this "might" suggest issues with whatever LW/SW schemes I'm using (typically RRTM/Dudhia), I've often been told that the problem is more likely that those schemes are getting bad input. One knowledgeable person has suggested the problem likely comes in as the input data at coarse resolution is being scaled down to my grid, causing all sorts of instabilities (he mentioned, in particular, vertical velocities), in the midst of wildly varying terrain.
I can often solve this problem by using a coarser topography (like 2m or, in the case of trying the Mt. McKinley area, 5m), but in some respects this kind of defeats the fun and purpose of doing 1km simulations. And/or, smaller timesteps will sometimes help, and in one case I was able to get this working (in the Alaska Range) by using the 5m topography AND cutting my timestep down to 1/20s (for a 1km grid). Once I got it to run for an hour so, I could then restart it with larger timesteps, but I was still stuck with the 5m topography.
I'm just wondering how folks have typically gotten around these issues. I don't "think" that an initial nesting down from coarse to fine resolution helps (e.g. nest down from 27km, to 9, to 3, to 1km). It seems like any initial instabilities are still present. But, I've wondered about doing a spinup simulation at coarser resolution, hoping to get the instabilities out of there, and then using that to initialize and drive my 1km resolution. It sounds like a royal pain, but maybe I just need to accept this is the way to go about it?!
Just wondering if folks have "tricks of the trade" they're willing to share
Thanks,
Don Morton
Arctic Region Supercomputing Center
http://www.arsc.edu/~morton/
