An experiment in weather forecast monitoring

In one of my older posts, I gave a diatribe on weather forecasting for ski areas. You can see it here.

Based on what I’m looking at, I see an opportunity this week to run a live experiment on the veracity of the forecasts. I will do my best to live capture the predictions for this week as they change and are updated through the week, and post them as I see them, keeping this post up to date as I go. Barring catastrophes, I will ski this week on Monday, Tuesday (maybe), Thursday (maybe), Friday and Saturday, so I will be able to SHOW you how these forecasts equate to real life.

So my goal in this experiments to capture the forecasts as they come in, and put them up on this post. My objectives are several:

  1. To show you how much and how frequently forecasts change;
  2. To show why no forecast is good;
  3. To help you develop your own forecasting sense of who to believe.

So off we go!

===================

Saturday, January 25, 11 PM:

Yesterday, when I looked at the forecast, they were predicting snow on Sunday, and sun for the rest of the week. I wish I had captured those. But we’re going to start today, and my focus isn’t the snow forecast for Sunday, and the storm event that is supposed to appear. It’s the forecast for later this week.

Environment Canada, Banff Forecast, as 0f 2300 hrs, 11/25

EC Banff 0126 0100

EC Banff 0126 0100Notice:

  • No snow in the forecast;
  • Sun all week starting Monday.

Snow-Forecast.com:

Snow-Forecast.com, 0100, Jan 26

Snow-Forecast.com, 0100, Jan 26

Notice:

  • Snow staring Tuesday night;
  • Overcast Wednesday;
  • 19 cm accumulation expected, starting Tuesday PM and finished by Thursday AM

I’m not tech savvy enough to capture the entire Washington Snow Forecast model loop for just the window we’re looking at to post here, but suffice to say here’s their a snapshot of their image for the Tuesday Jan 28 at 9 PM.

Washington State University Snow Prediction, as of 01/26/14

Washington State University Snow Prediction, as of 01/26/14

Washington State says “it’s gonna snow”. Which means clouds. Which disagrees with EnviroCan, but agrees with Snow-Forecast.

So, as of 0100 Jan 26th, according to 2 of the 3 forecasting sites I watch, it’s supposed to snow later this week. EnviroCan is the dissenter, saying its supposed to be sunny with no snow even in Banff.

Let’s watch, do the pole tests, and find out how well they do. Come back and visit this post. I promise to update it with real observations, updated forecasts as I capture them, through until next Saturday. Let’s see how well folks can forecast the weather.

=====================

Update #1: Mid Morning, Jan 26

Wow. What a difference 3 HOURS makes to a Snow-Forecast.com forecast. Now, instead of 19 cm, they’re calling for 11 cm, and falling in a smaller window. Temperatures have dropped from the -7° range on Wednesday to -11° to -9°. Winds which were to be westerly at 10 overnight Tuesday and into Wednesday are now forecast to be light to non-existant.

Snow-Forecast, 0400 on Jan 27.

Snow-Forecast, 0400 on Jan 26

And EnviroCan has climbed on board, and are now calling for snow on Wednesday. Their text forecast doesn’t say anything but “Snow” for Wednesday, and doesn’t mention Tuesday night at all.

As of 10 AM, Jan 26

As of 10 AM, Jan 26

==================

Update #2: Early Jan 27

EnviroCan changed their mind again. As of 0500 on Jan 27, now it’s only supposed to be cloudy on Wednesday, with the snow falling Thursday.

EnviroCan, 0500, Jan 27

EnviroCan, 0500, Jan 27

As for Snow-Forecast, they’ve backed up to something closer to the Saturday forecast. Back up to 17 cm, snow starting Tuesday night, ending Thursday morning again. Thursday morning’s temperature forecast has dropped from -7° to -17°. But then the wind direction switched 180° between the last forecast and this one.

SnowForecast, 0400, Jan 27

SnowForecast, 0400, Jan 27

The Washington State loop is showing the snow starting 10 PM Tuesday and finishing early Thursday morning. This hasn’t changed much. Hard to tell if the intensity of the snow event has changed.

=================

Update #3: Jan 27, 8 PM

And the hits just keep on coming. SnowForecast has contracted their forecast once again. According to them, the forecast accumulation for this storm event is now down to 10 cm. Snow isn’t supposed to start until Wednesday now, and peter out Thursday morning instead of Thursday afternoon.

Snow-Forecast, Jan 27, 1600 hrs

Snow-Forecast, Jan 27, 1600 hrs

EnviroCan? The essence of consistency — if by “consistency” you mean “consistently changing the forecast”. While it is still only supposed to be cloudy Wednesday, on Thursday, instead of snow, now we have “A mix of sun and cloud with 60 percent chance of flurries”. In the text forecast (not captured in this screen shot) they say “Periods of light snow beginning near midnight” on Tuesday. Snow on Tuesday hasn’t been mentioned before.

EnviroCan, Jan 27, 2000 hrs

EnviroCan, Jan 27, 2000 hrs

=============

Update # 4, Jan 28, 11 AM

I pulled these this morning. EnviroCan is getting more specific, which is good. What they’re not getting is more consistent.

EnviroCan, Jan 28, 11 AM

EnviroCan, Jan 28, 11 AM

Wednesday is back to snow after being cloud yesterday, snow the day before that, and sun the day before that. Thursday started at sun, then went to snow, then went to cloud, and is now back to sunny. The snow event is now down to a 40% chance of flurries on Wednesday (as opposed to a 60% chance on Thursday), and once again we now have a 40% possibility of snow Tuesday night.

Snow-Forecast is a bit better. There’s been virtually no change in snow timing or amounts (11 cm yesterday, down to 10 cm today, same pattern). Temperature predictions are holding fairly constant, winds reasonably so.

SnowForecast, Jan 28, 0400

SnowForecast, Jan 28, 0400

Part of the reason this storm event appears to be causing problems is that the front creating it it has changed. Compare the Washington State snow model above, captured last Saturday, with this one for essentially the same late Tuesday window.

Washington State model, as of Jan 28 0800, for 0100 Jan 29

Washington State model, captured 0800 Jan 28, for 0100 Jan 29

Notice a total lack of colour. But fact forward that model a few hours, and you see this:

Washington State model, captured 0800 Jan 28, for 1600 Jan 29

Washington State model, captured 0800 Jan 28, for 1600 Jan 29

See that big curved thing? It’s a front coming in from the south that’s come trucking in from the Pacific. That’s different than was forecast by the model on Saturday.

So the Washington State model is saying the weather pattern originally predicted for this week is different than originally forecast. It will still being snow, but it’s not the big event. that was supposed to show up.

Next up: as the snow event arrives, I’m going to try to capture the data from Sunshine’s automatic weather stations.

================

Update #5: Jan 29, 0900

SnowForecast has contracted again. Now snow is only supposed to fall starting late Wednesday and into Thursday morning, with snowfall totals of 6 cm. Everything else is staying fairly still, with minor swings in forecasted wind direction.

SnowForecast, 0129, 0400

SnowForecast, 0129, 0400

EnviroCan is being the essence of consistency, if by “consistency”, we mean “consistently changing”. For the 5th straight day, the forecast has changed. Thursday, forecast yesterday to be sunny, is now forecast to be snow in the morning then clearing in the afternoon. High temp for Wednesday, forecast yesterday at -7°, is now -3°, though they do note a falling temp through the day.

EnviroCan, Jan 29, 0900

EnviroCan, Jan 29, 0900

The Washington State loop again is telling the story as to why. The big frontal system forecast last Saturday is taking a more southerly track, and will mostly miss the Banff area. It has resulted now in snowfall warning for Pincher Creek. That front is being pushed south by an airmass containing a blob of snow coming in from the east — an upslope event — that will put more snow in Calgary than in Banff. Here’s two snapshots from today’s loop, at 10 AM and 4 PM.

Washing loop capture, Jan 29 0800, for 10 AM Jan 29

Washing loop capture, Jan 29 0800, for 10 AM Jan 29

Washing loop capture, Jan 29 0800, for 4 PM Jan 29

Washing loop capture, Jan 29 0800, for 4 PM Jan 29

So the majority of the big stuff is heading south; Fernie, Castle, Red & Whitewater will get the brunt of it, Kimberly a bit of it. It should still snow in the Banff area, but in much lower quantities than if the storm had hit us.

BTW, despite Snow-Forecast and EnviroCan and Washington State all forecasting snow overnight last night, essentially it barely snowed overnight. Here’s Sunshine’s automated station. HN24, new snow in 24 hrs, registered 0.5 cm when they cleared it at 6 AM.

Sunshine Automated site, Jan 29, 0900

Sunshine Automated site, Jan 29, 0900

============

Update #6: Jan 29, 5 PM

Tiny flakes started falling out of the sky in Canmore about 15 minutes ago. Take a peek at the radar image associated with this, and compare it to the Washington State loop from this morning:

EnviroCan Radar image, Jan 29, 1700

EnviroCan Radar image, Jan 29, 1700

Washing loop capture, Jan 29 0800, for 4 PM Jan 29

Washing loop capture, Jan 29 0800, for 4 PM Jan 29

Where it’s snowing was pretty accurately predicted by Washington State 16 hrs ago, and the bulge of heavier snow south and west of Calgary’s pretty accurate. But if you loop the radar, the snow is coming in more from the north than the Washington model predicted only 16 hrs ago — though the current model run is saying it will come in from the north.

As of this moment, the Washington model’s predicting an end to this snow event by 4 AM Thursday, 12 hrs from now. Also, as of this moment, Sunshine has had 0.5 cm of snow today, on par with SnowForecasts’ prediction of 12 hrs ago, but substantially less than any previous forecast.

===================

Update #7, Jan 29, 11:00 PM

It hasn’t snowed much in the west side of the Bow Valley tonight (I have only ~3 cm at my house) but I was over in Dead Man’s Flats tonight — 12 km east of my house — where it snowed all evening, was still snowing when I left at 10:30 PM, and ~8 cm had accumulated thus far. This is an upslope system hitting us, and with those storms it is common to have snow at Lac Des Arcs and Dead Man’s but less the farther west you go. However, what is much more interesting is this:

Sunshine has had 6 cm since 5 PM…

Sunshine's Stonehenge automated data, Jan 29, 11 PM

Sunshine’s Stonehenge automated data, Jan 29, 11 PM

…but according to the EnviroCan radar, it’s not snowing there.

EnviroCan Radar, Jan 29, 8:40 PM

EnviroCan Radar, Jan 29, 8:40 PM

According to the radar, not only is it not snowing at Sunshine, it’s not snowing at Dead Man’s Flats. Except I was there all night, and it snowed all night, and I had to brush it all off my car. Go back to my analysis of weather radar: EnviroCan’s radar sees what it’s supposed to see, and snow at Sunshine (or Dead Mans, or Lake Louise) isn’t what it’s looking for. Tonight, it has not seen it.

=================

Update #8: Thursday, Jan 30, 0800

Well, it’s all over but the shouting. The snow event is over, and Sunshine reported 6 cm out of it. Note that the temperature fell from -10° at the start of the snow to -16° or colder by the end of it.

Sunshine automate data, Jan 30, 0800

Sunshine automate data, Jan 30, 0800

My pole test today, taken at the top of the Gladerunner pitch, showed ~10 cm or slightly less.

That's ~10 cm. Top of Gladerunner

That’s ~10 cm. Top of Gladerunner

That snow came down with essentially no wind.

Sunshine's automatic data, top of Divide, Jan 30, 0800

Sunshine’s automatic data, top of Divide, Jan 30, 0800

EnviroCan kept up their perfect track record of changing their forecast (compare both Thursday & Friday to yesterday’s forecast), and they even got today wrong. It was sunny all day in Banff (Sunshine, too), and unless “early this morning” meant 4 hrs prior to the forecast, there were no flurries at all.

EnviroCan Banff, Jan 30, 0800

EnviroCan Banff, Jan 30, 0800

The Washington State Loop was not much better; at the 0400 run, it was supposed to stop snowing at Sunshine between 0800 and 1000, but you can see it basically quit snowing and went clear at about 4 AM. But yesterday, I said the forecasts were implying that the majority of the snow would go south and hit those resorts; even Washington State was predicting that Fenie would get “missed”:

Washington's radar loop: look at the hole at Fernie.

Washington’s radar loop: look at the hole at Fernie.

And that’s what happened. Fernie got 4 cm, while Kimberly got 15, Whitewater got 10, Red got 9.

Snow-Reports.ca's morning numbers

Snow-Reports.ca’s morning numbers

===================

In Summary:

Let’s now take a look back in the cold, hard light of day about how well this storm was forecast.

What really happened was that there was a single pulse of snow that showed up starting ~1800 on Jan 29 and stopped at ~0300 on Jan 30 — about 8 hrs worth, and just a bit over 6 cm fell. Temperatures started dropping in the afternoon of Jan 29, and fell all night long as a cold front arrived.

Let’s start with Snow-Forecast.com. Here’s how their storm total snow forecasts changed over time, and how those forecasts compared to what really happened:

Um... not so good.

Um… not so good.

The forecast they got right was the forecast issued on the day of the storm. Every other forecast was optimistic by at LEAST 66%.

Surely they predicted WHEN that snow would fall…

Um... no.

Um… mostly no.

They got one thing, and only one thing correct: the peak snowfall would occur overnight on January 29, and they started predicting that at least 5 days in advance.

How about the wind speed? Well, this storm didn’t “blow in” so the wind speed data isn’t as interesting as the wind direction.

Can we tell anything from this?

Can we tell anything from this?

About the only thing interesting in the wind speed data is the calm (0 km/hr) forecasts. This was never forecast as a windy event, and it wasn’t windy. So two thumbs up there. But…

The wind direction shift is of note

The wind direction shift is of note

The wind here generally blows from 225° to 270° (southwest through west). The forecasts correctly predicted a wind shift from the west-ish to the northeast-ish (45° or so). That shift did happen, but various forecasts had it occurring sometime between 1200 on the 29th and 0800 on the 30th. It actually occurred at 2000 on the 29th (almost dead in the middle).

Note the wind direction switch.

Note the wind direction switch.

Well, Snow-Forecast MUST have got the temperatures right…

Um... not so much.

Um… not so much.

Yes, you’re seeing 4° to 8° gaps between actual temps and forecasts, and similar swings between forecasts mere hours apart. They missed the temperature drop (and frontal arrival) by 12 hrs even the day it was happening.

Perhaps the forecasts of Environment Canada are “less variable”.

EnviroCan really isn't any better

EnviroCan really isn’t any better

Nope. Turns out EnviroCan’s temperature forecasts for Banff swing 9° between forecasts. And what’s REALLY shocking is how much their graphical forecasts change through a week. Look at Thursday.

Sun then cloud then snow then sun then...

Sun then snow then partially cloudy then sun then…

FYI: As I mentioned, after some morning cloud, which cleared by 9 AM, it was sunny and did not snow at all in Banff on Thursday.

My message in collecting all this data is simple:

QUIT RELYING ON WEATHER FORECASTS!

I’d love to say some are better than others, but none are that good. Take all of them with a grain of salt. Sure, read them all, BUT look for who’s agreeing with whom, who’s disagreeing and (if you can figure it out) why they’re disagreeing. If there’s a weather system on the horizon, watch the satellite imagery to see if it’s tracking where the various forecasters thought it would. See if you can train yourself to look at the maps on Snow-Forecast.com, SnowReports.ca (a new Calgary site who’s forecasts don’t make my heart go pitty-pat), the Washington site, EnviroCan, the Canadian Avalanche Association’s weather page, heck even David Spence’s, and see what they’re telling you.

Then go skiing anyway. Quit chasing powder (or at least, quit chasing forecasts of powder). About the only thing you should really read weather forecasts for is to decide how to dress for a given day. Or whether (like today, when it was -33° up in the village this morning) it’s just too darned cold to go skiing.