Score Tab

The Score Tab shows summary data for a variety of modeling methods. The purpose of this tab is to assist in determining the most appropriate model specification for the selected model through analysis of daily data.

Prior to version 7.1, the statistics were computed by estimating daily energy models using the standard Regression techniques and a set of daily models contained in a MetrixND Score project that was stored in the database.

Beginning with version 7.1, the Score project is no longer used. Instead, there is a check box named “Use in Scoring” that applies to each individual MetrixND template. This box may be checked for one or more templates, and this group of templates will be used for model scoring calculations. The standard Regression techniques will also be used, unless indicated otherwise in system settings. To exclude the standard Regression techniques set Models>ExclueRegressionModelsFromScoring to True.

The Score Tab displays results from the Score action, which is available on the Model Actions menu and through the corresponding toolbar button.

The Score Tab has four quadrants:

  • Concept Selector. Select the weather or XDriver concept to be used on the x-axis of the scatter plot.
  • Scatter Plot. Show the scatter plot of daily energy vs average value for the selected concept.
  • Model Scores. Relative score statistics for alternative modeling methods.
  • Data Statistics. Data statistics by year and month.

Concept Selector

The upper-left quadrant contains the Concept Selector. This selector controls the weather concept or XDriver used in on the x-axis of the Scatter Plot. The XDriver option is only available for XDrivers assigned to the model.

Scatter Plot

The upper-right quadrant contains the Scatter Plot. The scatter plot shows a plot of daily energy use on the y-axis against average daily weather (from Concept Selector) on the x-axis. Data that had been marked as bad does not appear in the Scatter plot. The displayed data points are controlled using the Mode action on the toolbar or in the View menu. The Mode action contains three options.

  • Daytype. This option color codes Saturday, Sundays, and weekdays.
  • Extended Daytype. This option color codes Saturdays, Sundays, Mondays, Fridays, and weekdays.
  • Year. This options color codes the years.

Data points may be toggled on and off using the legend check box in the lower right corner of the Scatter Plot.

Users can mark entire days of data as Bad by selecting the Mark Bad Mode button from the tool bar. If this is selected, the user can click on a point in the scatter plot to mark that day’s load data as bad. The point will disappear from the scatter plot when it is clicked.

Model Scores

The lower-left quadrant displays the accuracy scores for the scoring methods. Results are sorted from best (lowest) to worst (highest) based on Relative Score. The Scores are calculated when the Score command is invoked. This command will cause the following to occur (the details may vary according to the data tables present in the project templates that are in the scoring group).

  • Open the template to use for scoring from the database.

  • Insert daily load data for the selected model into the Daily Load table.
  • Update the Bad data table (if it exists) with bad spots marked in the database.
  • Insert weather data into the weather tables (e.g., DBT, WSP, and CLC).
  • Update the SunInput Parameters for the selected weather station.
  • Insert Calendar and DLS data.
  • Estimate the model.
  • Estimate all models for all standard Regression method.
  • Retrieve statistics from all estimated models and save to the Score table.

The statistic that is returned for a template or model is the average of the interval-level standard errors. All days that are not marked as bad are included in estimation and in the calculation of score statistics. The table in the lower left portion of the scoring tab shows the model scores, results are sorted from best (lowest) to worst (highest) score.

Data Statistics

The lower-right quadrant displays monthly and annual statistics computed from the hourly or daily energy values for the Model. The statistic's time scope is defined by the Data Statistics Calculation Length parameter in the System Settings, System Parameters Tab. This parameter is defined in days and specifies the number of days prior to the end of the meter’s interval data to include in the calculation. The following is a list of statistics provided in the grid.

  • Pct Use. This is the percentage of annual usage that occurs in each calendar month.
  • Energy, Peak Value, Min Value, Avg Value. The total energy, maximum load, minimum load, and average load.
  • Avg Daily Value. The average daily use (ADU).
  • Load Factor. The ratio of the average load to the peak load.
  • WkEnd Ratio. The ratio of average daily energy usage (ADU) on weekend days to the ADU on weekdays.
  • Day Fraction. The fraction of energy use that occurs during daytime hours (defined as 8 a.m. to 8 p.m.)
  • Days of Data. The number of days containing data over the interval.
  • Pct Complete. The percentage of intervals containing data values.
  • HLH Energy. The sum of the energy during high-load hours, defined as 7 a.m. to 10 p.m. Monday through Saturday.
  • LLH Energy. The sum of the energy during low-load hours, defined as Hours 1 a.m. to 6 a.m. and 11 p.m. – 12 p.m. Monday through Saturday and all hours on Sunday.

Actions

The following common actions are available on the Model Score Tab toolbar and in the View, Actions, and Tools menus.

The following action is available only on the Score Tab in the Model Module in the View Menu and on the toolbar.

  • Mode. The Mode controls the day-type data displayed as described in the Scatter Plot section above.
  • Mark Bad Mode. The Mark Bad Mode action applies to the Scatter Plot. When activated, the user can select data points in the Scatter Plot and changes the data status to Bad for the entire day of data.

The following actions are only available in the Model Module in the Actions menu and on the toolbar. These actions are further described in the Model Actions section.

  • Export to MetrixND.
  • Score.
  • Estimate.
  • Calibration Backcast.
  • Forecast.
  • Import Estimated Project File.
  • Export Estimated Project File.
  • Auto-Assign Template.
  • Export to XML.

The Models Module contains configuration tools relevant to configuring and forecasting models. These tools are discussed in the following sections.