Model Actions

The following common actions are available in the Actions Menu:

The following actions are only available in the Models Module in the Actions menu and on the toolbar.

Export to MetrixND. For custom models, populates the model template with load and energy data from the system and saves to a .ndm file.
Score. Executes the scoring methods and stores scoring results to the database.
Estimate. Estimates the selected model(s).
Calibration Backcast. Generates a backcast for the selected model(s) based on the backcast settings. These backcasts are based on actual weather data for historical days and are required to support error correction calculations based on backcast performance.
Forecast. Generates a forecast for the selected model(s).
Import Estimated Project File. Imports a MetrixND project file with estimated coefficients. This allows generation of forecasts using parameters estimated in another instance of the system (for example, estimate in Test environment and import into Production environment).
Export Estimated Project File. Exports a MetrixND project file with estimated coefficients. The resulting file can be imported into another instance of the system using the action to Import Estimated Project File.
Auto-Assign Template. Based on results for the scoring method, this selects a regression method or custom template based on model performance statistics. The settings for the best method are created and assigned to the model.

These actions are defined below.

Export to MetrixND

The Export to MetrixND command allows the user to export a Custom model from the system and save it as a MetrixND (.ndm) file. The project file is complete with historical and/or forecast data from the system database, transformation tables, models, group forecast objects, reports, and graphs. Settings for the export are:

Include Estimation Data. Exports the model with the data used for Estimation.
Include Forecast Data. Exports the estimated model updated with the data used to forecast.
Estimate Model. Estimates the model before the export action.

To execute this action, select a model that has MetrixND Custom Method checked. Select the Export to MetrixND option on the Actions menu or press the corresponding Toolbar button. These actions are available on all tabs of the Models module, but they will only be enabled if a single model is selected and that model uses the Custom MetrixND Model option. These actions open the Export to MetrixND dialog.

Set the Export Filename. The path and filename for the export file can be entered directly or by using the Browse button. The Browse button brings up a standard Save As dialog, which allows the user to browse to the output path and to enter the file name as desired. On the Save As dialog, press the Save button to install the selected path/filename or Cancel to abandon the browse action.
Select the Export Settings. Options are Model Estimation data, Model Forecast data, and Estimate Model. Check all three options to estimate the model and to include both estimation data and forecast data. To save time, only check the Estimate Model option if you want the system to estimate the model before the export.
Click the Export button to export the model to a MetrixND project file.
Click the Cancel button to abandon the export.

Score

The Score action computes accuracy statistics for the set of model templates that are set to use in scoring and (depending on a system setting) for the standard Regression methods The Score for a method is computed from the average of the estimated standard error values for the interval-level models. The results are stored in the system and are available for review on the Score Tab.

To compute scoring results for the scoring methods, select the Score option from the Actions menu. This will open the Scoring Models progress dialog, which will show the list of models to be scored and progress through the list.

Select a model or set of models to be scored.
Click the Score button or select the Score option from the Actions menu. These options are available on all tabs of the Models module (List, Date, Score, Estimation, Calibration, Forecast).
A Scoring Models progress dialog box appears showing the progress of the scoring.
Press the close button when scoring is complete. If the Auto Close box is checked, the Scoring Models dialog will auto close when scoring is done.
Go to the Score Tab to see the scoring results.

Starting with version 7.1, the Score action is silent in that it does not generate event log entries. If there is an error while attempting to calculate one of the scoring methods, there will be no results for that method. If the Score action is executed manually, the status in the Scoring Models dialog will show a status of Error. There will be no other indication that the scoring method failed other than the absence of a Score result for that method.

Estimate

Regression and Custom Models must be estimated before a forecast can be generated. On the List Tab, the first column (Needs Estimation) indicates whether a model needs to be estimated (a yellow exclamation symbol) or not (a green check). If any of the model settings are modified (like the estimation period, Regression model settings, or Custom model template or XDrivers), the model will need to be estimated. Results are presented on the Estimation Tab.

Select a model or set of models to estimate.
Click the Estimate button from the Toolbar or select the Estimate action from the Actions menu. These actions are available on all tabs of the Models module (List, Date, Score, Estimation, Calibration, Forecast).
A progress dialog (Estimating…) will appear showing the progress of the estimation process.
Press the Close button when estimation is complete. If the Auto Close check box at the bottom of Estimating dialog is checked, the dialog will auto close when estimation is done.
Go to the Estimation Tab to see the estimation results.

Calibration Backcast

Calibration factors are used to support multi-day error correction adjustments to the forecast based on recent model performance. If a model is configured to use calibration, the calibration factors are calculated by executing a model backcast. Adjustment factors are then computed from the backcast results and will be applied the next time a forecast is executed. Results are presented on the Calibration Tab.

Select a model or set of models for which to execute calibration backcasts.
Click the Calibration Backcast button on the Toolbar or select the Calibration Backcast option from the Actions menu. These actions are available on all tabs of the Models module (List, Date, Score, Estimation, Calibration, Forecast).
A Backcast progress dialog box appears showing the progress of the backcast calculations.
Press the Close button when the backcast process is complete. If the Auto Close check box at the bottom of Estimating dialog is checked, the dialog will auto close when the process is done.
Go to the Calibration Tab to see the backcast results and the calculated calibration factors.

Forecast

The Forecast command generates a forecast for the selected Models. When executed through the UI, the Forecast command will always re-compute the forecast.

When forecasting is run as a task, the Data Last Updated and Data Last Forecasted Fields are used to determine if the forecast for an item needs to be updated. If the Data Last Forecasted date is less than the Date Last Updated date, then the forecast action will be executed for the item. Results are presented on the Forecast Tab.

Select a model or set of models for which to execute forecasts.
Click the Forecast button on the or select the Forecast option from the Actions menu. These actions are available on all tabs of the Models module (List, Date, Score, Estimation, Calibration, Forecast).
A Forecasting Models progress dialog box appears showing the progress of the forecast calculations.
Press the Close button when the forecast process is complete. If the Auto Close check box at the bottom of Estimating dialog is checked, the dialog will auto close when the process is done.
Go to the Forecast Tab to see the forecast results

Import Estimated Project File

Custom models are normally estimated by assigning a MetrixND Template to a model. The Template provides all the framework for variable transformations and the model specification, but it does not have estimated coefficients. The Estimation step opens the template, populates it with data, estimates the model coefficients, and saves the estimated model to the database. The Forecast step then generates a forecast using the estimated model.

In some cases, it is useful to bring in a MetrixND project file that has been estimated externally or in another environment. This is done using the Import Estimated Project File action. When this action is executed, the external MetrixND project file is imported and saved to the database. The Forecast step will then generate a forecast using the coefficients that came with the externally estimated model. Note that if there is a Template assigned, the Estimate action will overwrite the externally estimated model with an internal model estimated using the Template-based process.

To import an externally estimated project file:

Select the model(s) to which the estimated project will apply. These models should all use the Custom modeling method. As part of the import process, a warning will be generated for any models included in the import action that use other methods.
Select the Import Estimated Project option from the Actions menu. This action is available on all tabs of the Models module (List, Date, Score, Estimation, Calibration, Forecast).
This brings up the Browse for Folder dialog. Select the folder that contains the estimated project file(s) that you wish to import. Note that if multiple models are selected, the project files for these models must all be in the same folder.
Press the OK button to proceed with the import. This will open a progress dialog (Importing…) showing the progress of the import process. For each model, the system will look in the selected Folder for a file that is named ModelID.ndm, where ModelID is the ID for that model.
Press the Close button when the import process is complete. If the Auto Close check box at the bottom of Importing dialog is checked, the dialog will auto close when the process is done.

Export Estimated Project File

This action is provided to support exporting an estimated project file from one environment and importing that project file in another environment using Import Estimated Project File. The result of this export action is different from the Export to ND action. Export to MetrixND allows the user to export historical and/or forecast load, weather, and calendar data as part of the exported file. Export Estimated Project File creates a MetrixND file that has no historical or forecast data. It does contain all of the data table and transformation table configuration information, and it does contain all of the estimated model coefficients in locked models.

To export an internal project file from the system:

Select the model(s) to which the export action will apply. These models should all use the Custom modeling method. As part of the export process, a warning will be generated for any models included in the export action that use other methods.
Select the Export Estimated Project option from the Actions menu. This action is available on all tabs of the Models module (List, Date, Score, Estimation, Calibration, Forecast).
This brings up the Browse for Folder dialog. Select the folder to which you want the estimated project file(s) to be exported.
Press the OK button to proceed with the export. This will open a progress dialog (Exporting…) showing the progress of the export process. For each model, the system will create a file that is named ModelID.ndm, where ModelID is the ID for that model.
Press the Close button when the export process is complete. If the Auto Close check box at the bottom of Importing dialog is checked, the dialog will auto close when the process is done.

This type of export can also be executed as tasks. There are two versions. The first is named ExportAllModelProjectFiles. The second is called ExportModelProjectFiles and includes a filter that allows exporting a selected subset of projects based on model properties and custom fields.

Auto-Assign Template

This functionality picks a model approach and a specification based on the results of the Score action. In the Score process, models are estimated using all the available methods, including the standard Regression models and model templates that are set to be used in scoring. The system will select the method with the best (lowest) score and assign that method to be used for the model. The score is computed as the average of the standard errors for the interval-level models.

To execute this action:

Select the model(s) to which the auto-assign action will apply.
Select the Auto-Assign Template option from the Actions menu. This action is available on all tabs of the Models module (List, Date, Score, Estimation, Calibration, Forecast).