Frequently asked questions (FAQ)

All answers in this section refer to IEE v7.0 SP3 HF13 and later.

Where do I begin troubleshooting?

You can assess the state of ARI by looking at services, database settings and tables, the time of day, and several logs.

  • Consider the Time of Day: The expected ARI behavior can differ based on the time of day. When you begin troubleshooting, consider the following questions.

    • Does the problem happen within the data collection window, or outside of it?

    • If it is within the DCW, is it within the estimation and finalization phase?

    • Is the current time before or after the current day's DCW?

    • Did an ARI process complete today?

  • Verify File Processing State: Open ARI Dashboard and verify the state of file processing within the ARI process.

    If you do not have access to ARI Dashboard, use the following query:

    select currentday, filetype, filestate, count(1) from fileinventory

    group by currentday, filetype, filestate

    order by currentday, filetype, filestate

  • Analyze the Log Files: Open the logs for the Receiver, VE service, and VE runners. Parse the logs for ERROR and CRITICAL entries for the last 24 hours.

Does ARI validate, estimate, and import readings for imported channels and formula channels?

No. ARI only processes meter channels.

What does ARI do when the validation rule parameter Route to Queue (Do Not Auto-Estimate) is disabled?

When a read fails a validation rule and the Route to Queue (Do Not Auto-Estimate) parameter is disabled for that rule, the VE runner marks the read with failed Val_and_estimation needed. The VE runner writes the failed readings to a clean reading file for import. Estimation does not occur.

What happens when historical register reads arrive without interval data?

The register readings alone are imported. Any previously imported interval data will not be re-validated with the new register reading.

How many IDM services should be running?

Configuring AMI Reading Import (ARI) for best performance depends on your system configuration. Each environment is unique. Itron recommends that you begin with one multi-threaded IDM Service, using three threads. However, depending on the speed of your configuration, two triple-threaded IDM services might render the best ARI performance.

When using Standard Reading Import - Enhanced (SRI-E), the recommended one multi-threaded IDM Service, using three threads is sufficient.

When troubleshooting, should I set my logging level to DEBUG?

Do not set the logging level to debug. On debug, ARI logs so much information that the log files roll over within minutes, regardless of how large the defined maximum log size.

Can the ARI services run on regular application servers, or do they require dedicated hardware?

Itron strongly recommends that each validation and estimation service run on a dedicated computer that does not run any other IEE services or tasks. This was determined through lab testing with high volumes of configuration and readings data. The issue is strictly memory related. The receiver uses a lot of memory and because it is always on it does not release it. The computer can be small (a dual core processor and 4 GB of memory). It is possible for validation and estimation services to run on regular application servers, but not recommended.

Can ARI process register readings?

Yes. ARI imports all register readings associated with a service point for which there is a single interval channel. Readings for register-only service points are routed to Workbin0 for standard import processing. ARI does not currently support register validation.

Can SFTP or other file transfer processes be used to move clean reading files to the database server?

Yes. ARI supports a third-party move process, using whatever tool the IT department decides to employ. This solution is based on the Direct file transfer option and is controlled by the following parameters:

  • Database File Transfer Type. To use a third-party transfer mechanism, select Direct.

  • Database BCP Path or Database File Path. The UNC path to which the VE services write clean reading (and ancillary) files. This is the location the third party process would poll to find files requiring transfer.

  • Retry Import for Missing Files Time Limit in Minutes. The length of time within which the import job will retry the import of clean read files listed as “ready for import” in the FileInventory table, but which are not yet physically present in the importer’s source directory. The intent is to allow an amount time for the third-party transfer process to move the files to the AMI Readings Import directory, using whatever tool the utility deems appropriate. The AMI Reading Import target directory is a parameter configured when the job is created in the database.

How can I completely redo the current day VE processing?

In cases where current day VE processing does not complete successfully, it may be desirable to completely redo VE processing for the most recent current day after the underlying issue has been identified and resolved. For example, the VE process might not complete if the VE processes terminate abnormally or if they are manually terminated because they are taking too long to complete for some other reason. This involves the VE services reprocessing the original work bin files, regardless of what may have already been imported.

To redo current day VE processing:

  1. Stop the VE services.

  1. When all VE services and runners are completely stopped, run the following statement to reset all Work Bin files for CurrentDay processing, typically replacing the CurrentDay constant to yesterday and clearing the commenting line from the RDBMS:



    UpdateFileInventory

    SetFileState = 0,

    StartTime = null,

    EndTime = null

    where FileType = 1

    and FileState != 0

    and CurrentDay = TIMESTAMP'2010-09-09 00:00:00'-- Oracle

    --and CurrentDay = '9/09/2010'-- SQL Server

  2. Commit the update, if necessary.

  3. Start the VE services.

Why do the logs report file paths as inaccessible?

If inaccessibility is intermittent and seemingly random, check the Active Directory or domain servers for authentication request overload.

If you are getting a lot of FTP resubmit files (FileType 2, FileState 8), check whether the FTP account can be accessed from the computer running the VE runners. The problem might be an expired password.

If the logs report file paths inaccessible 100 percent of the time, verify that all file paths configured in the IEE rich client are formatted as UNC paths. For example: //servername/folder/subfolder/

The account that runs the IEEApplicationServer service must have access to these paths. Typically, this account is a dedicated domain account.

If the services run as Local Services, write permissions for all shared directories must be included in the computer accounts that access the shares.

To test whether the account has access to the file paths:

  1. Open Windows Services.

  2. Note the account that owns the IEEApplicationServer service.

  3. Use the name and password of this account to map the UNC paths as network drives.

  4. Create a file in one of the directories.

There is a log message claiming a Windows I/O error, with the explanation that there is not enough disk space. What do I do?

Verify the mount points for the following directories: receiver, resubmit, WorkBin0, other work bins, and the clean reading destination. These directories can run out of disk space, especially if the transfer mode is Direct rather than FTP or SFTP.

If there appears to be enough space, verify whether the NAS/SAN device has settable parameters for the maximum number of files in the directory. It is possible that this number has been reached or exceeded.

If this message appears in the VE runner logs, you will probably have to reprocess all the daily XML files and the work bin files.

Some work bin files created during the last DCW still have state 0.

It is possible that some work bin files are still in state 0 and they have no value in the StartTime column in the file inventory, even after the finalization period. In this case, the work bins are not assigned to a VE service. You must assign the work bins to a VE service and then reprocess the work bin files as CurrentDay files.

To assign work bins to a VE service:

  1. In the IEE rich client, go AMI > ARI Dashboard.

  2. Click Service Configuration.

    The Service Configuration dialog appears.

  3. Right-click the VE service with the fewest service points assigned to it and select Properties.

  4. Select Add Workbins. If the list is empty, then no work bins are in state 0 without a StartTime value.

  5. Add all work bins on the list to this VE service. IEE begins to process these work bin files as Historical files.

To reprocess work bins as CurrentDay files:

  1. When safely outside of the data collection window, shut down all VE services.

  2. Temporarily reassign work bins from one VE service to other VE services.

  3. Assign all the missed work bins to the VE service that now has no assigned work bins.

  4. Set the data collection window (DCW) Start Time to the time you want to start reprocessing.

  5. Set the DCW End Time to that time plus however long the DCW usually needs. The DCW and finalization time must occur within one local calendar day, and outside of the normal DCW and finalization period.

  6. In the CurrentDay parameter, enter the day for which you are importing data. If this setting is blank, IEE defaults to Yesterday.

  7. Start the VE service with the missed work bins. Wait until its runners stop processing and finalizing data. This usually takes until end of the data collection window plus the finalization time.

  8. Shut down the VE service.

  9. Reassign all work bins so that there are no free-floating work bins.

  10. Change the DCW and CurrentDay parameters back to their original values.

  11. Start all VE services.

It is now some time after the finalization phase, but some work bin files for CurrentDay are still in state 1 and no historical runners appear to be running.

Most likely the finalization phase was too short.

Open the VE runner log files and verify whether the last messages end abruptly at or around the time when finalization time expires (the sum of the values in the System Setting parameters Data Collection End Time and Maximum Finalization in Minutes). A healthy VE runner log includes a message at the end that the VE runner’s processing window is closing. If this message is missing, the VE runner was stopped.

To correct the issue, reprocess all work bins for the VE runners that were cut off. See .

Then go to System Administration > System Settings > AMI Readings Import and set the Maximum Finalization in Minutes parameter to a higher number.

Location of maximum finalization in minutes.

ARI is functioning slower than expected.

One or more of the following issues can cause the ARI process to slow down so much that it becomes unusable.

  1. A file size setting is set too low. A file size set too low results in a huge number of small files. This can happen to work bin files and clean reading files. Go to System Administration > System Settings > AMI Readings Import and verify the values for the following parameters:

    • Database File Size. This defines the size of the clean reading files, in bytes (not kB or MB).

    • Maximum Files in Workbin Folder. This defines the number of files allowed in a work bin folder before IEE creates a new work bin. In Windows, file system access performance starts degrading when there are more than 20,000 files in a single folder. Older Windows computers may begin to slow down with only 10,000 files in a single folder.

  2. Database statistics have not been gathered properly, regularly, or at all. Efficient operation of a database requires regular analysis of table information. If this is not done daily, performance will degrade. Consult a database administrator if you suspect this is the case.

  3. IDM takes a long time to process an XML file. If large numbers of channels are getting separated out into WorkBin0 and banked files, the processing time for XML files can increase up to tenfold. For example, if every meter sends data for an additional channel that does not exist in IEE customer configuration, IEE separates these channels into WorkBin0 and banked files.

  4. The logging is set to DEBUG. For optimized performance, the logging level should be INFO. Setting the logging level to DEBUG causes large quantities of information to be written into the log files. Verify the logging level in the Process Logging Configuration code table.