Evaluation and Testing

PC-Rail Simulation Evaluation and Testing

25/04/23

Introduction

This Guide is provided to assist those who are involved with the testing of new simulations.

Current projects for attention will be listed in our List of Simulations for Evaluation and Testing.

Other simulations under development or planned are listed in our full List of Projects.

Those involved with testing need to be members of the pcrtest group.

You will also need to download and install the PC-Rail Plus software.

The simulation data files required can be downloaded using the links provided in the List of Simulations for Evaluation and Testing. Updated versions of the software will be issued from time to time, and it is most important that you ensure you are using the latest version for your testing.

Follow the procedure for Importing simulation data into the PC-Rail Plus software and you will then be able to run the simulation.

Evaluation and/or testing is carried out at different development stages. Please ensure you are aware of the stage which a simulation has reached, as this will determine the type of testing that is required. See below for further details.

Communicating the results of your testing

You should regularly report the results of your testing by posting messages to the pcrtest group. This ensures that the developer, as well as other testers, know of what you have done and what results have been obtained. Include the name of the simulation being tested in the Subject of the message. This feedback should be given, even if no problems are found.

All problems which you find in operating the simulation should be reported via the group. If a software problem is found, i.e. a System Error is reported while running the simulation, you should record full details to help in diagnosing the cause of the problem. See reporting software problems below.

Evaluation (Stage 2)

At this stage, the simulation layout is considered by the developer to be complete.
Comments and suggestions are welcome on any aspect of the simulation, including the appearance of the layout, what is included or not included on the display, and general user-friendliness.
Testing should include checking for:

correct setting of all signalled routes
the manual working of all points
train describer working
signal aspects

Trains should be run on all possible routes, checking for any problems, observing the operation of train descibers, etc.
The working of all menu options should be tested.
The working of the timetable editor can also be checked.
At this stage, no testing of the standard timetable is required.

Simulation Testing (Stage 3)

At this stage, all files for the simulation are considered by the developer to be complete.
The various checks carried out at Stage 2 (see above) should be repeated to ensure that subsequent data changes have not introduced any new problems.
Check all the other information provided to the user of the simulation, including:

Local Information
List of Routes
Timetable Notes
Carry out any further tests which you can think of, using the principle of "What happens if I try ... ??".

Timetable Testing (Stage 5)

The standard timetable is now complete and has been checked by the developer.

A complete test is required at skill level 5, without late running or failures. For the fullest possible check on timekeeping, the "Punctuality margin" can be set to zero.

Run the timetable (at level 5 if possible although it may be necessary to run at Level 9 during quiet periods) and make any notes covering:

conflicts with other trains/platforms
trains which cannot be run to time
unclear notes
spelling mistakes in notes

Try to work though a couple of hours or so of the timetable before posting a message to the group.

Additional testing with late running and/or failures in use is also beneficial in revealing hidden problems.

Final Testing (Stage 6)

At this stage, the simulation has already been extensively tested. It is considered complete and ready for a final quality check before releasing to the public.
A full re-test of the timetable is to be carried out as well as repeating all other checks and tests to ensure that nothing has been missed.
As always, report your findings via the pcrtest group. At his stage, reports of testing with no problems found are particularly important, and welcome!

Reporting Software Problems

First, make sure you are using the latest version of the program. If not, download the latest and install it.

Next, make sure you have a "saved session" which is saved just before the error occurs, for instance with the train approaching the signal which causes the error. If you do not have such a save, go back to the last save before it and restart from there. Don't forget to save at the right moment or you will have to go back again.

When the problem occurs, the error pop-up window also details the name of a 'diagnostic' file. Write down that name (number).

In the 'Help' menu, there is a button 'Diagnostics'. If you select this, a new window will open which allows you to select both the diagnostic file as detailed above, and the related saved session. Fill in the details of the error, e.g. which route for which train etc. caused the error to occur, and save the related diagnostic and saved session. Finally, send the generated zip-file directly to John Dennis (john@pcrail.co.uk).

John Dennis