The following Test Plan was written to test all of the facilities provided by BOTSsim. You will notice that the Test Plan addresses all of the facilities provided by each window or dialogue. The Context-Sensitive Online Help System will also be tested for each dialogue and window. As well as testing whether facilities work or not we have also planned to test whether BOTSsim meets the wishes made by our client. These tests appear at the end of the Test Plan.
1. Make sure that the appropriate Help Topic gets displayed.
1. Make sure that it updates the Status Bar correctly.
2. Make sure it updates the Title Bar correctly.
3. Make sure it loads the default Environment Template.
4. Make sure it loads the default Run Information Variables.
5. Make sure it enables and disables the appropriate functionality.
1. Make sure that it updates the Status Bar correctly.
2. Make sure it updates the Title Bar correctly.
3. Make sure it loads the default Environment Template.
4. Make sure it loads the default Run Information Variables.
5. Make sure it enables and disables the appropriate functionality.
6. Make sure it only loads valid Runs.
1. Make sure that it updates the Status Bar correctly.
2. Make sure it enables and disables the appropriate functionality.
3. Make sure it saves the Run correctly.
1. Make sure that it updates the Status Bar correctly.
2. Make sure it loads the correct Environment Template.
3. Make sure it only loads valid Environment Templates.
1. Make sure it passes control to TED.
1. Make sure it passes control to the Run Information Dialogue.
1. Make sure it starts the Brain component.
2. Make sure it starts the Environment component.
3. Make sure the Brain gets initialised correctly.
4. Make sure the Progress Bar gets updated.
1. Make sure it pauses the Brain component.
2. Make sure it pauses the Environment component.
3. Make sure it updates the Title Bar correctly.
4. Make sure the Progress Bar does not get updated.
1. Make sure it unpauses the Brain component.
2. Make sure it unpauses the Environment component.
3. Make sure it updates the Title Bar correctly.
4. Make sure the Progress Bar gets updated.
1. Make sure it stops the Brain component.
2. Make sure it stops the Environment component.
3. Make sure it updates the Title Bar correctly.
4. Make sure the Progress Bar gets reset (zeroed out).
1. Make sure the correct Environment gets displayed.
2. Make sure the Environment Number gets validated correctly.
3. Make sure multiple Environments can be displayed independently.
1. Make sure the correct Bot gets displayed.
2. Make sure the Bot ID gets validated correctly.
3. Make sure the Bot Menu Rankings are correct.
1. Make sure it passes control to the Histogram Window.
1. Make sure it passes control to the View Communications Dialogue.
1. Make sure the Context-Sensitive Online Help system gets launched correctly.
1. Make sure it shows the About BOTSsim Dialogue.
1. Make sure it exits BOTSsim.
1. Make sure that the appropriate Help Topic gets displayed.
1. Make sure that it loads the default Environment Template.
2. Make sure it asks you to save any changes before creating the new Template.
1. Make sure that it loads the correct Environment Template.
2. Make sure that invalid Environment Template are detected.
3. Make sure it asks you to save any changes before loading the existing Template.
1. Make sure that it saves the changes properly.
2. Make sure that it updates the Main Dialogue Status Bar correctly.
1. Make sure that it loads the appropriate properties dialogue for each object.
2. Make sure that it does not crash if no object is selected.
1. Make sure that it destroys the selected object only.
2. Make sure that it does not crash if no object is selected.
1. Make sure it creates the Wall in the right position.
2. Make sure it creates no more than MAX_WALLS Walls.
1. Make sure it creates the Bin in the right position.
2. Make sure it creates no more than MAX_BINS Bins.
1. Make sure it creates the DA in the right position.
2. Make sure it creates no more than MAX_DAS DAs.
1. Make sure it creates the Bot in the right position.
2. Make sure it creates no more than MAX_BOTS Bots.
1. Make sure the Context-Sensitive Online Help system gets launched correctly.
1. Make sure it shows the About BOTSsim Dialogue.
1. Make sure it asks you to save any changes before closing.
1. Make sure that the appropriate Help Topic gets displayed.
1. Make sure it updates the Bot start direction correctly.
2. Make sure it updates the Bot colour correctly.
1. Make sure that the appropriate Help Topic gets displayed.
1. Make sure it creates an Environment Template of the correct Width and Height.
1. Make sure that the appropriate Help Topic gets displayed.
1. Make sure that the correct Run Information File gets opened.
2. Make sure that invalid Run Information Files are detected.
3. Make sure it asks you to save any changes before loading the Run Information File.
1. Make sure that it saves the changes properly.
2. Make sure that it updates the Main Dialogue Status Bar correctly.
1. Make sure it asks you to save any changes before closing the dialogue.
1. Make sure that the appropriate Help Topic gets displayed.
1. Make sure that the Environment Number gets validated.
2. Make sure that the correct Environment gets displayed.
1. Make sure that the operation is definitely cancelled.
1. Make sure that the appropriate Help Topic gets displayed.
1. Check that Bots are displayed and animated correctly.
2. Check that Bot IDs are displayed correctly.
3. Check that Walls are displayed correctly.
4. Check that DAs are displayed and animated correctly.
5. Check that Bins are displayed and animated correctly.
6. Check that Blocks are displayed and animated correctly.
7. Check that zooming works correctly.
8. Check that scrolling works correctly.
9. Check that showing Bot Windows works correctly.
1. Make sure that the window is completely disposed of.
1. Make sure that the appropriate Help Topic gets displayed.
1. Make sure that the Bot ID gets validated.
2. Make sure that the correct Bot gets displayed.
1. Make sure that the operation is definitely cancelled.
1. Make sure that the appropriate Help Topic gets displayed.
1. Check that the Hits are displayed.
2. Check that the Pickups are displayed.
3. Check that the Drops are displayed.
4. Check that the correct Environment Number is displayed.
5. Check that the Raw Fitness is formatted correctly.
1. Check that the Bot's Wall Sensors are updated correctly.
2. Check that the Bot's Bot Sensors are updated correctly.
3. Check that the Bot's Block Sensors are updated correctly.
4. Check that the Bot's Bin Sensors are updated correctly.
5. Check that the Bot is updated correctly when:- Not holding a block, Holding a block, Not holding a block and Colliding with another object, Holding a block and Colliding with another object.
1. Check that the Rule Set is displayed in the correct format.
2. Check that the button changes to Hide Rule Set when the Rule Set is displayed.
3. Check that the button changes to Show Rule Set when the Rule Set is not displayed.
1. Check that the correct Environment is displayed--should correspond with the Environment Number in the Bot's Statistics Display.
1. Make sure that the window is completely disposed of.
1. Make sure that the appropriate Help Topic gets displayed.
1. Make sure that the Histogram is properly updated.
1. Make sure that the Hits Histogram is properly displayed.
1. Make sure that the Pickups Histogram is properly displayed.
1. Make sure that the Drops Histogram is properly displayed.
1. Make sure that the window is properly closed.
1. Make sure that only the most recently received 20 messages are displayed.
1. Make sure that only the most recently sent 20 messages are displayed.
1. Make sure that the window is properly closed.
1. Make sure that every piece of text that is underlined and is green in colour goes to the correct Help Topic when clicked on by the user.
1. Make sure that the Contents button links to the Contents Help Topic.
1. Make sure that the Index button displays the Index of all available Help Topics.
1. Make sure that the Back button links to the previously accessed Help Topic.
1. Make sure that the Print button prints the appropriate Help Topic.
1. Make sure that the << button links to the previous Help Topic.
1. Make sure that the >> button links to the next Help Topic.
Test that the Bots achieve Client Wish List Item 1 correctly and consistently.
Test that the Bots achieve Client Wish List Item 2 correctly and consistently.
Test that the Bots achieve Client Wish List Item 3 correctly and consistently.
Here are the Test Results that correspond to the test plan above.
1. The appropriate Help Topic does get displayed.
1. The Status Bar is updated correctly every time.
2. The Title Bar is updated correctly every time.
3. The default Environment Template is loaded and is blank.
4. The default Run Information Variables are loaded correctly.
5. You cannot Start, Pause, Reset, View or Save a Run.
1. The Status Bar is updated correctly every time.
2. The Title Bar is updated correctly every time.
3. The correct Environment Template is loaded.
4. The correct Run Information Variables are loaded.
5. You can Start a Run, but you cannot Pause, Reset, View or Save it.
6. Invalid Runs are detected and the user is informed about them.
1. The Status Bar is updated correctly every time.
2. You cannot Start a Run, but you can Pause, Reset, View or Save it.
3. The Run is saved correctly every time (assuming there is no disk error).
1. The Status Bar is updated correctly every time.
2. The correct Environment Template is loaded.
3. Invalid Environment Templates are detected and the user is informed about them.
1. Control is passed to TED.
1. Control is passed to the Run Information Dialogue.
1. The Brain component is started correctly (check this in Windows NT Task Manager).
2. The Environment component is started correctly (because simulation is performed).
3. The Brain gets initialised correctly, we can trace all messages to and from the Brain.
4. The Progress Bar gets updated to zero, and incremented as the Run progresses.
1. The Brain component is paused--it receives no input, so it does no processing.
2. The Environment component is paused because animation stops.
3. The Title Bar is updated correctly every time.
4. The Progress Bar does not get updated, you can see it stop when pause is pressed.
1. The Brain component is unpaused--it receives input, so it does processing.
2. The Environment component is unpaused--animation resumes.
3. The Title Bar is updated correctly every time.
4. The Progress Bar does get updated, you can see it resume when unpause is pressed.
1. The Brain component stops because it is sent a close message and terminated.
2. The Environment component stops because all animation and simulation ceases.
3. The Title Bar is updated correctly every time.
4. The Progress Bar does get zeroed out (it is fairly easy to see).
1. The correct Environment does get displayed every time.
2. The Environment Number does get validated correctly. If an invalid Environment Number is entered a dialogue box pops up letting the user know what is wrong.
3. Multiple Environments can be displayed independently. We know this because animation and zooming occur independently (this can be seen on the screen). We also know that the Windows are destroyed properly because you can always have up to 10 Environment Windows open no matter how many of them you open and close.
1. The correct Bot does get displayed.
2. The Bot ID does get validated correctly. If an invalid Environment Number is entered a dialogue box pops up letting the user know what is wrong.
3. The Bot Menu Rankings are correct. This was tested by comparing the fitnesses of individual Bots in the same population (view the Bot Window).
1. Control is passed to the Histogram Window.
1. Control is passed to the View Communications Dialogue.
1. The Context-Sensitive Online Help System does get launched correctly.
1. The About BOTSsim Dialogue is shown.
1. This obviously works. Try it for yourself, we did on many occasions.
1. The appropriate Help Topic does get displayed.
1. The default Environment Template is loaded (a blank one).
2. You are asked to save any changes before creating the new Template.
1. The correct Environment Template is loaded.
2. Invalid Environment Templates are detected and the user is informed about them.
3. You are asked to save any changes before loading an existing Template.
1. The Environment Template is saved correctly.
2. The Main Dialogue Status Bar is updated correctly.
1. The appropriate properties dialogue for each object is loaded. The Bot Properties Dialogue for Bots and the Change Colour Dialogue for any other object.
2. Nothing happens if there is no object selected.
1. Only the selected object is destroyed.
2. Nothing happens if there is no object selected.
1. The Wall is created in the position that the user wants.
2. You can create as many Walls as you like (within reason).
1. The Bin is created in the position that the user wants.
2. The user is informed if they try to create too many Bins.
1. The DA is created in the position that the user wants.
2. The user is informed if they try to create too many DAs.
1. The Bot is created in the position that the user wants.
2. The user is informed if they try to create too many Bots.
1. The Context-Sensitive Online Help System does get launched correctly.
1. The About BOTSsim Dialogue is shown.
1. You are asked to save any changes before closing the window.
1. The appropriate Help Topic does get displayed.
1. The Bot start direction is updated correctly.
2. The Bot colour is updated correctly. You must de-select the Bot to see this.
1. The appropriate Help Topic does get displayed.
1. An Environment Template of the correct Width and Height is created.
1. The appropriate Help Topic does get displayed.
1. The correct Run Information File is opened.
2. Invalid Run Information Files are detected and the user is informed about them.
3. You are asked to save any changes before loading an existing Run Information File..
1. All changes are saved properly.
2. The Main Dialogue Status Bar is updated correctly.
1. You are asked to save any changes before closing the dialogue.
1. The appropriate Help Topic does get displayed.
1. The Environment Number gets validated. If it is invalid then a message is displayed
2. The correct Environment gets displayed, this is obvious from the Environment Number shown in the Window Title.
1. The operation is cancelled and nothing happens.
1. The appropriate Help Topic does get displayed.
1. Bots are displayed and animated correctly.
2. Bot IDs are displayed correctly.
3. Walls are displayed correctly.
4. DAs are displayed and animated correctly.
5. Bins are displayed and animated correctly.
6. Blocks are displayed and animated correctly.
7. Zooming works correctly, even if it isn't intuitive.
8. Scrolling does not work correctly. We spent a lot of time on this and still could not figure out exactly what was wrong. Now that I think about it I do know what's wrong--the scroll bar dimensions are not updated when the zoom factor is changed.
9. Showing Bot Windows works correctly, this is done by right clicking on the Bot.
1. The Environment Window is completely disposed of.
1. The appropriate Help Topic does get displayed.
1. The Bot ID gets validated. If it is invalid then an error message is displayed
2. The correct Bot gets displayed, this is obvious from the Bot ID in the Window Title.
1. The operation is cancelled and nothing happens.
1. The appropriate Help Topic does get displayed.
1. The Hits are displayed correctly--all values correlate with the Hits Histogram.
2. The Pickups are displayed correctly--all values correlate with the Pickups Histogram.
3. The Drops are displayed correctly--all values correlate with the Drops Histogram.
4. The correct Environment Number is displayed correctly, this is proven by clicking the Show Environment button and checking if that Bot is shown in that Environment.
5. The Raw Fitness is not formatted correctly sometimes the number becomes too long.
1. The Bot's Wall Sensors are updated correctly--this is checked by viewing the Environment Window that that Bot is operating in.
2. The Bot's Bot Sensors are updated correctly--this is checked by viewing the Environment Window that that Bot is operating in.
3. The Bot's Block Sensors are updated correctly--this is checked by viewing the Environment Window that that Bot is operating in.
4. The Bot's Bin Sensors are updated correctly--this is checked by viewing the Environment Window that that Bot is operating in.
5. The Bot is updated correctly when:- Not holding a block, Holding a block, Not holding a block and Colliding with another object, Holding a block and Colliding with another object. Again this is also checked by viewing the Environment Window that that Bot is operating in and seeing if it has a block or is involved in a collision.
1. The Rule Set is displayed in the correct format.
2. The button caption changes to Hide Rule Set when the Rule Set is displayed.
3. The button caption changes to Show Rule Set when the Rule Set is not displayed.
1. The correct Environment is displayed. We know this because the Environment Number in the new Environment Window is the same as the one in the Bot Window.
1. The Bot Window is closed properly
1. The appropriate Help Topic does get displayed.
1. The Histogram is properly updated. This can be observed by comparing the Histogram results with the appropriate values for all of the Bots. We have done this for reasonably large populations and gave up about one tenth of the way through when we knew that it was most probably correct. This assumption was based on the fact that it always worked correctly for small populations of between 10 and 20.
1. The Hits Histogram is properly displayed. The proof for this is similar to the proof for the Update Histogram Facility.
1. The Pickups Histogram is properly displayed. The proof for this is similar to the proof for the Update Histogram Facility.
1. The Drops Histogram is properly displayed. The proof for this is similar to the proof for the Update Histogram Facility.
1. The Histogram Window is properly closed.
1. Only the most recently received 20 messages are displayed. That is when the DEBUG flag is turned on. You will not be able to see this as the version you receive will be the release version and not the debug version.
1. Only the most recently sent 20 messages are displayed. That is when the DEBUG flag is turned on. You will not be able to see this as the version you receive will be the release version and not the debug version.
1. The View Communications Dialogue is properly closed.
1. It took a hell of a long time but I am 95% sure that all of the Hyper Links work correctly. As it is a reasonably complex Help System I cannot be completely sure but a am 95% sure. The Context-Sensitive links do work, I am 100% sure of this fact.
1. The Contents button does in fact link to the Contents Help Topic.
1. The Index button does display an Index of all available Help Topics.
1. The Back button does link to the previously accessed Help Topic. The Back button is disabled when you are looking at your first topic and have not followed any Hyper Links.
1. The Print button should print the appropriate Help Topic if printing is configured properly. You are presented with the standard Print Dialogue in all cases anyway.
1. The << button links to the previous logical Help Topic. This was easy to prove.
1. The >> button links to the next logical Help Topic. This was easy to prove.
Test that the Bots achieve Client Wish List Item 1 correctly and consistently...
Test that the Bots achieve Client Wish List Item 2 correctly and consistently...
Test that the Bots achieve Client Wish List Item 3 correctly and consistently...
This has not been achieved. However, an attempt to do this has been mirroring, described in full in section 3.3 on page 13 of this document.
The problem is that mirroring doesn't have the desired effect. The Rule Set is mirrored properly, this can be seen by looking at the Rule Set of the 1 Bot left. The major problem is that Bots don't take a symmetric approach to getting a block and placing it in a bin.
(BlockLeft (WallLeft (WallLeft (BlockRight (BinCollide DropBlock TurnRight ) (BlockCollide GetBlock GoBackward ) ) (BotForward (BotForward NoOp GetBlock ) (WallLeft NoOp (BotLeft GoBackward NoOp ) ) ) ) (BotForward (BotRight (BinCollide DropBlock TurnRight ) (BlockCollide (BotRight TurnLeft NoOp ) (BlockCollide GetBlock GoBackward ) ) ) (BotForward (BlockForward (WallBackward TurnLeft TurnLeft ) (HasBlock (BotForward NoOp TurnLeft ) (BinCollide DropBlock TurnRight ) ) ) (BinCollide (WallBackward TurnLeft TurnLeft ) TurnRight ) ) ) ) (BlockCollide GetBlock GoBackward ) )
(HasBlock (BinLeft (WallLeft (WallLeft (BinLeft NoOp (BinCollide (BotForward (BlockBackward (WallLeft TurnLeft NoOp ) (BlockCollide DropBlock GoBackward ) ) (BinCollide (BlockBackward (WallLeft GetBlock NoOp ) (WallBackward TurnLeft TurnLeft ) ) TurnRight ) ) TurnRight ) ) (BinCollide (BinCollide (BlockCollide DropBlock GoBackward ) TurnRight ) TurnRight ) ) (BinCollide GoForward TurnRight ) ) (BlockCollide DropBlock GoBackward ) ) (BlockLeft (WallLeft (WallLeft (BlockLeft NoOp (BinCollide (BotForward (BinBackward (WallLeft TurnLeft NoOp ) (BlockCollide GetBlock GoBackward ) ) (BinCollide (BinBackward (WallLeft DropBlock NoOp ) (WallBackward TurnLeft TurnLeft ) ) TurnRight ) ) TurnRight ) ) (BinCollide (BinCollide (BlockCollide GetBlock GoBackward ) TurnRight ) TurnRight ) ) (BinCollide GoForward TurnRight ) ) (BlockCollide GetBlock GoBackward ) ) )
Selection was tested by seeding the Brain. This testing has not been documented because to produce the results we would have to redo the testing that we did early this semester. That would take a very long time to do and would not be the same testing. We can say that Selection does work correctly.
These cannot be shown unless we change the source files and recompile. The results of testing the fitness functions are shown in the fitness table. See section 3.2 on page 11 of this document for more information about fitness functions.