General test of a Windows App
Is your app done? Is it complete? Didn’t forget anything? Or testing an app someone else created? The problem is – what to test, where to start and not to miss important things. Maybe this quick checklist can help you cover the basics. It was created by using information from the App Fast Track team. First here is the brief checklist, then we go a bit more into detail.
- User Experience
- Navigation and App bars
- Suspend mode
- Tile and Notification
- Touch, keyboard and mouse
- Windows App Certification Kit (WACK)
This information is also available in PDF format including a checklist for testing an app.
1. User Experience
The basics for this can be found on http://design.windows.com with more on the UX guidelines, also in PDF format. There are a few things that are easy to check:
- First the app must have a clear focus which can be described in a ‘Best at’ statement.
- The app should deliver the core scenario’s to support that focus to the user. It should be supported natively in the app without using browser controls.
- The landing page should highlight the goals of the app at a glance.
See more on Planning Windows Store apps.
A bit more detailed things to look for in this context are:
2. Navigation and App bars
- Check if there is a clear way to navigate through the app/data. Focus on where you are, not on where you can go.
See Navigation Design for apps.
- Semantic Zoom should be used to jump around a long list of content.
See Guidelines for Semantic Zoom.
- Additional commands are placed in the bottom app bar, navigation is placed in the top app bar. The app bars are only available on a page where needed and contains contextual commands.
See Guidelines for app bars.
- Check handling of network unavailability and disruptions.
- When in the app, disable the network.
- With the network still disabled, terminate the app and restart it.
- If you have access to a network which requires a webpage to enter credentials, use that one, don’t enter credentials and start the app. The app will check for network, but any request will result in another webpage then expected. Does this work/fail?
If you are using a wired connection make sure you disable/enable the adapter (Go to Start screen, type “view network connections”, select “Settings” and click “View network connections”, select the adapter and click Disable/Enable).
For wireless connections, simply enable/disable the Airplane mode (Go to desktop, click the network icon in tray and set Airplane mode on/off).
4. Suspend mode
- Start the app, navigate to a detail page, switch to the desktop and start the Task Manager (Ctrl+Shift+ESC). Wait until the app is in Suspend mode and switch back to the app. Are you still where you were before?
If you can’t see if the app is suspended in the Task Manager, click the View menu, click Status values and make sure “Show suspended status” is selected.
See Guidelines for app suspend and resume.
This is only for apps where the search from the charms bar should be supported. If it’s not supported and you would expect it, then you know what to do.
- While in the app, go to the search charm for these tests (WinFlg+Q)
- Determine a specific word in the content of the app and search for it. Does it return the item that was originally spotted? Is there a difference where the text is (for instance title or content) and is that to be expected?
- Enter the same word but use spaces around it (e.g. “ text “). Is trimming applied to the search term?
- Leave the text to search for empty and search. What’s the result?
- Enter a few spaces as text to search for and search. What’s the result? (should be the same as the previous test)
- Enter something that cannot be found (e.g. “xxxxxxxxxxxxxxx”) and search. Does the result indicate that nothing could be found?
- If there are lots of results, is there a way to filter the content?
See Guidelines and checklist for search.
An app can be a share target and/or a share source. This only applies for apps where you would expect either one of them or both.
- Navigate to content and go to the share charm (WinFlg+H).
- Can you share content? Is it useful/expected information?
- Navigate to other content and share again. Is the correct content shared?
- Navigate to a page that doesn’t support sharing, perhaps the landing page. What happens now? Should result in “nothing to share”.
See Guidelines for sharing content.
- Go to the app settings in the charms bar (WinFlg+I)
- If an internet connection is used, is there a Privacy Statement?
- Is there an About with a support link (even if it is a mailto)?
- Are settings grouped in clear groups? Understandable for the user?
- Can you navigate through the settings (using the back button)?
See Guidelines for app settings.
- Check for snapping support by snapping the app beside another app. If your resolution doesn’t support snapping (need at least 1366x768), use the Windows Simulator.
- Start the Windows Simulator for the next steps and launch the app in it.
- What happens if you rotate the device (try all 4 – normal, 90, 180 and 270 degrees). Is the view expected? Navigate through the app as well.
- Try the app in the resolutions 10.6” 1024x768, 10.6” 1366x768, 23” 1920x1080 and 27” 2560x1440. What happens with the content? Does it extend, reposition or scale? Is it to expected?
Checking for scaling can be easily done with the Windows Simulator which comes with Visual Studio 2012. You can usually find it in C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Simulator\11.0 and it’s called “Microsoft.Windows.Simulator”. This can be pinned to the Start screen for easy access outside of Visual Studio.
See Guidelines for scaling to screens.
- Can you interact with the items on the screen as expected? Tapping, selecting, swiping, semantic zoom, etcetera).
- Are animations applied for feedback (tilting)
- Are grids and lists aligned so that content just of the screen are ‘peeked’ (just of the screen to hint that there is more)?
See Guidelines for common user interactions.
10. Tile and Notifications
- Good tile design? Branding okay? No ads in tile?!
- Check the size of the tile. Large tile only for live tiles, otherwise just a small tile should be used.
- Are secondary tiles implemented? Do they work okay?
- Are notifications raised? And are they also visible in the live tile and the app?
See Guidelines and for tiles, Guidelines for secondary tiles and Guidelines for toast notifications.
11. Touch, keyboard and mouse
- Is touch supported as expected?
- Are the gestures within the common set defined in the Windows 8 Touch Language?
- Can the app be used with the mouse? Also semantic zoom?
- When using the mouse, are there tooltips used where appropriate?
- Can the app be used with just the keyboard? Is the current focus visible?
- When keyboard shortcuts can be used, are they mentioned in tooltips?
See The mouse, the pen, the keyboard and a touch-first world.
12. Windows App Certification Kit (WACK)
- Test the app with the Windows App Certification Kit
See Testing your app with the Windows App Certification Kit.
- What’s the overall feel of the performance of the app? Is it responsive?
- If you can, test the app on an ARM device. What’s the performance on a low-powered device?
- Is the appropriate progress indicator used for long running actions?
See How to improve performance in your app and Guidelines for progress controls.