KWWidgets/GUI Testing
From KitwarePublic
Contents |
Papers, Articles
- GUI Software Testing (Wikipedia)
- Testing GUI Applications (Paul Gerrard, 1997)
- Elements of GUI Testing (Abbot)
- Effective GUI Testing Automation: Developing an Automated GUI Testing Tool, chapter 2 (Li, Wu)
- Important Considerations for Test Automation (Bazman)
- Testing GUI applications, part 1, part 2 (Laird & Soraiz, 2001)
- GUI Testing (Cunningham & Cunningham, Inc.)
- General Guidelines for GUI Testing (Mikhail Rakhunov, SQAtester)
- Tester Utility, Take 3: Adding Mouse Recording and Playback (John Robbins, 2002)
Solutions
- List of GUI Test Drivers (testingfaqs.org)
- Windows GUI test tools by popularity (kegel.com)
- Commercial Product Comparison Sheet (AdventNet)
- TestApps.com, info about automated software testing tools
(eTester, Functional Tester, QuickTest Pro, Robot, SilkTest, TestPartner, WinRunner)
For reference, here are the (simple) applications that were being tested while evaluating some of the applications listed below: Windows Calculator (Win32/MFC), the Tcl/Tk-based KWWidgets Wizard Dialog example, and a Tk Calculator (pure Tcl/Tk).
 Windows Calculator (Win32/MFC) |
 KWWidgets Wizard Dialog Example (KWWidgets) |
 Tk Calculator (Tcl/Tk) |
 Tk Calculator (Tcl/Tk Source Code) |
Freeware
- VNCRobot (How To Use, Demo)
=> Free, Cross platform, VNC client extension, framebuffer level, mouse/key, no notion of querying/inspecting/acting-on specific widgets, image-based checks only - Android
=> Free, X11, no notion of control ID/caption, just click on screen at specific coords, and key events, recorder. - Automated GUI Testing with Perl (George Nistorica, 2005) Win32::GuiTest, X11::GuiTest
=> Free, Win32/Perl based, no support for Tcl/Tk, TkChild object obfuscate native controls, no caption/control IDs - AutoIt
=> Free, Win32, script only, no recorder, MFC only (no Tk) - GUI Testing FrAmewoRk (GUITAR) (papers)
=> Free, Cross Platform (Java), not working for me on Vista (GUI Ripper not ripping) - Automated GUI testing with Eclipse (Long, 2007) An Introduction to Using TPTP's Automated GUI Recorder
=> Free, Cross platform, only supports testing of Eclipse itself, Eclipse-based plugins, or Eclipse-based Rich Client Applications (i.e. Java/SWT). - dogtail, Automated GUI testing with Dogtail (DiMaggio, 2006) , Linux Desktop Testing Project
=> Free, Python, requires an AT-SPI-enabled desktop (i.e. GNOME). - The MS ScriptIt Utility (1998)
Free, Win32, only send keystrokes, need scripting (no recorder)
 VNCRobot (Starting a Session) |
 VNCRobot (Replaying a script on the KWWidgets Wizard Dialog Example) |
 Android (xcalc Script) |
 Win32::GuiTest (Windows Calculator Script) |
 WinSpy++ (Inspecting the Windows Calculator) |
 WinSpy++ (Inspecting the KWWidgets Wizard Dialog) |
 AutoIt3 (Windows Calculator Script) |
 AutoIt3 (Inspecting Windows Calculator) |
 GUITAR |
Commercial
- TestComplete
=> $999 - $1999, Win32, very powerful, VBScript/JScript/C++/C# scripting language, recorder, MFC (no Tcl/Tk support), application states, test suites, window inspector, data/database driven (injection/template), testing agent & distributed testing, OCR, verification/checkpoints. - Ranorex
=> €590 - €1990, requires Vista's UAC to be Off, recorder, XML/C#/Python/C++ OO scripting, MFC (no Tk support), window inspector. - Silktest (Borland)
=> $6500, Win32, very powerful, 4Test scripting language, recorder, MFC (no Tcl/Tk support), application states, test suites, window inspector, data/database driven (injection/template), testing agent & distributed testing. - Vermont HighTest Plus
=> $$$, Win32, OO script, recorder, screen & UI controls capture/comparison, breakpoints, test suites, data driven, playback control, window inspector, doesn't work for Vista right now. - vTask
=> $150, Win32, MFC, scripting language, no recorder. - Squish, Squish for Tk
=> >€2500, Win32/Qt/Java/Web/Tk, scripting in Tcl/Python/JavaScript/Perl, recorder, window inspector, breakpoints/verification, no support for KWW - TestWorks, eValid (for Web)
=> $$$, Unix/Win32, hard to use, non standard UI - WinRunner
=> $5000 - $8000, Win32 - X Remote Control
=> $2500, X11, maintained? - AutoPilot
=> $$$, MacOS 9/X
 TestComplete (Windows Calculator Script) |
 TestComplete (Inspecting Windows Calculator) |
 SilkTest (Windows Calculator Script) |
 Ranorex (Windows Calculator Script) |
 Ranorex Recorder (Recording Windows Calculator Interaction) |
 vTask (Windows Notepad Script) |
 Squish (Recording Tk Calculator Interaction) |
 Squish (Tk Calculator Script) |
 Squish (Creating a Verification Point with the Spy) |
 Squish (Verification Point in Tk Calculator Script) |
 Squish (Verification Point & Test Result) |
Feature Requests
- Workflow: Plan -> Capture -> Create -> Run -> Report
- Projects, Test suites
- Application States (from which a test can be run)
- Scripting
- Recorder
- Window Inspector, Easy Identification of Controls (caption > control ID > child rank)
- Verification Points (screen cap, UI/Control cap (ex: text fields), file output)
- Data Driven (variable replacement, injection)
| |
