Robert V. Binder

Open Source Tools for Model-Based Testing

Open Source Repositories

I promised to provide a list of open source and free model-based testing tools in a QUEST panel session, and have updated this list a few times since.  There is wide variation in maturity, stability, and provisioning.  

Nearly all of following information is from provider’s web sites – YMMV. The Environment column indicates the source code language and/or run-time environment(s); “*nix” indicates a tool written to run on most Linux/Unix variants.  Open source licenses vary widely, click here for an overview.

 If you note any errors or omissions, please post a comment and I will revise accordingly.  A revision history follows the last table.

 

Binaries

Runtime binaries (executable files) or installation media without source code.

Name  Description Environment License

JUMBL 5.0

The Java Usage Model Builder Library (JUMBL) supports automated model-based statistical testing of systems. Includes command-line and GUI interfaces. Limited availability. Java Univ of Tennesse

Spec Explorer 3.5

A Visual Studio add-in that generates test suites from C# model programs; provides visualization and modeling features. Windows, Visual Studio 2012 Microsoft

Torx 3.10.5

Supports conformance testing of reactive software; checks the correct behaviour of a real implementation during its execution based on its formal specification. Can execute a Torx specification as SUT. *nix, Windows with Cygwin Apache 2.0

Uppaal TRON 1.5

An integrated tool environment for modeling validation and verification of real-time systems modeled as networks of timed automata extended with data types (bounded integers). Non-commercial use only. *nix Not specified

 

Fielded Codebases

Tools with an active user community using the tool for non-research work; developers are at arms-length with the users.

Name Description Environment License

NModel

1.0.21029

A model-based testing and analysis framework. User writes model programs in C#. NModel generates tests from your model program in advance or on-the-fly as the test run executes. Users may add custom test generation strategies in C# to achieve any coverage criteria. SUT not limited to Windows.;C#. Requires .NET, but not Visual Studio. Precursor to Spec Explorer; lacks many features (e.g., Cord Files) supported in Spec Explorer.

Windows

Dot Net

Microsoft Open Source License

TopCased

5.1

Model-driven development and testing environment for critical hw/sw embedded systems. Supports formal methods, SysML, UML, and OCL. Part of a suite of open source tools for embedded system development.

Java

Eclipse

Eclipse Public

 

Club Codebases

Tools whose codebase has a small user community using it for non-research work.

Name Description Environment License

Graphwalker

2.5.16

Generates offline and online test sequences from state machines modeled with GraphML — no UML to learn. User selects any of seven built-in coverage (stopping) criteria for test generation. Tool may be integrated with a Java test harness or called with SOAP as web service from the developer’s server. Use of graphics to represent models limits scalability. Java MIT

IFX Omega

2.0

A compiler/simulator/model-checker for a rich subset of UML diagrams.  based on the VERIMAG IF model checker. May be used to validate and verify asynchronous timed system models with complex dynamics. Uses rigorous semantics suited for critical real-time embedded systems.

Java

Eclipse

Not Specified

ModelJunit

2.0

Models are written as Java classes which directly interact with a Java SUT or  adapter. Uses the annotations feature of Java 5.0. Processes models to generate calls to the SUT and evaluate response (no Junit code is produced.) Java, JDK 1.6 or higher GPL

fMBT 

0.12

The fMBT (Free Model-based Testing) tool is suitable for testing anything from individual C++ classes to GUI applications, mobile devices, and distributed systems as well as different platforms. fMBT provides a model editor, test generator, adapters for testing through various interfaces, and tools for analyzing logs.  *nix LGPL

 

Beta Codebases

Tools whose codebase has been recently released and has limited use.

Name Description Environment License
ATS4 AppModel Model app specification, model based testing, and test script generation. Simple interface to manage complex models. Developed at Nokia for mobile device testing. Java  BSD

Autofocus 3 

0.3.0

Supports modeling and analyzing the structure and behavior of distributed reactive and timed computer-based systems. Java/Eclipse Apache 2.0

Fastest

1.6

Generates tests from models expressed in the Z notation following the Test Template Framework proposed by Phil Stocks and David Carrington. Supports unit testing. Java Not Specified

JTorx

1.10.0

Generates tests to check that the ioco testing relation holds between an IUT and a labeled transition system model expressed in Aldeberan or GraphML. Derived from Torx. Java  BSD

GUITAR

1.0

Parses GUI framework code and monitors controlled execution of a GUI to  extract structure and API calls. Generates an Event-Flow Graph (test model) from this model. Generates executable test suites from the model.

Java,

*nix, Windows/Cygwin

GPL

MISTA

1.0

Uses lightweight high-level Petri nets as a visual modeling notation. Test models can be animated and verified. Generates executable test code fom a test model in Java, C, C++, C#, VB, or HTML and/or test engines JUnit, NUnit, Selenium. Java Not Specified

TEMA

3.2

A prototype tool and methodology for online test generation for smartphone GUI testing. Based on the GUI keywords/action word testing strategy.

Java

MIT

 

Development Codebases

Tools whose codebase is incomplete or used primarily by its developers.

Name Description Environment License

EvoMT

A model based software test utility using genetic algorithms to generate test oracles. Java Not Specified

NModel Remote Stepper

1.0

Extensions to NModel (see above) enabling model-based testing of systems implemented in Java or other languages. More …  Java Apache 2.0

OSMO Tester

2.1.1

Open Source Modelling Objects (OSMO) includes tools and libraries to model software behaviour with extended state machines in Java for online/offline test generation and non-deterministic online models.  Java  LPGL

Parteg

1.3.3

Partition Test Generator: generates tests meeting control flow-based coverage criteria (e.g.  MC/DC) and boundary-based coverage criteria.  Java Eclipse Eclipse Public

 

Revision History

Date Change
20 Jan 2014 Reformatted, verified all links and versions
26 Sep 2013 Added fMBT
31 May 2012 Added Fastest
25 April 2012 Formatting and clarifications
20 April 2012 Added GUITAR
15 April 2012 First posting


23 Comments


  1. That is an interesting list. Thanks for compiling it! What about the GUITAR model based GUI testing framework?

    http://sourceforge.net/apps/mediawiki/guitar/index.php?title=GUITAR_Home_Page

    • Robert V. Binder

      I will add it. I’ve been following GUITAR research for quite awhile and I’m glad to see that it has finally been released.

  2. Thanks for compiling the list. You may also add Fastest. It’s available from http://www.fceia.unr.edu.ar/~mcristia/index-en.html. Fastest is a MBT tool for the Z notation. Regards.

  3. Which of the tools given above under ‘Binaries’ category used for Black box testing?

    • All of them. A “binary” here refers to a tool that is released as an executable only file – the source code is not public.

  4. Hi,

    Did you already saw the DTM tool? For more information, please visit http://www.dtmtool.com.

    Cheers,
    Silvio

    • Hi Silvio

      Thanks for the suggestion. DTM looks interesting, especially in the way you’ve oriented it for Agile development.

      However, it doesn’t seem like it qualifies as open source.

  5. Hi,
    Firstly thanks for compiling the list..
    I would like to know if any tool can parse a .vsdx file.?

    • Hi Nevlin

      Assuming you mean a Visio file, not to my knowledge. However, SpecExplorer is a Dot Net/Visual Studio app, so I expect one could take advantage of any existing vsdx parsers without too much trouble.

  6. Hi,
    First of all thank you for this compilation of tools. I would like to as about The CREST EFSM slicing tool from University College London. Is it open source? if yes what is its download link?

    Thank you in advance and best regards.

  7. One more MBT tool, on it’s way to open source, thing is, made by a human for humans.
    http://valkoinenrapu.blogspot.com/2014/01/about-time.html

  8. Hi,

    Nice info, will any MBT generates TCL scripts?

  9. I stumbled upon Gallio and MB Unit today. Gallio is a open source test automation platform for Windows intended to integrate many testing tools, including MbUnit. MbUnit is a Dot Net test automation framework compatible with NUnit and adds many features. Although MbUnit is said to mean “Model based Unit Testing Framework”, neither tool supports generation of tests from models. The site http://code.google.com/p/mb-unit/ lists over 20000 downloads of the most recent (2012) release.

Trackbacks

  1. Open Source MBT · Model-Based Testing Community

Leave a Reply

Comment moderation is enabled, no need to resubmit any comments posted.