phpBB

Development Wiki

Difference between revisions of "Automated Tests"

From phpBB Development Wiki

(Compiled all the Unit Tests and Functional Tests wiki pages into one.)
 
Line 6: Line 6:
  
 
Unit tests simply verify that specific portions of the source code work properly. These tests are located in the main phpBB repository at ./tests/ and are grouped into folders based on commonality between the functionality they are testing. For instance, tests that are meant to test the config will be in ./tests/config/.
 
Unit tests simply verify that specific portions of the source code work properly. These tests are located in the main phpBB repository at ./tests/ and are grouped into folders based on commonality between the functionality they are testing. For instance, tests that are meant to test the config will be in ./tests/config/.
 +
 +
=== Running Unit Tests ===
 +
==== Install PHP extensions ====
 +
Unit tests use several PHP extensions that phpBB itself does not use. These need to be installed and enabled for unit tests to work.
 +
 +
Unit tests themselves need:
 +
 +
* ctype
 +
* simplexml
 +
 +
For database tests you additionally need to have pdo and pdo drivers for each of the databases you want to test with (e.g. pdo_pgsql for postgres).
 +
 +
PHPUnit needs:
 +
 +
* ctype
 +
* dom
 +
 +
==== Installing phpunit ====
 +
 +
You need to install PHPUnit to your server: http://www.phpunit.de/
 +
In order to install PHPUnit correctly, you need to run the following commands:
 +
<pre>pear channel-update pear.php.net
 +
pear install PEAR-1.9.1
 +
pear channel-discover pear.phpunit.de
 +
pear channel-discover components.ez.no
 +
pear channel-discover pear.symfony-project.com
 +
pear install --alldeps phpunit/PHPUnit
 +
pear install phpunit/DbUnit</pre>
 +
 +
Depending on what versions of phpunit and pear you have installed or had installed earlier, phpunit may silently break itself. If you get errors about missing files when running tests, uninstall phpunit and pear completely and then reinstall the latest versions of pear and phpunit.
 +
 +
==== Run Tests ====
 +
 +
You need to switch into the root directory of the git repository and then run:
 +
* Linux:<pre>phpunit</pre>
 +
* Windows:<pre>YOUR_DIRECTORY_WHERE_THE_FILE_IS/phpunit.bat</pre> (Exp. C:/xampp/php/phpunit.bat )
 +
 +
Afterwards you should see something like:
 +
<pre>PHPUnit 3.4.11 by Sebastian Bergmann.
 +
 +
............................................................  60 / 688
 +
............................................................ 120 / 688
 +
IIIIIIIIIII................................................. 180 / 688
 +
............................................................ 240 / 688
 +
............................................................ 300 / 688
 +
............................................................ 360 / 688
 +
............................................................ 420 / 688
 +
............................................................ 480 / 688
 +
............................................................ 540 / 688
 +
............................................................ 600 / 688
 +
............................................................ 660 / 688
 +
............................
 +
 +
Time: 5 seconds, Memory: 13.00Mb
 +
 +
OK, but incomplete or skipped tests!
 +
Tests: 688, Assertions: 878, Incomplete: 11.</pre>
 +
If there are no errors, your test was successful. Otherwise either a test was wrong, or the functionality itself is wrong. Then you should go and fix it.
  
 
=== Writing Unit Tests ===
 
=== Writing Unit Tests ===
Line 122: Line 180:
 
</dataset>
 
</dataset>
 
</pre>
 
</pre>
=== Running Unit Tests ===
 
==== Install PHP extensions ====
 
Unit tests use several PHP extensions that phpBB itself does not use. These need to be installed and enabled for unit tests to work.
 
 
Unit tests themselves need:
 
 
* ctype
 
* simplexml
 
 
For database tests you additionally need to have pdo and pdo drivers for each of the databases you want to test with (e.g. pdo_pgsql for postgres).
 
 
PHPUnit needs:
 
 
* ctype
 
* dom
 
 
==== Installing phpunit ====
 
 
You need to install PHPUnit to your server: http://www.phpunit.de/
 
In order to install PHPUnit correctly, you need to run the following commands:
 
<pre>pear channel-update pear.php.net
 
pear install PEAR-1.9.1
 
pear channel-discover pear.phpunit.de
 
pear channel-discover components.ez.no
 
pear channel-discover pear.symfony-project.com
 
pear install --alldeps phpunit/PHPUnit
 
pear install phpunit/DbUnit</pre>
 
 
Depending on what versions of phpunit and pear you have installed or had installed earlier, phpunit may silently break itself. If you get errors about missing files when running tests, uninstall phpunit and pear completely and then reinstall the latest versions of pear and phpunit.
 
 
==== Run Tests ====
 
 
You need to switch into the root directory of the git repository and then run:
 
* Linux:<pre>phpunit</pre>
 
* Windows:<pre>YOUR_DIRECTORY_WHERE_THE_FILE_IS/phpunit.bat</pre> (Exp. C:/xampp/php/phpunit.bat )
 
 
Afterwards you should see something like:
 
<pre>PHPUnit 3.4.11 by Sebastian Bergmann.
 
 
............................................................  60 / 688
 
............................................................ 120 / 688
 
IIIIIIIIIII................................................. 180 / 688
 
............................................................ 240 / 688
 
............................................................ 300 / 688
 
............................................................ 360 / 688
 
............................................................ 420 / 688
 
............................................................ 480 / 688
 
............................................................ 540 / 688
 
............................................................ 600 / 688
 
............................................................ 660 / 688
 
............................
 
 
Time: 5 seconds, Memory: 13.00Mb
 
 
OK, but incomplete or skipped tests!
 
Tests: 688, Assertions: 878, Incomplete: 11.</pre>
 
If there are no errors, your test was successful. Otherwise either a test was wrong, or the functionality itself is wrong. Then you should go and fix it.
 
  
 
== Functional Tests ==
 
== Functional Tests ==

Revision as of 23:57, 21 March 2012

To ensure code integrity as new features are added and existing features are altered by patches, phpBB utilizes the PHPUnit test framework. This way, the development team can keep track of whether or not a change made to the core will effect other existing features in unforeseen ways.

There are two main types of tests that can be written and run: unit tests, and functional tests.

Unit Tests

Unit tests simply verify that specific portions of the source code work properly. These tests are located in the main phpBB repository at ./tests/ and are grouped into folders based on commonality between the functionality they are testing. For instance, tests that are meant to test the config will be in ./tests/config/.

Running Unit Tests

Install PHP extensions

Unit tests use several PHP extensions that phpBB itself does not use. These need to be installed and enabled for unit tests to work.

Unit tests themselves need:

  • ctype
  • simplexml

For database tests you additionally need to have pdo and pdo drivers for each of the databases you want to test with (e.g. pdo_pgsql for postgres).

PHPUnit needs:

  • ctype
  • dom

Installing phpunit

You need to install PHPUnit to your server: http://www.phpunit.de/ In order to install PHPUnit correctly, you need to run the following commands:

pear channel-update pear.php.net
pear install PEAR-1.9.1
pear channel-discover pear.phpunit.de
pear channel-discover components.ez.no
pear channel-discover pear.symfony-project.com
pear install --alldeps phpunit/PHPUnit
pear install phpunit/DbUnit

Depending on what versions of phpunit and pear you have installed or had installed earlier, phpunit may silently break itself. If you get errors about missing files when running tests, uninstall phpunit and pear completely and then reinstall the latest versions of pear and phpunit.

Run Tests

You need to switch into the root directory of the git repository and then run:

  • Linux:
    phpunit
  • Windows:
    YOUR_DIRECTORY_WHERE_THE_FILE_IS/phpunit.bat
    (Exp. C:/xampp/php/phpunit.bat )

Afterwards you should see something like:

PHPUnit 3.4.11 by Sebastian Bergmann.

............................................................  60 / 688
............................................................ 120 / 688
IIIIIIIIIII................................................. 180 / 688
............................................................ 240 / 688
............................................................ 300 / 688
............................................................ 360 / 688
............................................................ 420 / 688
............................................................ 480 / 688
............................................................ 540 / 688
............................................................ 600 / 688
............................................................ 660 / 688
............................

Time: 5 seconds, Memory: 13.00Mb

OK, but incomplete or skipped tests!
Tests: 688, Assertions: 878, Incomplete: 11.

If there are no errors, your test was successful. Otherwise either a test was wrong, or the functionality itself is wrong. Then you should go and fix it.

Writing Unit Tests

tests/demo/demo_test.php

Within unit tests, there are two categories: tests that need a database, and those that do not. If a test does not need a database then don't use it!

Without a Database
<?php
/**
*
* @package testing
* @copyright (c) 2008 phpBB Group
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*
*/

require_once dirname(__FILE__).'/../phpBB/includes/functions.php';

class 
phpbb_demo_demo_test extends phpbb_test_case
{
    public static function 
phpbb_email_hash_data()
    {
        return array(
            array(
'nickvergessen@gmx.de'126830126620),
            array(
''0),
        );
    }

    
/**
    * @dataProvider phpbb_email_hash_data
    */
    
public function test_phpbb_email_hash($email$expected)
    {
        
$this->assertEquals($expectedphpbb_email_hash($email));
    }
}

You may have noted, that in the test-function there is actually no data. There's a dataProvider-function stated in the comment, which is used for the test. The test is then run with every array in the dataProvider array, having the values as parameters for the test-function-call.

Using a Database
<?php
/**
*
* @package testing
* @copyright (c) 2008 phpBB Group
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*
*/

class phpbb_demo_db_demo_test extends phpbb_database_test_case
{
    public function 
getDataSet()
    {
        return 
$this->createXMLDataSet(dirname(__FILE__).'/fixtures/three_users.xml');
    }

    public static function 
fetchrow_data()
    {
        return array(
            array(
'', array(array('username_clean' => 'barfoo'),
                array(
'username_clean' => 'foobar'),
                array(
'username_clean' => 'bertie'))),
            array(
'user_id = 2', array(array('username_clean' => 'foobar'))),
            array(
"username_clean = 'bertie'", array(array('username_clean' => 'bertie'))),
            array(
"username_clean = 'phpBB'", array()),
        );
    }

    
/**
    * @dataProvider fetchrow_data
    */
    
public function test_fetchrow($where$expected)
    {
        
// The function from phpbb_test_case_helpers returns a new db for every test.
        
$db $this->new_dbal();

        
$result $db->sql_query('SELECT username_clean
            FROM phpbb_users
            ' 
. (($where) ? ' WHERE ' $where '') . '
            ORDER BY user_id ASC'
);

        
$ary = array();
        while (
$row $db->sql_fetchrow($result))
        {
            
$ary[] = $row;
        }
        
$db->sql_freeresult($result);

        
$this->assertEquals($expected$ary);
    }
}

Most important to know for db-tests is:

  1. All data from the database is truncated first.
  2. The data from the getDataSet function is loaded into the database. No data from any other test is available!

Code of a DB-DataSet

<?xml version="1.0" encoding="UTF-8" ?>
<dataset>
	<table name="table_name">
		<column>column_name_1</column>
		<column>column_name_2</column>
		<row>
			<value>value for column 1</value>
			<value>value for column 2</value>
		</row>
		<row>
			<value>another row, value for column 1</value>
			<value>another row, value for column 2</value>
		</row>
	</table>
</dataset>

Functional Tests

Functional tests test software the way a user would. They simulate a user browsing the website, but they do these steps in an automated way.

phpBB allows you to write such tests. This document will tell you how.

Writing Functional Tests

Your test case will have to inherit from the phpbb_functional_test_case class. You will be able to make requests to a fresh phpBB installation.

<?php

class phpbb_my_test extends phpbb_functional_test_case
{
    public function 
bootstrap()
    {
        
// setup code
    
}

    public function 
test_index()
    {
        
// test code
    
}
}

There is a bootstrap function where you can put code that is run before the test. Here you can set up some initial state required for your test.

Then you can define your tests, just as you would normally, by prefixing methods with "test_". In this case we will be testing the index.

Goutte and Assertions

The functional testing framework is built using Goutte [1], which makes HTTP requests and allows us to interact with the page. The Goutte "client" is available in your tests through $this->request($method, $path). Alternatively you can also access it directly through $this->client.

When calling request(), you will get a crawler, which allows you to fetch parts of the page using CSS selectors. Here is an example:

class phpbb_my_test extends phpbb_functional_test_case
{
    public function 
test_index()
    {
        
$crawler $this->request('GET''index.php');
        
$this->assertGreaterThan(0$crawler->filter('.topiclist')->count());
    }
}

We perform a GET request to "index.php", then apply a CSS filter, querying for the .topiclist class. Then we make sure our query matched something.

If the board's index page were broken, our test would catch this, allowing us to detect the issue and fix it. The more things you test, the more breakages you will catch early.

For more information what you can do with Goutte, check out the GitHub project, Ryan Weaver's Goutte tutorial and Symfony's testing documentation.

Running Functional Tests

Requirements & Configuration

To run the functional tests, you will need PHP 5.3 or greater.

Running the tests requires your phpBB3 repository to be accessible through a local web server. You will also need to supply the URL to this server in the 'tests/test_config.php' file. This is as simple as defining the '$phpbb_functional_url', which contains the URL for the directory containing the board. Make sure you include the trailing slash.

Note that without extensive changes to the test framework, you cannot use a board outside of the repository on which to run tests.

<?php
$dbms 
'mysqli';
$dbhost 'localhost';
$dbport '';
$dbname 'phpbb_test';
$dbuser 'root';
$dbpasswd '';
$phpbb_functional_url 'http://localhost/phpBB3/';

Testing

To then run the tests, you run PHPUnit, but use the phpunit.xml.functional config file instead of the default one. Specify this through the "-c" option:

phpunit -c phpunit.xml.functional

This will change your board's config.php file, but it makes a backup at config_dev.php, so you can restore it after the test run is complete.