phpBB

Development Wiki

Difference between revisions of "Automated Tests"

From phpBB Development Wiki

(Requirements & Configuration)
m (Update link of phpBB repository)
 
(11 intermediate revisions by 5 users not shown)
Line 8: Line 8:
  
 
=== Running Unit Tests ===
 
=== Running Unit Tests ===
==== Install PHP extensions ====
+
Information on how to run tests is available in the GitHub repository at [https://github.com/phpbb/phpbb/blob/master/tests/RUNNING_TESTS.md Running Tests]. You can switch the branch to check instructions for a specific version of phpBB.
Unit tests use several PHP extensions that phpBB itself does not use. These need to be installed and enabled for unit tests to work.
+
 
+
Unit tests themselves need:
+
 
+
* ctype
+
* simplexml
+
 
+
For database tests you additionally need to have pdo and pdo drivers for each of the databases you want to test with (e.g. pdo_pgsql for postgres).
+
 
+
PHPUnit needs:
+
 
+
* ctype
+
* dom
+
 
+
==== Installing phpunit ====
+
 
+
You need to install PHPUnit to your server: http://www.phpunit.de/
+
In order to install PHPUnit correctly, you need to run the following commands:
+
<pre>pear channel-update pear.php.net
+
pear install PEAR-1.9.1
+
pear channel-discover pear.phpunit.de
+
pear channel-discover components.ez.no
+
pear channel-discover pear.symfony-project.com
+
pear install --alldeps phpunit/PHPUnit
+
pear install phpunit/DbUnit</pre>
+
 
+
Depending on what versions of phpunit and pear you have installed or had installed earlier, phpunit may silently break itself. If you get errors about missing files when running tests, uninstall phpunit and pear completely and then reinstall the latest versions of pear and phpunit.
+
 
+
==== Run Tests ====
+
 
+
You need to switch into the root directory of the git repository and then run:
+
* Linux:<pre>phpunit</pre>
+
* Windows:<pre>YOUR_DIRECTORY_WHERE_THE_FILE_IS/phpunit.bat</pre> (Exp. C:/xampp/php/phpunit.bat )
+
 
+
Afterwards you should see something like:
+
<pre>PHPUnit 3.4.11 by Sebastian Bergmann.
+
 
+
............................................................  60 / 688
+
............................................................ 120 / 688
+
IIIIIIIIIII................................................. 180 / 688
+
............................................................ 240 / 688
+
............................................................ 300 / 688
+
............................................................ 360 / 688
+
............................................................ 420 / 688
+
............................................................ 480 / 688
+
............................................................ 540 / 688
+
............................................................ 600 / 688
+
............................................................ 660 / 688
+
............................
+
 
+
Time: 5 seconds, Memory: 13.00Mb
+
 
+
OK, but incomplete or skipped tests!
+
Tests: 688, Assertions: 878, Incomplete: 11.</pre>
+
If there are no errors, your test was successful. Otherwise either a test was wrong, or the functionality itself is wrong. Then you should go and fix it.
+
  
 
=== Writing Unit Tests ===
 
=== Writing Unit Tests ===
 +
Files that contain tests should have a `_test` suffix, e.g.: tests/demo/demo_test.php
 
==== tests/demo/demo_test.php ====
 
==== tests/demo/demo_test.php ====
 
Within unit tests, there are two categories: tests that need a database, and those that do not. '''If a test does not need a database then don't use it!'''
 
Within unit tests, there are two categories: tests that need a database, and those that do not. '''If a test does not need a database then don't use it!'''
Line 86: Line 32:
 
{
 
{
 
return array(
 
return array(
array('nickvergessen@gmx.de', 126830126620),
+
array('wiki@phpbb.com', 126830126620),
 
array('', 0),
 
array('', 0),
 
);
 
);
Line 161: Line 107:
 
# All data from the database is truncated first.
 
# All data from the database is truncated first.
 
# The data from the getDataSet function is loaded into the database.  '''No data from any other test is available!'''
 
# The data from the getDataSet function is loaded into the database.  '''No data from any other test is available!'''
# If you use a table that has a column which has no default value specified (such as text columns), be sure to specify them. (see [http://tracker.phpbb.com/browse/PHPBB3-10667 PHPBB3-10667])
+
# If you use a table that has a column which has no default value specified (such as text columns), be sure to specify them. (See [http://tracker.phpbb.com/browse/PHPBB3-10667 PHPBB3-10667])
  
 
===== Code of a DB-DataSet =====
 
===== Code of a DB-DataSet =====
Line 184: Line 130:
 
===== Using permissions ($auth) =====
 
===== Using permissions ($auth) =====
 
When you need to use the auth class, you should use a mock object for it. here is a short example of how to use it. Please note, that you need to specify the result for every function and all values you use. Also when the forum_id is not casted to int, you need to specify a second row for each permission. (Marked with '''// Called without int cast''' below)
 
When you need to use the auth class, you should use a mock object for it. here is a short example of how to use it. Please note, that you need to specify the result for every function and all values you use. Also when the forum_id is not casted to int, you need to specify a second row for each permission. (Marked with '''// Called without int cast''' below)
For more information how these test doubles work, see: http://www.phpunit.de/manual/3.6/en/test-doubles.html
+
For more information how these test doubles work, see: [https://phpunit.de/manual/current/en/test-doubles.html Test Doubles]
 
<php>
 
<php>
 
<?php
 
<?php
Line 230: Line 176:
 
=== Running Functional Tests ===
 
=== Running Functional Tests ===
  
==== Requirements & Configuration ====
+
Information on how to run tests is available in the GitHub repository at [https://github.com/phpbb/phpbb/blob/master/tests/RUNNING_TESTS.md Running Tests]. You can switch the branch to check instructions for a specific version of phpBB.
To run the functional tests, you will need PHP 5.3 or greater with the cURL module.
+
 
+
Running the tests requires your phpBB3 repository to be accessible through a local web server. As of PHP 5.4 a builtin webserver is available. If you are on PHP 5.3 you will also need to supply the URL to a webserver of your own in the 'tests/test_config.php' file.  This is as simple as defining the '$phpbb_functional_url', which contains the URL for the directory containing the board. Make sure you include the trailing slash. Testing makes use of a seperate database defined in this config file and before running the tests each time this database is deleted.
+
 
+
Note that without extensive changes to the test framework, '''you cannot use a board outside of the repository on which to run tests'''.
+
 
+
<php>
+
<?php
+
$dbms = 'mysqli';
+
$dbhost = 'localhost';
+
$dbport = '';
+
$dbname = 'phpbb_test';
+
$dbuser = 'root';
+
$dbpasswd = '';
+
$phpbb_functional_url = 'http://localhost/phpBB3/';
+
</php>
+
 
+
On PHP 5.4 you do not need the <code>$phpbb_functional_url</code> parameter but you can configure the port the builtin webserver runs on using
+
<php>
+
$phpbb_functional_port = 8000;
+
</php>
+
 
+
==== Testing ====
+
 
+
To then run the tests, you run PHPUnit, but use the phpunit.xml.functional config file instead of the default one. Specify this through the "-c" option:
+
 
+
<code>
+
phpunit -c phpunit.xml.functional
+
</code>
+
 
+
This will change your board's config.php file, but it makes a backup at config_dev.php, so you can restore it after the test run is complete.
+
 
+
  
 
=== Writing Functional Tests ===
 
=== Writing Functional Tests ===
  
 
Your test case will have to inherit from the phpbb_functional_test_case class. You will be able to make requests to a fresh phpBB installation.
 
Your test case will have to inherit from the phpbb_functional_test_case class. You will be able to make requests to a fresh phpBB installation.
 +
Also be sure, to put the file into the "tests/functional/" folder and to add the <code>@group functional</code> doc block above the class, so the tests are correctly executed.
  
 
<php>
 
<php>
 
<?php
 
<?php
  
class phpbb_my_test extends phpbb_functional_test_case
+
/**
 +
* @group functional
 +
*/
 +
class phpbb_functional_my_test extends phpbb_functional_test_case
 
{
 
{
 
     public function bootstrap()
 
     public function bootstrap()
Line 291: Line 209:
 
==== Goutte and Assertions ====
 
==== Goutte and Assertions ====
  
The functional testing framework is built using Goutte [https://github.com/fabpot/Goutte], which makes HTTP requests and allows us to interact with the page. The Goutte "client" is available in your tests through <code>$this->request($method, $path)</code>. Alternatively you can also access it directly through <code>$this->client</code>.
+
The functional testing framework is built using [https://github.com/fabpot/Goutte Goutte], which makes HTTP requests and allows us to interact with the page. The Goutte "client" is available in your tests through <code>$this->request($method, $path)</code>. Alternatively you can also access it directly through <code>$this->client</code>.
  
 
When calling request(), you will get a crawler, which allows you to fetch parts of the page using CSS selectors. Here is an example:
 
When calling request(), you will get a crawler, which allows you to fetch parts of the page using CSS selectors. Here is an example:

Latest revision as of 11:41, 7 May 2016

To ensure code integrity as new features are added and existing features are altered by patches, phpBB utilizes the PHPUnit test framework. This way, the development team can keep track of whether or not a change made to the core will effect other existing features in unforeseen ways.

There are two main types of tests that can be written and run: unit tests, and functional tests.

Unit Tests

Unit tests simply verify that specific portions of the source code work properly. These tests are located in the main phpBB repository at ./tests/ and are grouped into folders based on commonality between the functionality they are testing. For instance, tests that are meant to test the config will be in ./tests/config/.

Running Unit Tests

Information on how to run tests is available in the GitHub repository at Running Tests. You can switch the branch to check instructions for a specific version of phpBB.

Writing Unit Tests

Files that contain tests should have a `_test` suffix, e.g.: tests/demo/demo_test.php

tests/demo/demo_test.php

Within unit tests, there are two categories: tests that need a database, and those that do not. If a test does not need a database then don't use it!

Without a Database
<?php
/**
*
* @package testing
* @copyright (c) 2008 phpBB Group
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*
*/

require_once dirname(__FILE__).'/../phpBB/includes/functions.php';

class 
phpbb_demo_demo_test extends phpbb_test_case
{
    public static function 
phpbb_email_hash_data()
    {
        return array(
            array(
'wiki@phpbb.com'126830126620),
            array(
''0),
        );
    }

    
/**
    * @dataProvider phpbb_email_hash_data
    */
    
public function test_phpbb_email_hash($email$expected)
    {
        
$this->assertEquals($expectedphpbb_email_hash($email));
    }
}

You may have noted, that in the test-function there is actually no data. There's a dataProvider-function stated in the comment, which is used for the test. The test is then run with every array in the dataProvider array, having the values as parameters for the test-function-call.

Using a Database
<?php
/**
*
* @package testing
* @copyright (c) 2008 phpBB Group
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*
*/

class phpbb_demo_db_demo_test extends phpbb_database_test_case
{
    public function 
getDataSet()
    {
        return 
$this->createXMLDataSet(dirname(__FILE__).'/fixtures/three_users.xml');
    }

    public static function 
fetchrow_data()
    {
        return array(
            array(
'', array(array('username_clean' => 'barfoo'),
                array(
'username_clean' => 'foobar'),
                array(
'username_clean' => 'bertie'))),
            array(
'user_id = 2', array(array('username_clean' => 'foobar'))),
            array(
"username_clean = 'bertie'", array(array('username_clean' => 'bertie'))),
            array(
"username_clean = 'phpBB'", array()),
        );
    }

    
/**
    * @dataProvider fetchrow_data
    */
    
public function test_fetchrow($where$expected)
    {
        
// The function from phpbb_test_case_helpers returns a new db for every test.
        
$db $this->new_dbal();

        
$result $db->sql_query('SELECT username_clean
            FROM phpbb_users
            ' 
. (($where) ? ' WHERE ' $where '') . '
            ORDER BY user_id ASC'
);

        
$ary = array();
        while (
$row $db->sql_fetchrow($result))
        {
            
$ary[] = $row;
        }
        
$db->sql_freeresult($result);

        
$this->assertEquals($expected$ary);
    }
}

Most important to know for db-tests is:

  1. All data from the database is truncated first.
  2. The data from the getDataSet function is loaded into the database. No data from any other test is available!
  3. If you use a table that has a column which has no default value specified (such as text columns), be sure to specify them. (See PHPBB3-10667)
Code of a DB-DataSet
<?xml version="1.0" encoding="UTF-8" ?>
<dataset>
	<table name="table_name">
		<column>column_name_1</column>
		<column>column_name_2</column>
		<row>
			<value>value for column 1</value>
			<value>value for column 2</value>
		</row>
		<row>
			<value>another row, value for column 1</value>
			<value>another row, value for column 2</value>
		</row>
	</table>
</dataset>
Using permissions ($auth)

When you need to use the auth class, you should use a mock object for it. here is a short example of how to use it. Please note, that you need to specify the result for every function and all values you use. Also when the forum_id is not casted to int, you need to specify a second row for each permission. (Marked with // Called without int cast below) For more information how these test doubles work, see: Test Doubles

<?php
/**
*
* @package testing
* @copyright (c) 2012 phpBB Group
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*
*/

require_once dirname(__FILE__).'/../phpBB/includes/auth.php';

class 
phpbb_demo_demo_test extends phpbb_test_case
{
    public function 
test_auth_mock_hash($email$expected)
    
        
$auth $this->getMock('auth');
        
$acl_get_map = array(
            array(
'f_read'23true),
            array(
'f_read''23'true),// Called without int cast
            
array('m_'23true),
            array(
'm_''23'true),// Called without int cast
        
);

        
$auth->expects($this->any())
            ->
method('acl_get')
            ->
with($this->stringContains('_'),
                
$this->anything())
            ->
will($this->returnValueMap($acl_get_map));
        
$this->assertTrue($auth->acl_get('f_read'23));
        
$this->assertTrue($auth->acl_get('f_read''23'));
        
$this->assertFalse($auth->acl_get('f_read'12));
        
$this->assertFalse($auth->acl_get('f_read''12'));
    }
}

Functional Tests

Functional tests test software the way a user would. They simulate a user browsing the website, but they do these steps in an automated way.

phpBB allows you to write such tests. This document will tell you how.

Running Functional Tests

Information on how to run tests is available in the GitHub repository at Running Tests. You can switch the branch to check instructions for a specific version of phpBB.

Writing Functional Tests

Your test case will have to inherit from the phpbb_functional_test_case class. You will be able to make requests to a fresh phpBB installation. Also be sure, to put the file into the "tests/functional/" folder and to add the @group functional doc block above the class, so the tests are correctly executed.

<?php

/**
* @group functional
*/
class phpbb_functional_my_test extends phpbb_functional_test_case
{
    public function 
bootstrap()
    {
        
// setup code
    
}

    public function 
test_index()
    {
        
// test code
    
}
}

There is a bootstrap function where you can put code that is run before the test. Here you can set up some initial state required for your test.

Then you can define your tests, just as you would normally, by prefixing methods with "test_". In this case we will be testing the index.

Goutte and Assertions

The functional testing framework is built using Goutte, which makes HTTP requests and allows us to interact with the page. The Goutte "client" is available in your tests through $this->request($method, $path). Alternatively you can also access it directly through $this->client.

When calling request(), you will get a crawler, which allows you to fetch parts of the page using CSS selectors. Here is an example:

class phpbb_my_test extends phpbb_functional_test_case
{
    public function 
test_index()
    {
        
$crawler $this->request('GET''index.php');
        
$this->assertGreaterThan(0$crawler->filter('.topiclist')->count());
    }
}

We perform a GET request to "index.php", then apply a CSS filter, querying for the .topiclist class. Then we make sure our query matched something.

If the board's index page were broken, our test would catch this, allowing us to detect the issue and fix it. The more things you test, the more breakages you will catch early.

For more information what you can do with Goutte, check out the GitHub project, Ryan Weaver's Goutte tutorial and Symfony's testing documentation.

Authentication

Many functional tests will require the user to be logged in. To do so, you can use the login() method inherited from the phpbb_functional_test_case class. This method will assign the user's SID to the inherited class property $this->sid. You will need to append this to the URLs that the logged-in user will be navigating to in order to hold the session. For usage examples, view the logout test in tests/functional/auth_test.php.

Localization

In tests, as in phpBB itself, do not use hard-coded language unless absolutely necessary. Instead, use the $this->add_lang() and $this->lang() methods. Both act very similarly to their counterparts in the phpbb_user class. Note that language/en/common.php is automatically loaded by the test framework.

function test_example()
{
    
// include the file at ./phpBB/language/en/ucp.php
    
$this->add_lang('ucp');

    
// we can also include multiple ones:
    
$this->add_lang(array('memberlist''mcp'));

    
// Let's use a language key
    
$this->assertEquals('Login'$this->lang('LOGIN'));

    
// And let's use a language key formatted for use with sprintf
    
$this->assertEquals('Logout [ user ]'$this->lang('LOGOUT_USER''user'));
}

For more usage examples, please view ./tests/functional/lang_test.php.