Development Wiki

User:Terrye/A phpBB Use Case Discussion

From phpBB Development Wiki

Analysis by function

Usage by function

The following analysis is based the web requests aggregated across all nine forums available to the OpenOffice User Community (by analysing on week's worth of Apache logs). These patterns are consistent across all ten forums, as well as for the VirtualBox forum which I also help to run and have analysed.

  • As an indication of volumes, we processed just under one million PHP transactions last week on OOo with the breakdown as shown in the piechart.
  • This breakdown is in terms PHP requests, but only the breakout segments on the right are directly due to a user clicking in the a View Topic or similar link.
  • If I focus on the 56% of PHP webpage (HTML) requests and break those out in percentage terms totalling 100, the viewtopic function dominates the work load (77%), with the other main types being viewforum (8.7%), ucp (mostly logons) (5.2%), board index (3.2%), search (3.0%), posting (2.1%), and the rest are crumbs.
  • The other 44% of the URI requests are the embedded requests to load style sheets and images such as avatars and embedded pictures. I haven't included the cache check requests (where the browser cached version still valid), nor any of the other static requests such as the GIF and PNG 'furniture' that generate another couple of million logged the Apache GET requests.
  • The web bots were responsible for over 50% of our viewtopic and viewforum hits. MSN was the biggest offender here generating more requests than all other bots combined, which is ironic as we get almost no references from MSN, but quite a lot from Google. (In the case of the VirtualBox forum the bots generated over two-thirds!). Hence it is essential that you properly configure your robots.txt if you want to minimise the impact on your forum. I would suggest the following as a starting point but increase the Crawler-Delay to 3-5 sec if you are running on a shared Hosting Service.
   Crawl-Delay: 1
   Allow: /index.php
   Allow: /viewforum.php
   Allow: /viewtopic.php
   Disallow: /   
  • In general the bots only download the HTML content from viewtopic and ignore the other webpage furniture. Hence the proportion of furniture requests will increase slightly if your site isn't being widely indexed.
  • Modern browsers do a lot of local caching on the end-user PC. By analysing where interactive requests are not downloading page furniture, I can estimate these local browser cache percentages: for the top avatars (~85%), top images (63%), javascripts (81%), CSS (81%).
  • However I do see multiple phpBB users per IP (typically where these users are logging in via a corporate or ISP proxy), and from an analysis of posts which are identified by both userID and IP, I can get an estimate of how frequent multiple users per IP is. What I found really surprised me: 75% of posts were made from shared IPs in the last 10 days. These proxies also cache static files and therefore skew these stats. If the majority of your user population is home users that I suspect that your percentage of proxy hits will be smaller.

Use Cases

Any performance tuning analysis is best grounded in a hard target system to provide the instrumentation data. My analysis above was for forums running on a dedicated server in the case of OOo and on a VM in the case of VirtualBox. In the case of a product such as phpBB, we need to identify typical use cases and I focus on three: a shared web service, a hosted virtual service (also known as a virtual private server) and a dedicated hosting service. Note that I've used Linux terminology in the following descriptions, but the equivalent MS stacks based on W2003/2008 and IIS are also commonly available. I also use the description administrator in the context of the person who buys the service from the HSP and runs the phpBB service, to differentiate from the end-users who will ultimate access it through the Internet.

In these cases I often use the term contention ratio, and what I mean in this context is the number of users or services which share the resource. If you are running your service on a dedicated server then you full control over the entire machine's resources. If you are running your service on a shared web service along with 400 other users then when an end-user wants to access your phpBB instance, the phpBB software has to contend for machine resources such disk, memory and I/O operations with 399 other applications. OK 300 might be doing virtually nothing, but if you are unlucky then 20 or so might be using the server heavily. The higher the contention ratio, the worse this competition gets.

Shared Web Service (SWS)

Typical price per annum $50-250
Service Description Many users are co-located on a LAMP server farm in some Internet-connected datacentre with a private file hierarchy served through a shared Apache2 instance and the ability to run PHP5 scripts and use a private MySQL database. Mail services and SMTP access are also typically provided. This type of service is the minimum entry service capable of hosting a phpBB installation.
Resources The service provider might map 100-400 such accounts / services onto a single rack-mounted quad server, though some of the larger service providers load balance individual user accounts across a farm of servers. The administrator's domain is DNS mapped onto a shared IP served by the server, with its Apache configured to map the user's domain onto the users account and file hierarchy. One of two basic templates is usually used: either the account are allocated to individual servers using a local (in-server) D/B and disk storage, or a tiered template where user storage is moved on Fibre/Gbit switch Enet connected shared NAS storage with NFS/NTFS access and separate dedicated D/B servers.
Access Models FTP access to personal file hierarchy, phpMyAdmin access to MySQL, adhoc access through PHP and CGI Perl scripts, able to configure .htaccess. No SSH access, cron or batch.
Performance Constraints  These types of shared services rarely employ cached based PHP Accelerator technology, though use of the Zend platform and optimiser is quite common. With such contention ratios, it is likely that a lot of program and data fetches will have aged out of memory caches and therefore generate cache misses and hard (physical) I/Os; for both file and database stored information.
Typical Page Latency The high percentage of cache misses causes a knock-on increase in aggregate I/O latencies (together with the consequential rescheduling and context switches). These delays will often dominate end-to-end page processing times and therefore page processing times can be multiple seconds
Typical Volumetrics The aggregate is rarely in excess of a few page views per minute (which can still build up to 10K per day).

Hosted Virtual Service (HVS)

Typical price per annum Typical costs range from $250-$1000 per annum.
Service Description The service provider offers each administrator a personal VM instance, typically running a minimal LAMP configuration under an Enterprise VM solution such as Virtuozo, Xen Enterprise or VMware ESX. Hence the Apache2 instance, the PHP engine and MySQL instance all fall within this VM footprint.
Resources The key service differentiators are the guaranteed RAM available to the guest VM, the contention ratio (the number of VMs per host server) and resources such as disk capacity and Internet capacity per month. An entry level service might guarantee 200Mb RAM and have a contention ratio of 40:1 with the top end having say 1Gb RAM and a dedicated core. All of these VMs will typically share access to a SATA RAID-1 or RAID-5 array locally connected to the host hardware. It is also common allocate a fixed publicly-registered IP address to each the VM.
Access Models The administrator is given root access to the VM typically through SFTP and SSH, as well as normal FTP and web access to a management console. The administrator has full configuration control over the guest OS subject to personal administration skills and service provider “reasonable use” guidelines.
Performance Constraints  Given the committed memory constraints it is still possible to configure a phpBB instance where the code is fully cached using a memory accelerator such as Xcache or APC as well as the phpBB cached data, and still have enough memory allocated to MySQL to guarantee all the board hot indexes are fully cached for a medium sized forum (say <100K posts) in a 256Kb LAMP VM. Any data or page faults requiring physical I/O will still represent a major performance risk as the disk I/O capacity is share across all VMs
Typical Page Latency ~ 1 sec for a tuned VM.
Typical Volumetrics 60 web pages / min for a tuned VM. 5 web pages /min for an “out-of-the-box” phpBB installation on a default LAMP configuration

Dedicated Hosting Service (DHS)

Typical price per annum Around $1000 per annum for an entry level system, but anything up to $10K per annum depending on server specification.
Service Description Here you have a dedicated server with full control over its configuration and use. For entry level dedicated servers, the administrator has full control of the machine and its I/O resources to configure it largely as desired. Service providers typically have a number of standard system templates base on a number of LAMP configurations and Windows (2003 or 2008). Most service providers also provide console services to access backup and other shared services, and to provision the server at service start-up and in case of disaster.
Resources One might say: similar to a VM but more of it and without the contention ratio. At a typical entry level, the service might comprise a dedicated low power dual core with 512Mb RAM and a single dedicated SATA, with a shared backup service. Adding extra cores, memory, I/O capacity, shared SAN access, hardware fail-over can all increase the capacity of a shared service at a commensurate increase in cost. There are also federated services using “cloud computing” models, though these aren't relevant to the vast majority of uses of phpBB. The key service differentiator is the guaranteed machine resource RAM and dedicated I/O, with none of the contention risks of a HVS approach.
Access Models Largely the same as HVS.
Performance Constraints  Given the increased resources compared to a typical HVS it is straightforward to configure a phpBB instance where the code is fully cached using a memory accelerator such as Xcache or APC. Enough memory can be allocated to MySQL to guarantee all the board indexes are fully cached for a large sized forum as well as data cache large enough to largely remove database cache misses, and when they occur the dedicate I/O capacity means that the server can have predictable performance.
Typical Page Latency <1 sec unless the board is overloaded.
Typical Volumetrics 200-400 web pages / min for a tuned system. 20-50 web pages /min for an "out-of-the-box" phpBB installation on a default LAMP configuration

A Comparison Discussion

The differences between the HVS and DHS are largely a question of degree. The main differences between configurations tuned to these two cases are really limited to the Apache MPM settings and the MySQL my.cnf settings to ensure that the database can fully make use of the extra memory resources to improve internal cache hit ratios and limit physical disk I/O. Except for these two factors, the phpBB tuned configurations are quite similar and the overall issues are the same. On the other hand, the SWS installation and scenario is very different:

  • You have no control over the Apache (or IIS configuration)
  • You have little control over the PHP engine or its tuning. The PHP configuration will largely by what is defined in the system php.ini file and build as shown in a phpinfo() report. It unlikely that any form of code or variable caching will be available to you.
  • Because of the high administrator contention ratios and relatively infrequent page loads, the file system cache hit rate for all the files in a first page load will typically be low and therefore require a lot of physical I/Os to retrieve the data from disk (though the hit rate might improve for subsequent similar requests as long as the inter-page rate is less than a minute or so). For example, executing a single viewtopic.php requires loading and compiling 650Kb of PHP source from 26 separate files, as well a dozen or more data files from the cache directory. The main latency here isn't from compiling the 650Kb of source; it is all the HDD seeks needed at a worst case to read those 50 plus inodes and 200 odd data blocks. Worse, each separate file open as well as each read will cause a process reschedule / context switch (unless fully cached) before the next I/Os can be scheduled. If the shared service is on a busy server with active contention this could take tens of seconds.
  • The situation for the database functions is similar because the MySQL servers on such systems also have a high contention ratio and might be serving thousands of tables across hundreds of databases. In this scenario, the likelihood of good caching characteristics is low, so SQL queries will also tend to generate a lot of physical I/Os.

This situation is fundamentally different even for an entry level HVS with a 512Kb LAMP, as this is sufficient to install a PHP accelerator such as Xcache or APC.

  • With a standard phpBB application, a 8Mb code cache is sufficient to get a 99.5% hit rate on PHP code. (See Figure 1 for my background analysis). More importantly (if you have stat checking turned off), this generates zero I/O. A 4Mb data cache is sufficient to cache all stored variables enabling the use of acm_xcache or acm_apc to remove the physical I/O associated with stored variables that would otherwise be stored in the cache directory.
  • In other words a HVS with PHP code and data caching enabled will effectively remove the bulk physical I/Os which cause process rescheduling. The main delay now becomes the execution of that 20K lines or so of code needed to render the webpage, plus any the delays in executing the uncached SQL queries.
  • The VM's file system caches that are dedicated to your application mean that most writes are write-though and non-blocking and therefore do not impact on throughput.
  • A 512Mb LAMP would probably be best configured with an MPM worker setup because of memory footprint. Despite the caveats in the Apache guidelines, this is fine for a single application LAMP setup.
  • A 512Mb LAMP has enough memory to allow a my.cnf that enables the indexes for a typical medium BB (say less then 50K posts) to be fully cached. Hot data rows will also tend to get cached so I/O will be limited to row retrieval only.

Hence the I/O rates for a SWS will typically be a order of magnitude greater than for a tuned entry level HVS at comparable request volumes, and the contention ratio for other services hosted on the server will be greater. This is why an SWS can typically take an order of magnitude or more longer to process a phpBB page request and why its throughput is accordingly lower.

An HVS RAM of 512Mb or more allows some optimisation of the MySQL database configuration to improve the effectiveness of MySQL caching for all but the largest forums. With high cache rates, the effect of the HDD as a single resource constraint falls considerably and the true concurrency of the web service increases. Sustained throughputs of one page per second can be considered.

Main Article: I/O Performance Impacts on phpBB installations

Notwithstanding this, a shared web service is an entirely practical option for a board which averages, say, one page view per minute or less. OK, so pages may be slightly slow to load, but this won't cause queuing problems at this request rate. If you use volumes rise much above this then you need to start considering a move to a hosted virtual service.