d(a)emonkeeper's purgatory

From time to time you find yourself discovering a piece of software you never heard before but you immediately realize you certainly missed something like that up to that point. Apache Traffic Server is such a piece of software, a comparable HTTP proxy server which virtually appeared from nowhere. Read on to learn more on Apache Traffic Server and how to configure it on Debian Linux and why it is way better than both, Squid and Varnish.

What is Apache Traffic Server?

Traffic Server is a web cache, similar to Squid and Varnish. While it is possible and reasonable to run Trafficserver as classical forwarding proxy, where it really shines is as a reverse proxy. A reverse proxy acts as transparent cache in front of a heavy loaded web site to minimize site load of the actual web servers by serving content from a data cache whenever possible. That’s a transparent operation to the visitor of a site as the server’s response is returned as if it came directly from the origin server.

Since the proxy does not include any intelligent operation logic the entire cache content can easily be replicated to further backup locations and served from there. Thus, you will greatly reduce load of your web- and database servers. Depending on the popularity of the website it could be well possible that a single web cache is still not enough to stand the work load, hence one eventually wants to run a cluster of web proxies in front of the origin servers. This is what people typically really mean when they are talking about Content Delivery Networks.

In fact, that is where Traffic Server’s roots lie in. Traffic Server may be new relatively in the free software ecosystem, but it is not new software written from scratch. Traffic Server has a long and successful history as a commercial proxy server which originally was developed at Inktomi and was eventually acquired by Yahoo in 2002. At Yahoo, Traffic Server serves more than 30 Billion objects and over 400 TiB of traffic per day on no more than 150 commodity servers, as they stated themselves.

In 2009, Yahoo donated Traffic Server to the Apache Software Foundation where it soon became a top level project, amongst best known Apache projects such as the HTTP server and Subversion. That also implied to liberate Traffic Server from its proprietary roots and making it free software available under the Apache 2.0 license.

These are a few good reasons to use Traffic Server:

• It fully supports and understands HTTP
• It is scalable: run it as single reverse proxy, in cache hierarchies or as cluster
• It works well on modern multi-core CPUs
• It supports a cluster mode which automatically synchronizes configuration among all nodes
• It is well suited to run Content Delivery Networks: You can easily configure multiple origins with different caching settings and even partition your disk cache.
• It is highly flexible and extendable.

Installing Traffic Server

Traffic Server is available in Debian Wheezy and later. Ubuntu Oneiric will soon be the first distribution for end users shipping Debian’s Traffic Server package. I’ve prepared a Debian Squeeze backport of the latest 3.0.1-2 release you can use as well. The download links can be found on the bottom of this page.

If you downloaded the packages from this page, or if you made a backport yourself, install the package with "dpkg -i trafficserver_3.0.1-2~wavecon+1_amd64.deb". Eventually you need then to do a apt-get -f install to satisfy any dependency the package may have. If your distribution already includes Traffic Server, just do “apt-get install trafficserver“.

It’s crucial to understand Traffic Server’s architecture to operate with it. Traffic Server consists of three daemon processes which do all work together to serve requests.

• The traffic_server is the working horse. It handles incoming connections, does the caching, pass-through fetching and so on.
• The traffic_manager is the command and control instance. It launches and monitors the traffic_server process and interacts with other nodes when running in cluster mode. Moreover, it’s the service which is public facing and accepts connections handled by traffic_server later.
• The traffic_cop is a watchdog which makes sure that both, traffic_server and traffic_manager are actually running and restarts both if necessary.

While it is possible, it is not recommended to run traffic_server all alone. You better start traffic_cop only and let it do the rest.

Configuring Traffic Server

As I outlined above, Traffic Server has a long history as a proprietary program. You still encounter this heritage in some places. For example it has a strange understanding how to deal with certain situations. For example, its configuration files don’t look like anything you may have seen so far and tend to be overwritten by Traffic Server occasionally. Moreover, don’t even try to leave comments in these files. You will find configuration files in /etc/trafficserver. It has many of them, but you will only need to look in a few to understand the basics:

records.config

The main configuration file. This is where you configure core settings like global configuration, socket setup and certain features you want to enable. You can use the traffic_line command to change settings here.

remap.config

Configure mappings here. In reverse proxy mode, mappings link requests made to the web cache with the actual origin server providing the data. This is where you need to tell Traffic Server about your actual web servers.

cache.config

You don’t necessarily need to edit this file, but it may come handy to you if you find yourself required to override the client’s browser or the origin web server in certain cases. For example you can tweak caching parameters here if you want to force objects to be cached (or not) depending on certain criterias like destination domains or certain URLs.

storage.config

Configure the data store. This is, where you configure the location and type of your data storage which holds cached objects.

Everything beyond those four files is not needed to get started with the Traffic Server.

In records.config, the main configuration file, you need to configure the most basic setup first. You can either manually edit the configuration file or use the traffic_line command. This configuration file consists of a long list of settings which do look like this:

CONFIG proxy.config.proxy_name STRING wv-tmp2

Please don’t remove settings where you think the default is ok. Traffic Server does not really like minimal configuration files. A setting consists of one entry per line, starting with the token “CONFIG“. It is followed by the variable name, the type of the variable and its value you can change. Have a look to the official documentation to learn about the meaning of Traffic Server configuration settings. If you are familiar with Squid, you can find a mapping of configuration directives in the Wiki.

If you want to use the traffic_line command, make sure Traffic Server is actually running. It does not touch the configuration files directly, but communicates with Traffic Server through its management API. On Debian, you can use the init.d script to start Traffic Server. To do so, please enable to start traffic_cop in your /etc/default/trafficserver file:

TC_START=yes

Afterwards you can simply do “/etc/init.d/trafficserver start“. As I explained above, this will automatically start traffic_server and traffic_manager. Hence you will find a corresponding message in your syslog (note that traffic_server itself is logging to /var/log/trafficserver/traffic.out by default):

Oct  3 13:26:26 wv-tmp2 traffic_cop[1604]: --- Cop Starting [Version: Apache Traffic Server - traffic_cop - 3.0.1 - (build # 7319 on Aug 31 2011 at 09:31:44)] ---
Oct  3 13:26:26 wv-tmp2 traffic_cop[1604]: traffic_manager not running, making sure traffic_server is dead
Oct  3 13:26:26 wv-tmp2 traffic_cop[1604]: spawning traffic_manager
Oct  3 13:26:26 wv-tmp2 traffic_manager[1609]: NOTE: --- Manager Starting ---
Oct  3 13:26:26 wv-tmp2 traffic_manager[1609]: NOTE: Manager Version: Apache Traffic Server - traffic_manager - 3.0.1 - (build # 7319 on Aug 31 2011 at 09:28:05)

And in /var/log/trafficserver/traffic.out:

[E. Mgmt] log ==> [TrafficManager] using root directory '/usr'
[Oct  2 16:46:26.683] Manager {140228948637472} NOTE: [LocalManager::listenForProxy] Listening on port: 8080
[Oct  2 16:46:26.683] Manager {140228948637472} NOTE: [TrafficManager] Setup complete
[Oct  2 16:46:27.699] Manager {140228948637472} NOTE: [LocalManager::startProxy] Launching ts process
[TrafficServer] using root directory '/usr'
[Oct  2 16:46:27.711] Manager {140228948637472} NOTE: [LocalManager::pollMgmtProcessServer] New process connecting fd '11'
[Oct  2 16:46:27.711] Manager {140228948637472} NOTE: [Alarms::signalAlarm] Server Process born
[Oct  2 16:46:28.722] {47262945265984} STATUS: opened /var/log/trafficserver/diags.log
[Oct  2 16:46:28.722] {47262945265984} NOTE: updated diags config
[Oct  2 16:46:28.727] Server {47262945265984} NOTE: cache clustering disabled
[Oct  2 16:46:28.727] Server {47262945265984} NOTE: clearing statistics
[Oct  2 16:46:28.737] Server {47262945265984} WARNING: configuration changed: [hostdb.config] : reinitializing database
[Oct  2 16:46:28.737] Server {47262945265984} NOTE: reconfiguring host database
[Oct  2 16:46:28.737] Server {47262945265984} NOTE: file '/var/cache/trafficserver/host.db' size changed from 0.00MB to 30.91MB
[Oct  2 16:46:28.780] Server {47262945265984} NOTE: cache clustering disabled
[Oct  2 16:46:28.792] Server {47262960551680} WARNING: disk header different for disk /var/cache/trafficserver/cache.db: clearing the disk
[Oct  2 16:46:28.793] Server {47262959499008} NOTE: Clearing Disk: /var/cache/trafficserver/cache.db
[Oct  2 16:46:28.803] Server {47262959499008} NOTE: clearing cache directory '/var/cache/trafficserver/cache.db 16384:24575'
[Oct  2 16:46:28.823] Server {47262945265984} NOTE: logging initialized[7], logging_mode = 3
[Oct  2 16:46:28.833] Server {47262945265984} NOTE: traffic server running
[Oct  2 16:46:28.873] Server {47262959499008} NOTE: cache enabled

Configuring the Server

Now, as the server is running you can configure it by using the traffic_line utility:

traffic_line -s proxy.config.proxy_name -v wv-tmp2
traffic_line -s proxy.config.http.server_port -v 80
traffic_line -s proxy.config.admin.number_config_bak -v 0
traffic_line -s proxy.config.cache.max_doc_size -v 0
traffic_line -s proxy.config.http.insert_response_via_str -v 1
traffic_line -s proxy.config.http.verbose_via_str -v 2

Possibly you may want to change some settings. By default, Traffic Server listens on port 8080 for incoming connections. As reverse proxy it makes sense to have it listening on port 80 (HTTP) though. Hence I changed proxy.config.http.server_port to reflect that. Also I do found I annoying that Traffic Server keeps backups of rolled out configuration files before overwriting them. Thus I disabled that as well. Not only, but in particular for debugging it is very helpful to send out the cache status in the HTTP response header. You need to enable proxy.config.http.insert_response_via_str to get Traffic Server to write a HTTP “Via” header. Eventually it is also helpful to be verbose in this header if you want cache status information in it. Hence make sure you set proxy.config.http.verbose_via_str to 2. I am going to explain the caching below based on this response header. Also, config.cache.max_doc_size comes handy if you aren’t sure about the size of your objects, or you want to make sure you cache every document. Set it to 0 to cache every document, or to any positive integer indicating the maximum file size in bytes.

In my introduction I mentioned that Traffic Server fully understands the HTTP protocol. That’s a good thing you may think – but sometimes it is not. For example, the web is full of crappy HTTP client and server implementations, many being RFC ignorant and thus not being legit peers, technically speaking. Also, some site administrators configure their web servers to send out wrong or inappropriate HTTP headers or both. Thus, you may need to configure Traffic Server to be a bit ignorant as well.

Out of the box, Traffic Server is only caching requests which are being marked as cacheable explicitly – for example by having respective cache friendly headers. Some servers do not send them out correctly, so Traffic Server’s “when to cache” heuristic may not work very well. Possibly you therefore want to make sure such objects are being cached, despite of no clear evidence that the object is cacheable:

traffic_line -s proxy.config.http.cache.when_to_revalidate  -v 4
traffic_line -s proxy.config.http.cache.required_headers -v 0

This may also include setting “proxy.config.http.cache.cache_urls_that_look_dynamic” to 1 which does pretty much what the same suggests. Finally you need to configure Traffic Server to enable reverse proxy operations. To do so, tweak the following settings:

1
2
3
4

traffic_line -s proxy.config.reverse_proxy.enabled -v 1
traffic_line -s proxy.config.url_remap.remap_required -v 1
traffic_line -s proxy.config.header.parse.no_host_url_redirect -v http://example.com
traffic_line -s proxy.config.url_remap.pristine_host_hdr -v 0

In line 1 the reverse proxy operation is actually enabled. Line 2 forces a mapping to an origin server. This means: Traffic Server will reject to forward requests which could not be translated to an origin server explicitly. You may want to use this option to make sure, nobody can abuse your proxy for redirections you didn’t explicitly allow. The next line defines the mapping of last resort: If Traffic Server couldn’t translate any mapping into an origin host you configured, the client is being redirected to the URL given in there.

Whether you want to set proxy.config.url_remap.pristine_host_hdr depends on your personal scenario. Setting it to 1 causes Traffic Server to keep the HTTP 1.1 “Host” header the client sent along his request as is. If you have name based virtual hosts on your origin web server you need to make sure it still matches the name being sent in this header then.

Consider you have a website running on “mywebsite.example.com“. Now you decide to setup a reverse proxy at “proxy.example.com” which is supposed to cache requests for “mywebsite.example.com“. To route visitors through your proxy, you are updating the URLs to actually make use of it. Once you did, your client’s browser will then start sending “proxy.example.com” as HTTP “Host” header. If you set proxy.config.url_remap.pristine_host_hdr to 0, Traffic Server will leave this header untouched and forward the request as is to “mywebsite.example.com“. If you configured a virtual host on that web server which matches the domain name, it fails to find the appropriate host, because the request host was “proxy.example.com” not “mywebsite.example.com” . However setting proxy.config.url_remap.pristine_host_hdr to 1 yields Traffic Server to rewrite the Host header to the name you gave in the destination mapping – “mywebsite.example.com” in my example.

Configuring Server Mappings and the Cache

Now, as the server configuration is complete you may want to configure actual mapping and disk storage. First open the remap.config file:

map http://internal.proxy.server.com http://{cache} @action=allow @src_ip=127.0.0.1
map http://your.proxy.server.com http://daemonkeeper.net
#regex_map http://([a-z\-0-9\.]+) http://daemonkeeper.com?request=$1

As the name indicates, this file consists of mappings. If a client issues a new request for a document to your proxy server, it does so by sending “your.proxy.server.com” as host header. Traffic Server will translate this to a request to “http://daemonkeeper.net” as origin. This means, requests directed to “your.proxy.server.com” will be redirected to “daemonkeeper.net” by automatically caching objects found to be cacheable on its way. For successive requests, the proxy will fetch further objects if necessary, or deliver them from the own local cache instead.
In line 3 I give an example for a wild card rule which makes use of regular expressions to redirect requests to an origin when they match a certain pattern.

The first line enables a mapping to the caching server itself. This is useful if you want to access the web user interface to inspect the cache contents. This interface is built into Traffic Server. Note the filter arguments which can be added after a mapping rule. Those can restrict access to a mapping rule. In my case I am restricting access to the web interface to the local host itself. Note, if you want to access the web interface you need to enable it in addition with “traffic_line -s proxy.config.http_ui_enabled -v 1“. Filters can also be used more generic: If you want, you can also use filters like “@method=GET” which will restrict Traffic Server to forward HTTP GET requests only to a destination.

Forward mappings (“map“) like those I was explaining above are only one of five different possibilities to describe mapping rules. Others are map_with_referer, reverse_map, redirect, and redirect_temporary. Refer to the respective documentation to learn more on these alternatives.

Finally you need to configure the actual cache. Traffic Server comes out of the box with a 256 MiB cache stored on your local file system. You may want to change that to something which matches your site’s requirements. In storage.config set

/var/cache/trafficserver 1024M # 1 GiB

On a production server you may want to use raw disk access. Just give a block device, e.g. /dev/sdb1 or even /dev/sdb and Traffic Server will happily use it as storage. Be sure you don’t overwrite your operating system when starting Traffic Server with a raw disk device in its storage.config!

Once you finished configuring, you need to tell Traffic Server to flush its running configuration and reread the new configuration from the disk. You can achieve this by a simple command:

$ traffic_line -x

In your log you should be able to see something like this then:

[Oct  3 15:57:04.954] Manager {139887277762304} NOTE: User has changed config file cache.config
[Oct  3 15:57:04.955] Manager {139887277762304} NOTE: [fileUpdated] storage.config changed, need restart auto-rebuild mode
[Oct  3 15:57:04.955] Manager {139887277762304} NOTE: User has changed config file storage.config

Please note: Currently traffic_line -x does not warn you if a given change you made could not be applied on the running instance. Some settings require a server restart until they become effective. Changing the listening port is one of them, changing the data store another. Hence you need to use /etc/init.d/trafficserver restart instead.

Testing and Benchmarking the cache

As you can see above, I configured Traffic Server as reverse proxy for my site. Thus one can transparently use it to access my site. For example I uploaded an image gallery a few years ago. That’s content which is perfectly cacheable (even though my server does not make it easy to cache content). You can access those images directly:

$ GET -Ued http://daemonkeeper.net/wp-content/myfotos/ddos/ddos2.png
GET http://daemonkeeper.net/wp-content/myfotos/ddos/ddos2.png
User-Agent: lwp-request/5.834 libwww-perl/5.837
 
Connection: close
Date: Mon, 03 Oct 2011 16:03:11 GMT
Accept-Ranges: bytes
ETag: "1b40ee-3222c-499ec43764a80"
Server: Apache/2.2.16 (Debian) DAV/2 SVN/1.6.12 PHP/5.3.3-7+squeeze3 with Suhosin-Patch mod_ssl/2.2.16 OpenSSL/0.9.8o
Content-Length: 205356
Content-Type: image/png
Last-Modified: Sun, 16 Jan 2011 01:05:30 GMT
Client-Date: Mon, 03 Oct 2011 16:01:55 GMT
Client-Peer: 81.95.6.56:80
Client-Response-Num: 1

Now, I can try to access the same URL through the reverse proxy:

$ GET -Ued http://your.proxy.server.com/wp-content/myfotos/ddos/ddos2.png
GET http://your.proxy.server.com/wp-content/myfotos/ddos/ddos2.png
User-Agent: lwp-request/5.834 libwww-perl/5.837
 
Connection: close
Date: Sun, 02 Oct 2011 20:27:34 GMT
Via: http/1.1 wv-tmp2 (ApacheTrafficServer/3.0.1 [uScMsSfWpSeN:t cCMi p sS])
Accept-Ranges: bytes
Age: 0
ETag: "1b40ee-3222c-499ec43764a80"
Server: ATS/3.0.1
Content-Length: 205356
Content-Type: image/png
Last-Modified: Sun, 16 Jan 2011 01:05:30 GMT
Client-Date: Sun, 02 Oct 2011 20:26:18 GMT
Client-Peer: 82.199.139.49
Client-Response-Num: 1

Seems to work. It’s the Via: header which is interesting. This is what I enabled by setting proxy.config.http.insert_response_via_str above. The cryptic flags in the header value give some precious information on the sate of the request. The most important flag is “cM” which means “cache Miss”. Moreover “sS” means “server Served” and “fW” “cache-fill Written into the cache”. That basically means for the particular request above: The request could not be satisfied by the cache, however the proxy server received a valid response from its origin which, in turn, was added to the proxy cache. This is pretty much what one can expect for the very first request to an object. You can learn more on the meaning of the remaining fields by consulting the documentation.

The request above implied the object should be cached now. So let’s try again:

$ GET -Ued http://your.proxy.server.com/wp-content/myfotos/ddos/ddos2.png
GET http://your.proxy.server.com/wp-content/myfotos/ddos/ddos2.png
User-Agent: lwp-request/5.834 libwww-perl/5.837
 
Connection: close
Date: Sun, 02 Oct 2011 20:27:34 GMT
Via: http/1.1 wv-tmp2 (ApacheTrafficServer/3.0.1 [uScHs f p eN:t cCHi p s ])
Accept-Ranges: bytes
Age: 7
ETag: "1b40ee-3222c-499ec43764a80"
Server: ATS/3.0.1
Content-Length: 205356
Content-Type: image/png
Last-Modified: Sun, 16 Jan 2011 01:05:30 GMT
Client-Date: Sun, 02 Oct 2011 20:26:24 GMT
Client-Peer: 82.199.139.49:81
Client-Response-Num: 1

The result is pretty much the same, except of one very important change in the headers: The “cM” from above turned into “cH” now. “cH” stands for “cache Hit”. Good!

Looking into the logs proves what the response header told (use the traffic_logcat tool to read the binary logs):

$ traffic_logcat -f /var/log/trafficserver/squid.blog
1317587011.452 40 92.229.59.124 TCP_MISS/200 205689 GET http://daemonkeeper.net/wp-content/myfotos/ddos/ddos2.png - DIRECT/daemonkeeper.net image/png -
1317587018.015 0 92.229.59.124 TCP_HIT/200 205689 GET http://daemonkeeper.net/wp-content/myfotos/ddos/ddos2.png - NONE/- image/png -

If you ever happen to find yourself in a situation where you need to purge the entire cache, you can always do this easily by using the traffic_server command by the way:

$ traffic_server -Cclear
[TrafficServer] using root directory '/usr'
[Oct  2 20:25:53.504] {140099960268576} STATUS: opened /var/log/trafficserver/diags.log
[Oct  2 20:25:53.504] {140099960268576} NOTE: updated diags config
[Oct  2 20:25:53.506] Server {140099960268576} NOTE: cache clustering disabled
[Oct  2 20:25:53.514] Server {140099960268576} NOTE: CLEAR
[Oct  2 20:25:53.514] Server {140099960268576} NOTE: Clearing Configuration
[Oct  2 20:25:53.518] Server {140099960268576} NOTE: Clearing Host Database
[Oct  2 20:25:53.548] Server {140099960268576} NOTE: Clearing Cache
[Oct  2 20:25:53.555] Server {140099893581568} NOTE: Clearing Disk: /var/cache/trafficserver/cache.db
[Oct  2 20:25:53.559] Server {140099893581568} NOTE: clearing cache directory '/var/cache/trafficserver/cache.db 16384:122879'
[Oct  2 20:25:53.564] Server {140099893581568} NOTE: cache enabled
[Oct  2 20:25:54.525] Server {140099894634240} NOTE: CLEAR, succeeded

Benchmarking Traffic Server, Varnish and Squid

To compare performance of these three web proxies I created three test files. The files were 1 KiB, 5 KiB and 10 KiB large and I put them onto my server:

dd if=/dev/urandom of=1K bs=1K count=1
dd if=/dev/urandom of=5K bs=1K count=5
dd if=/dev/urandom of=10K bs=1K count=10

Next I used http_load to benchmark server performance. The servers were idle when not running benchmarks and all were running on the same virtual machine but on different ports. Traffic Server on port 80, Varnish on port 81 and Squid on port 82. Please note, I didn’t make use of any particular tuning options for either one daemon. The client running the benchmark was connected through physical Gb NICs through an enterprise class switch (both client and proxy running in paravirtualized virtual machines though). The proxy server runs in a Xen vritual machine with two cores of a Intel Xeon E5530 @ 2.40GHz CPU available and 2 GiB memory dedicated to the machine. There were no other running services on the machine, besides of SSH.

Configuration files for Varnish and Squid can be found below, all I changed from the default setup was to configure them as reverse proxy to the same origin as Traffic Server above.

  squid-varnish-confs.tar.gz (3.6 KiB, 501 hits)

The file set to test were for each server the same. I tested random HTTP requests to http://your.proxy.server.com/wp-content/files/{1K,5K,10K}” for 60 seconds with 20 connections in parallel. That’s not really something which should produce any significant load on modern hardware. These are the results:

http_load -parallel 20 -seconds 60 {ats,varnish,squid}_files

Traffic Server:

549261 fetches, 20 max parallel, 2.99858e+09 bytes, in 60.0001 seconds
5459.31 mean bytes/connection
9154.34 fetches/sec, 4.99763e+07 bytes/sec
msecs/connect: 0.406156 mean, 2999.32 max, 0.094 min
msecs/first-response: 1.5772 mean, 1076.07 max, 0.204 min
HTTP response codes:
  code 200 -- 549261

Varnish:

519935 fetches, 20 max parallel, 2.8453e+09 bytes, in 60.0001 seconds
5472.42 mean bytes/connection
8665.56 fetches/sec, 4.74216e+07 bytes/sec
msecs/connect: 0.398676 mean, 2998.62 max, 0.1 min
msecs/first-response: 1.73151 mean, 14.395 max, 0.28 min
HTTP response codes:
  code 200 -- 519935

Squid:

342467 fetches, 20 max parallel, 1.86764e+09 bytes, in 60 seconds
5453.5 mean bytes/connection
5707.78 fetches/sec, 3.11274e+07 bytes/sec
msecs/connect: 0.299169 mean, 1.745 max, 0.086 min
msecs/first-response: 2.14572 mean, 3738.78 max, 0.265 min
HTTP response codes:
  code 200 -- 342467

You can clearly see, all servers pass the test, but Traffic Server delivers about 30,000 files more than Varnish in the same time and about 200,000(!) more than Squid. Also its very interesting to see, that Traffic Server, despite of delivering much more files, still manages to keep the mean response times lower than every other cache I tested.

Next I tested the same scenario again, but this time using massive parallel connections. I set the concurrency level to 1000 parallel connections to the server. This apparently has been too much for Varnish and Squid. At least they started to deliver broken data or refused to talk at all to me:

http_load -parallel 1000 -seconds 60 {ats,varnish,squid}_files

Traffic Server:

904849 fetches, 1000 max parallel, 4.93673e+09 bytes, in 60.0002 seconds
5455.86 mean bytes/connection
15080.8 fetches/sec, 8.22785e+07 bytes/sec
msecs/connect: 4.22969 mean, 3009.6 max, 0.102 min
msecs/first-response: 10.9157 mean, 6076.57 max, 0.305 min
HTTP response codes:
  code 200 -- 904849

Varnish:

http://your.proxy.server.com:82/wp-content/files/10K: byte count wrong
http://your.proxy.server.com:82/wp-content/files/10K: Connection timed out
http://your.proxy.server.com:82/wp-content/files/10K: byte count wrong
http://your.proxy.server.com:82/wp-content/files/5K: Connection timed out
http://your.proxy.server.com:82/wp-content/files/5K: byte count wrong
...
358528 fetches, 1000 max parallel, 1.9567e+09 bytes, in 60 seconds
5457.58 mean bytes/connection
5975.46 fetches/sec, 3.26116e+07 bytes/sec
msecs/connect: 119.183 mean, 9001.94 max, 0.112 min
msecs/first-response: 32.8481 mean, 9923.29 max, 6.416 min
142 bad byte counts
HTTP response codes:
  code 200 -- 358386

Squid:

http://your.proxy.server.com:81/wp-content/files/1K: Connection timed out
http://your.proxy.server.com:81/wp-content/files/1K: byte count wrong
http://your.proxy.server.com:81/wp-content/files/1K: Connection timed out
http://your.proxy.server.com:81/wp-content/files/1K: byte count wrong
http://your.proxy.server.com:81/wp-content/files/5K: Connection timed out
...
555818 fetches, 1000 max parallel, 3.03109e+09 bytes, in 60.0001 seconds
5453.38 mean bytes/connection
9263.62 fetches/sec, 5.0518e+07 bytes/sec
msecs/connect: 82.4403 mean, 9001.53 max, 0.123 min
msecs/first-response: 18.6487 mean, 6820 max, 0.356 min
84 bad byte counts
HTTP response codes:
  code 200 -- 555734

Truly impressive, isn’t it? Traffic Server managed to deliver 904,849 requests without a single failure. That’s another magnitude as the 358,386 Varnish managed to deliver. I guess Varnish as heavy CPU bound service was penalized by thread piling and a somewhat limited CPU here. Squid managed to deliver 555,734 requests. That’s ok, but Traffic Server still delivered almost twice as much. Its worth to be noted that both, Varnish and Squid failed to deliver all files correctly. Both delivered faulty responses in 142 and 84 cases respectively. For all three daemons the response times increased: That’s ok for heavy loaded servers and a mean response time of 10.91 ms is acceptable. Varnish loses completely here: 32.8 ms is quite a lot for static content, while Squid’s 18.6 ms are still ok.

Getting the packages

The downloads below are AMD64 binaries for Debian Squeeze:

[sorry, no packages available here anymore. Meanwhile Trafficserver is included in Debian Wheezy and recent versions Ubuntu out of the box]