Posts Tagged ‘ Apache ’

Sphinx Search Engine Performance

The following is a summary of some real-world data collected from the Sphinx query logs on a cluster of 15 servers. Each server runs its own copy of Sphinx, Apache, a busy web application, MySQL and miscellaneous services.

The dataset contains 453 million query log instances from 180 Sphinx indexes, collected over several months, using Sphinx version 0.9.8 on Linux kernel 2.6.18. The servers are all Dell PowerEdge 1950 with Quad Core Intel® Xeon® E5335, 2×4MB Cache, 2.0GHz, 1333MHz FSB, SATA drives, 7200rpm.

Keep in mind, though, that this is real world data and not a controlled test. This is how Sphinx performed in our environment, for the particular way we use Sphinx.

The graph below displays the response time distribution for all servers and all indexes, and shows, for example, that 60% of queries complete within 0.01 secs, 80% within 0.1 secs and 99% within 0.5 secs. Response times tend to occur in 3 bands (corresponding to the peaks in the frequency graph) – <0.001 sec, 0.03 sec and 0.3secs, which partly relates to the number of disk accesses required to fulfil a request. At 0.001 sec, all data is in memory, while at 0.3 secs, several disk accesses are occurring. Whilst the middle peak is not so obvious in this graph, the per-server or per-index graphs often have different distributions but still tend to have peaks at one or more of these three bands.
Sphinx Query Response Times Total for all servers, all indexes

The next observation is that query word count affects performance, but not necessarily in proportion to the number of query words, as shown in the graph below. 1-4 word queries consistently offer best performance. The 6-50 words range is consistently the slowest, most likely because the chance of finding documents with multiple matches is high so there is extra ranking effort involved. Above 50, there is presumably a higher chance of having words with few matches, which speeds up the ranking process.
Sphinx Query Response Time by Query Word Count

Finally, we see that the size of the inverted index (.spd files) also affects performance. The three graphs below show how the response time distribution tends to move to the right as the index size increases. The larger the index, the higher the chance that data will need to be re-read from disk (rather than from Sphinx-internal or system buffers/cache), hence this is not unexpected.
Sphinx Query Response Times for Index Sizes 1MB - 3MB
Sphinx Query Response Times for Index Sizes 3MB - 30MBSphinx Query Response Times for Index Sizes >30MB

Here is a PDF summary of Sphinx performance for this dataset, including many additional graphs of the data by server and by index.

Advertisements

What is WAMPServer?

WAMP is a form of mini-server that can run on almost any Windows Operating System. WAMP includes Apache 2, PHP 5 (SMTP ports are disabled), and MySQL (phpMyAdmin and SQLitemanager are installed to manage your databases) preinstalled.An icon on the taskbar tray displays the status of WAMP, letting you know if;

a) WAMP is running but no services are opened (the icon will appear red),

b) WAMP is running and one service is opened (the icon will appear yellow)  or

c) WAMP is running with all services opened (the icon will appear white).

Apache and MySQL are considered to be services (they can be disabled by left-clicking on the taskbar icon, guiding your cursor over the service you wish to disable and selecting “Stop Service”).

The files/web pages that are hosted on your WAMP server can be accessed by typing http://localhost/ or http://127.0.0.1/ in the address bar of your web browser. WAMP must be running in order to access either of the above addresses.

If you would like to share your files/web pages with others, click on the icon located on your taskbar tray and select “Put Online.” You must have access to the Internet in order to continue.

Send the people that you would like to give access to the files/web pages hosted on your WAMP server your IP Address. Your can find your IP address here:

Your IP Address and Browser Details

Your IP: 218.248.25.5
Hostname: 218.248.25.5
Web browser: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2

(notes: IP address based on your current browser)

If you are a dial-up user or if you have a dynamic IP Address (one that changes every so often), you will need to keep the people that have your IP Address updated every time you connect to the internet.

You can download WAMPServer from WampServer.com.