[Solved] Growing pains, my site is exceeding my host limits

[cohmstede]

My site (http://www.whycanticonnect.com/) is already exceeding my host resource limitations after one week. The problem seems to be a limitation of only being able to run 15 simultaneous processes. This wouldn't generally be a problem but a download is considered a process. My downloads are not large (the largest is just under 15 meg) but I'm currently getting about 200 a day and I expect this number to increase. My current host is http://www.tmdhosting.com and I knew nothing of this limitation when I first signed up (this is my first site so everything is new to me). Their proposed resolution is to switch to a commercial account (ouch, expensive). Is the 15 simultaneous processes limitation common among hosts? Does anyone have a better host they recommend?

Thanks

[calguy1000]

By 'processes' I assume that they mean php scripts. (really more definition is needed here). Could also be database connections.

15 simultaneous connections is usually a reasonable amount. It means 15 computers can request things via http->php at the same time.

But that 'reasonable amount' depends on alot of things. Including
- The number of visitors
- The functionality of the site
- The number of 'php' requests for one page request (ajax would also probably be a processing request).
- The processing time it takes to complete the request.
- The hosts load (are they overselling their servers?)
- The hosts bandwidth
- Added: Slow database connection on the host
(optimally the database should be on the same host as the web server so that the loopback adapter can be used).
- Added: The host filesystem architecture (if you are reading and sending a file that is hosted on a slow drive/mounted drive)
- The efficiency of the code being run
- Database errors, and sub-optimal indexing
- The end users bandwidth
AFAIK - a user with a slow bandwidth connection trying to download a large file will keep the 'process' running longer than a user with a high bandwidth connection.
- Network connections between the host and the user (fat pipes vs. skinny pipes etc)

Some ways to optimize your requests:
- Using GooglePageSpeed and YSlow etc, and optimizing your site as much as possible.
- Browser caching (so that users don't have to download the same resource multiple times)
- Using CDN's or other resources to host large files
- Server side caching (so that php doesn't have to process the template for each request, thus reducing request times).
- persistent db connections (possibly)
- Optimizing your database
- Query analysis to create indexes tuned to your site to reduce request time

Unfortunately, some of these optimization suggestions require advanced knowledge, and a considerable amount of time and effort.

[cohmstede]

Hi calguy1000

Thanks for your quick response. I'm new to this so I'm still learning. I chose CMS Made Simple because it looked like my best option to be able to create my site by just putting together existing pieces. I'm a C++ programmer and I've always said the best code is created with copy and paste. Anyway, I've set all my pages to cache (except login and register). The first time I saw the resource problem, I talked to the tech guys at the host and they said all the running processes were download.php from, the module: DownloadManager. This module is perfect for my needs and was easy to integrate into my site. I wouldn't presume to be able to optimize this beyond what the original programmer wrote and I avoid making any changes as I have to maintain them in upgrades. You mentioned using a CDN for hosting my downloads. I'm not happy that I would need another service but if it gets rid of the problem and is cost effective, why not. How would this work? How would I hand of the process to the cloud service? I know the DownloadManager module is not yours but would you have any idea if I could integrate this into the module or would I need to create my own download functionality?

Thanks again

[cohmstede]

OK, I think I got this all working. I'm still keeping an eye on my usage totals on my site to be sure. First off, picking the CDN was more difficult than I thought it would be, in fact, some don't even allow direct download links. I finally decided on Amazon S3. Amazon S3 allows you to add up the services you want to use where the others gave me full blown packages that mostly just got in my way. Kind of like getting a sandwich, everything on it, fries and a drink when all I wanted was two slices of bread and a piece of cheese. I was able to still use DownloadManager by simply replacing the file locations with my new Amazon S3 URLs. This did however, break the counters. I could have looked into it more but it was just as easy to replace them with the HitCounter module. There's still a few things I want to tweak but I'm going to let the dust settle first.

Thanks for your help.

[cohmstede]

I've removed all the background images from the headers and footers replacing them with simple background colors. This dropped my I/O and greatly decreased my loading time.

[cohmstede]

After my latest software release I'm having no problem handling the increased traffic.