Page 2 of 3

Re: Dynamically generated but statically served pages

Posted: Wed Sep 27, 2006 5:38 pm
by Pierre M.
Hi,

I don't know Smarty and I'm not a PHP dev, but I like to think that Piratos is right :
-the $smarty->use_sub_dir tends to make me think that Smarty can send the output to a disk hierarchy rather than the HTTP tube. This should solve my "implementation point #1".
-the $smarty->caching reminds me that Smarty should take care of changes globally (page, news, calendar entry, bloc, template... AND even structure of the heirarchy) so if Smarty reiterates all pages, I'm optimistic this solves my "implementation point #2".

It would be nice : just Smarty tuning, no code. Smart and bugproof :-)

I hope a dev is going to follow Piratos' wisdom

PM

PS@Kavrick : I agree URLs should be pretty and SE Friendly and content should not be duplicated.

Re: Dynamically generated but statically served pages

Posted: Fri Oct 13, 2006 3:36 am
by mahjong
Piratos wrote:That is easy - you can make it in some minutes.
It's easy to say, but how can it actually be done? Some instructions, please...

Re: Dynamically generated but statically served pages

Posted: Sun Oct 15, 2006 8:31 pm
by allfg
Im really interesting in this feate too

Re: Dynamically generated but statically served pages

Posted: Sun Oct 22, 2006 4:31 pm
by PatDeLux
Yes, I am super interested in that too, and I guess using the Smarty way is the easiest. It also has the advantage of not beeing an Apache only solution.

FYI, I have measured the response time of my homepage - (thanks to free tool www.site24x7.com) :

CMSms version: around 600ms (see report: http://www.site24x7.com/login/status.do ... 0000002036)
Pure html version: around 30ms (see report: http://www.site24x7.com/login/status.do ... 0000006003)

Both are on the same server ! The CMS version is very usable because I am having very little traffic, but that a 20 times factor seems a lot to me, specially since my pages will hardly change more than once a week.

Re: Dynamically generated but statically served pages

Posted: Mon Oct 23, 2006 1:01 am
by Ted
We had $smarty->use_sub_dir on in 0.13.  It caused SO many problems that we had to turn it back off.  Unfortunately, it's not very friendly to a lot of different setups... and I don't have a good reason as to why.  I agree that it would be nice to have a totally html page after it's done, but I think that's beyond the scope of CMSMS.  There are tools out there (wget) that'll do this for you.

Re: Dynamically generated but statically served pages

Posted: Wed Oct 25, 2006 4:04 pm
by mahjong
I think that's beyond the scope of CMSMS.
It can be seen otherwise. To me, managing the content of the cache is part of a complete content management system and it should not be left to users, especially for a simple to use CMS.

Too bad you have given up. It's feasible since I've seen it done. For instance on SPIP CMS, the output is not send directly to the browser, but to a cache first. All internal CMS tags requiring fetching data in the database are digested, but all remaining embed PHP code left to be rendered upon user requests. The front end engine always fetches PHP pages out of the cache, requesting them to be generated by the CMS backend when needed or too old.

Image
We had $smarty->use_sub_dir on in 0.13.  It caused SO many problems that we had to turn it back off.  Unfortunately, it's not very friendly to a lot of different setups...
Does it mean, if I turn it on again, it could work perfectly fine on my set-up, even tough others have problems with it? If yes, why not simply make it an at-your-own-risk option in the admin control panel?

Re: Dynamically generated but statically served pages

Posted: Wed Oct 25, 2006 7:48 pm
by calguy1000
Does it mean, if I turn it on again, it could work perfectly fine on my set-up, even tough others have problems with it? If yes, why not simply make it an at-your-own-risk option in the admin control panel?
Unfortunately if we say 'use at your own risk' it never works out that way.  People expect it to work, and expect support, and then complain when they invest time in something then finally read the 'use at your own risk' part. 

Many CMS sites are very dynamically generated.  Things like News and Calendar entries are not even fine to cache, because the page hasn't changed doesn't mean that the contents of some of the modules that may be used on that page hasn't.  As well, Pages that display forms, login info, or even information based on the date, time, or user group that you're in can certainly not be cached at all.  And more and more sites are integrating this type of functionality, which reduces the need and use of caching every day.

For those sites that are mostly static, then an external package like HTTRACK, or wget are great as they'll allow the website to be embedded on a CD, or statically from a web host.  Hell, people can even schedule a job to recursively crawl through the web site, generate the static pages, and ftp them from they're local server to the web server.

Re: Dynamically generated but statically served pages

Posted: Thu Oct 26, 2006 8:57 pm
by mahjong
News and Calendar entries ... forms, login info, or even information based on the date, time, or user group... can certainly not be cached...
You seem to think the task of deciding which part should be cached and which should not is too complicated for the software. That's not the case. It can be extremely simple :
  • 1) calculate all tags in content blocks,
  • 2) insert those rendered blocks into templates, don't render the templates yet
  • 3) save those unfinished pages into a cache


In effect, the parts inserted into content blocks will be cached and those inserted into templates will be rendered in real time. The author of a static page would one have to make sure his template is composed only of pure HTML.
And more and more sites are integrating this type of functionality, which reduces the need and use of caching every day.
The more generated content you have, the more advantageous it is to not recalculate everything, each time, over and over.
For those sites that are mostly static, then an external package like HTTRACK, or wget are great
Quite the opposite. With wget or HTTRACK you take a snapshot, therefore transforming a partially dynamic site into a fully static one. That's not an acceptable solution. As for truly static sites, there is no point of using a CMS in first place. Use a HTML editor, instead.
Hell, people can even schedule a job to recursively crawl through the web site, generate the static pages, and ftp them from they're local server to the web server.
-"Hell, people can write their own Content Management System to manage the output of CMSMS." What kind of answer is this? You can not be serious. I understand you had difficulties solving this, but it's not a reason to say the problem is better solved by quitting.

Instead of viewing this problem as too difficult to solve with the approach you've already tried, why not look at it from fresh new angle?

Instead of expecting CMSMS's output to be HTML pages directly viewable with a browser, think of it as generator of PHP pages for a cache. The front end of the cache will do the job of converting those PHP pages into HTML pages. It's a lot simpler that way.
Things like News and Calendar entries are not even fine to cache, because the page hasn't changed doesn't mean that the contents of some of the modules that may be used on that page hasn't.
News and calendars entries are not updated all the time. Besides, nothing forbids me to manually recalculate a cached page before it's scheduled to be refreshed, or even the entire website. At night, when no ones is editing the content, why should the site not be mostly cached?

With a cache, the gains of performance are so great you cannot simply dismiss it.

Re: Dynamically generated but statically served pages

Posted: Fri Oct 27, 2006 3:34 pm
by Pierre M.
Hi All !
Too bad you have given up... I understand you had difficulties solving this... why not look at it from fresh new angle?
These could be my words. Thank you mahjong and thank you all the devs for the great product so far.

A°)Functionality advocation :
IT should reflect human relationship to information : you read 10 articles and post one. you receive 10 emails and send one. you read 10 web pages and you update one. So IT is about writing once and reading ten times. No just about content management, but with transactional systems too : you read/compare ten times before you place an auction.
So, well made content broadcasting system are designed around this. Compare Apache and lighttpd : lighttpd is aggressively cache oriented by design (for example it knows it hasn't to compute the day of week every second, just every 86400 seconds).
And this is why I have been respectfully requesting this feature of serving static content : it is just a detail about broadcasting performance, it is independant of the very dynamic content management.

Advantages :
-Performance : very nice for high traffic sites. very nice on shared hosting.
-Pretty URLs, the same way, with any web server : because you generate the tree that fits your needs.
-Security : the URL of your CMSms installation is very obfuscated (/my/secret/path/to/it). So is the admin console.
-Suitable for more hosting solutions : you consume less PHP and DB, you fit in your mem & cpu quotas.
-Static export feature included for free. (run undefaceable, form CD, backup...)
-CSS and alike are static content.

Drawbacks :
-Sometimes you should clear your cache. (with the admin interface or your FTP client)
-please fill here.

B°)Implementation guesses/bets :
-May be Smarty can do it, so may be modules should be rewritten to become "Smarty caching" aware ?
-If not touching the modules, may be pages/templates could have a "don't cache" flag to simplify ?
-Is it that difficult to change "echo $content_from_smarty" to "write_file($url,$content_from_smarty)" if Smarty tuning is incompatible with modules & co ? Or pipe CURL as previously suggested in this thread ?
-Very Simple Algorythm (to start with) : whenever a single simple tiny change is made, regenerate all URLs. With a job queue to smoothen the load.
Unfortunately if we say 'use at your own risk' it never works out that way. People expect it to work, and expect support, and then complain...
There are odd, ungratefull people. But this shouldn't prevent you from making even better software.
See this thread : you have happy users who like your work and wish this feature.
May be you could solve the "use at your own risk" the pretty URLs way : turn it off by default.

Hope it helps. Have fun hacking and using CMSms !
PM

Re: Dynamically generated but statically served pages

Posted: Sat Oct 28, 2006 12:01 pm
by moorezilla
Should this functionality be approached as a module? If the changes to the core are relatively simple, it seems like this could be handled within the module system. For sites that didn't change much, an admin/customer could just export the site manually through the admin section, and for other sites a crontab could be set to export the site one or two times a day.

I don't know enough about the process to really argue about it, but this smells like a module opportunity to me. If it works well as a module, maybe then it could be included in the core as a later option. 

Re: Dynamically generated but statically served pages

Posted: Tue Dec 12, 2006 3:36 am
by philight
And what's new about this beautiful wished feature? A fabulous christmas gift?? ;D

Re: Dynamically generated but statically served pages

Posted: Tue Dec 12, 2006 11:12 pm
by Pierre M.
An interesting background article about this :
http://www.onlamp.com/pub/a/onlamp/2006 ... cache.html

PM

Re: Dynamically generated but statically served pages

Posted: Wed Jan 10, 2007 6:21 am
by cyberman
Please look at my simple solution too :) ...

http://dev.cmsmadesimple.org/projects/cache/

Re: Dynamically generated but statically served pages

Posted: Wed Jan 10, 2007 11:31 am
by Pierre M.
Well...
Apache (or lightttpd) have a mod_cache reverse proxy to statically serve all unfrequently updated sites (frequency>1h).
You have a promising (self described "quick&dirty") module with good results provided with some added complexity (new tags/template tweaking).
Ted has a promising v2.0 with "full page caching" integrated without the complexity. (see http://forum.cmsmadesimple.org/index.ph ... l#msg46230 )

And some have knowledges about Smarty caching/static rendering (or a light template engine).

At least the first 3 solutions seems compatible with each other.
I'm confident CMSms has good devs who are going to get the best of this mix and this thread will be closed before 2008 :-)

PM

Re: Dynamically generated but statically served pages

Posted: Mon Feb 26, 2007 12:42 am
by SwiftCoder
I am sure someone must have suggested this before, but I don't see it anywhere, so here goes:

What about enabling pretty-urls and then running wget or similar on the whole site?

This will basically mirror a static copy of the site into a local directory, and then it can be FTP'd to the server. I use this method with typo3, and it works very well. It allows me to run the CMS on my local dev machine, and then upload a static copy to the server whenever I make a change. It also allows me to fully preview the site before committing to the web server.

FYI, I am asking this because I am thinking of switching to CMSMS soon, but I would like to use this same method, and if anyone can confirm/deny that it works, or supply any unusual caveats, it would help my decision. I am aware that truly dynamic stuff, such as forums, don't work this way, but I don't need those anyway.

Thanks for any help,

Tristam MacDonald