• twitter image
  • facebook image
  • youtube image
  • linkedin image
Language: CMS Made Simple Czech CMS Made Simple France CMS Made Simple Spain CMS Made Simple Hungary CMS Made Simple Russia CMS Made Simple Netherlands

All times are UTC




Post new topic Reply to topic  [ 25 posts ]  Go to page Previous  1, 2
Author Message
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Wed Oct 26, 2005 7:58 pm 
Offline
Forum Members
Forum Members

Joined: Sun Oct 09, 2005 10:49 am
Posts: 61
Location: Nijmegen (The Netherlands)
Isn't this the wrong Topic for a discussion on PiSearch?
Hans


Top
  Profile  
 
Share On:
Share on Facebook Share on Twitter Share on Google+
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Thu Oct 27, 2005 12:10 am 
Offline
Power Poster
Power Poster
User avatar

Joined: Thu Jan 27, 2005 5:11 pm
Posts: 310
Location: Los Angeles, CA
Isn't this the wrong Topic for a discussion on PiSearch?
Probably :)

I've been discussing the requirements for a robust search module.

My issue with the approach PiSearch takes is that it'll work fine on a site with a couple of pages and/or a site with very light traffic, but will utterly destroy the performance of a server that doesn't meet those criteria. Now, that may not matter to some users, so PiSearch is fine for sites that are small and lightly trafficked. Unfortunately, I have client sites that have some fairly heavy traffic, and some that are quite extensive (hundreds of pages). I'm currently doing search on those sites using external programs (Swish++), but that's not an option for everyone.

Also, as a side note, PiSearch's optimization to only load content of type "content" will not work with News, for example, or any other module-based content type that's not embedded in another page. Again, this won't matter to everyone. However, I'm 80% done with a Catalog module, which relies heavily on its own content types. Any search I would use on such a site would need to be able to handle *all* content types.

In any case, here's what I think the minimum acceptable requirements for a professional search module would be (these, of course, are only my opinion... however, my opinion has been shaped by long years of experience and painful lessons learned):
  • Search must utilize an index generated asynchronously (e.g., don't generate your index every time you perform a search)
  • Search should build the index by spidering the site, so page content is seen as the end-user would see them.
  • Search should not induce a heavy load on the server, so the spidering should happen nightly and/or should be incrementally updated on content changes
  • Search should support arbitrary content types without any custom configuration work.
  • Search should not care about which underlying database is used by the CMS.

I've been meaning to write something like this, but it's not at the top of my priority queue right now.  My current solution works for my clients, because I can influence where they host, and can do my own Unixy thing on those machines. As soon as I have a client where that is not the case but who still needs search, I'll be building a search module.

_________________
Many modules available from the http://dev.cmsmadesimple.org
The CMS Made Simple Developer Cookbook is now available from Packt Publishers!


Top
  Profile  
 
Share On:
Share on Facebook Share on Twitter Share on Google+
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Thu Oct 27, 2005 3:21 pm 
Look here  http://forum.cmsmadesimple.org/http://p ... 534#p10534

That is speed:

2005 pages are searched
Generated in 6.541597 seconds by CMS Made Simple 0.11-beta4 (cached) using 2024 SQL queries
205 pages are searched
Generated in 1.212142 seconds by CMS Made Simple 0.11-beta4 (cached) using 224 SQL queries

Addodb is in use !!!

Pages are always fetched if in the content is a smartytag.

it is better to test something and than write about it.


Top
   
 
Share On:
Share on Facebook Share on Twitter Share on Google+
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Thu Oct 27, 2005 6:21 pm 
Offline
Power Poster
Power Poster
User avatar

Joined: Thu Jan 27, 2005 5:11 pm
Posts: 310
Location: Los Angeles, CA
Hey, if those numbers work for you, great!

As I said before, if the performance is good enough for your site, then the module is fine.

However, on some of the sites I have deployed, it's not acceptable for a search performed by a single user to take several seconds (consider how that scales when dozens of users are searching simultaneously). I'm not making these statements to be obnoxious. I'm merely stating my requirements for client sites. If it came across as a slam or attack, I apologize.

For what it's worth, the Swish++ approach yields the following timings for a single user's search of a site with roughly 1000 pages:

26 results found for "light"
real    0m0.007s
user    0m0.010s
sys    0m0.000s

(this is from the live server, with a fair number of people using it)

$ netstat -t | grep -i http | wc -l
    1352

_________________
Many modules available from the http://dev.cmsmadesimple.org
The CMS Made Simple Developer Cookbook is now available from Packt Publishers!


Top
  Profile  
 
Share On:
Share on Facebook Share on Twitter Share on Google+
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Thu Oct 27, 2005 6:27 pm 
Offline
Forum Members
Forum Members

Joined: Sun Oct 09, 2005 10:49 am
Posts: 61
Location: Nijmegen (The Netherlands)
Maybe it's a dumb question but it is possible to make a module of Swish++?
I read the site and it seems a good start for the badly needed searchmodule for CMSMS
Hans


Top
  Profile  
 
Share On:
Share on Facebook Share on Twitter Share on Google+
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Thu Oct 27, 2005 6:45 pm 
Offline
Power Poster
Power Poster
User avatar

Joined: Thu Jan 27, 2005 5:11 pm
Posts: 310
Location: Los Angeles, CA
It shouldn't be too hard to make a Swish++ module, but it would require users have shell access. It would also only work under Unix.

_________________
Many modules available from the http://dev.cmsmadesimple.org
The CMS Made Simple Developer Cookbook is now available from Packt Publishers!


Top
  Profile  
 
Share On:
Share on Facebook Share on Twitter Share on Google+
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Thu Oct 27, 2005 7:19 pm 
This is the speed of cmsmadesimple home
Generated in 1.219569 seconds by CMS Made Simple 0.10.3 (cached) using 18 SQL queries

This is the speed by testing piserach (205 pages):

Generated in 1.051407 seconds by CMS Made Simple 0.11-beta4 (cached) using 20 SQL queries

Speed of the cms and searching plugins must go hand in hand - no one needs here a search-plugin that is faster as the cms and with this cms only some people are using more as 50 pages.


Top
   
 
Share On:
Share on Facebook Share on Twitter Share on Google+
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Fri Oct 28, 2005 7:09 am 
Another result. Contents from the standrad new installation added with 5000 contents with a contentmakescript.

The time to present the results is  15.29580 seconds (word is found in 5000 pages worst case)

But the searchtime is only 4.384 seconds  - the difference is the time the cms always need .

The cms needs 10.911801 seconds to present the  homepage without any searching  (total 5005 records in the database)  !!!

And each time if you will view another page (without searching) you must wait this time.

Wishy's way to generate menusystems is wrong. That is the reason why the cms is so slow.

This - so i mean - is the first he has to do something - alle other is secondary.


Top
   
 
Share On:
Share on Facebook Share on Twitter Share on Google+
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Fri Oct 28, 2005 8:57 am 
Offline
Power Poster
Power Poster
User avatar

Joined: Wed Feb 09, 2005 9:30 pm
Posts: 282
Location: behind my desk
sjg \sjg:
It shouldn't be too hard to make a Swish++ module, but it would require users have shell access. It would also only work under Unix.


I use phpdig, works great. It works like a search engine should, it 'reads' the webpages, not just the content from the database (that's lame).
Maybe somebody could join so we can make it into a module?

_________________
Mambo sucks, that's why I am here.
Now they call it Joomla, but it still sucks!

CMSMS rules!


Top
  Profile  
 
Share On:
Share on Facebook Share on Twitter Share on Google+
 Post subject: Re: Creation of CMSMS Project Groups
PostPosted: Tue Nov 28, 2006 9:39 pm 
Offline
Forum Members
Forum Members

Joined: Fri Nov 10, 2006 7:56 pm
Posts: 24
That would be great. Is there any work being done to the searching mechanism? Either on the standard module or a new one?

Ideally I would like one that can search pdf files. Swish-E or PhpDig can do it, but I'm not sure if I can enough time to make it into a module.

Any updates on this?


Top
  Profile  
 
Share On:
Share on Facebook Share on Twitter Share on Google+
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 25 posts ]  Go to page Previous  1, 2

All times are UTC


Who is online

Users browsing this forum: No registered users


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
A2 Hosting