CMS Made Simple Forums

Hi,

A discussion started on the IRC channel last week concerning the standard of modules and the possibility of having community based peer reviews of each module.  I have started to develop an extension to the core and some additions to ModuleManager which will allow the administrator of each site to provide a rating of each installed module. The reviews will be available from the ModuleManager and also (hopefully) on the individual project pages in the Forge.

The idea behind this is that new users of CMSMS will be not be distracted or put off by installing modules which are in beta mode, not fully developed, buggy  etc...

So far, I have implemented the following in my local version - I will submit this to Ted & co. when it is complete.

Each module is rated (1 to 5 stars) according to

• Does it function as described?
• Is it feature rich ?
• Is it easy to Use / Configure?
• Is documentation abundant and clear?
• Is it easy to customise the look and feel of the module output?
• From a developer standpoint does the code conform to CMSMS standard?

A rating is per install, that is each module can only be rated once per domain name. Suggestions as to how to improve this are welcome - we want to avoid the situation where a module can be bumped unfairly.

I should have a beta version of this early next week, all comments regarding this matter are welcome and  I will implement those that we reach concenus on.

Alan

I think vote for module from particular domain should expire in 2 years. I hope modules would improve during that time.

Hi,

That's a good point, how about if once a new version of the module is installed, a vote can be cast again.

This way the vote is always a combination of: by  domain, by  module, by  version

Alan

Quality of dependencies is another factor as well and would be good to note in overall ratings. For example...if a module was made 2 years ago and relied on modules A and B there is a good chance modules A and B have changed and may have affected the module attempted to be used (See the forge for XSPF Player for an example).

This isn't a one man job, email me if I can assist you in any way as this takes shape. CMSMS NEEDS this - (quoting another topic.."The forge is a mess")

Other items:
Module security (are params all known and properly cleaned)
Does the module follow tightly to the documented API?

Thanks,
It starting to strike me that two levels of appraisal may be needed.

1. From a user perspective
2. From a developer perspective.

SjG has offered to implment the forge / data storage side of things

Alan

For me is every rating very subjective.

Example 1: I am a developer and have enough experience with CMS Made Simple, I have no problem in configuring any module = 5 stars
Example 2: I am not a developer, but pretend to be one. I am not able to configure the module due to the lack of knowledge = 1 star
Example 3: I am developer and have no expereince with CMS Made Simple. I have not understand the general logic behind modules = 2 stars

For me, it is more important to know:
-------------------------------------------------------
1. What does the module do? It is not enough to read some sentences in description of the module given in the forge. I prefer full text search through the module help sections.
2. I would like to see download statistics for the modules. Not the one I can see in forge now, it is a lifetime statistic and new modules do not have any chance to appear there. I would like to see statistic broken down to the actual month or the last 30-90 days.
3. I would like to have RSS Feed for the intial releases of the modules. There are a lot of modules that are announced and never developed. I do not need the list of new empty projects, but the list of the modules that have at least one release.
4. I would appreciate if the module developers better mark-up alpha, beta and stable releases. There are a lot of alpha modules, that work like a sharm and "stable" releases that screw the admin panel right after installation.

Some objective criterias:
-----------------------------------------------
1. Quality of code. Enable $config['debug'] and see if and how much warnings are shown. Too much warnings means the code is not clean enough. The developer does not debug his code. It might be buggy and inefficient.
2. Bug report of the module. Are the bugs are handled or just ignored? Too much old unhandled and unclosed bugs can indicate that the module developer is not interested in the quality of the module or has left the development.
3. Translations. Can the module be found in the Translation Center? If not, it indicates that the develper has not follow the instructions on how the module translations can be realized. On the other hand, the larger the number of translations files for the module is, the more popular is the module, as the translation are not made for buggy or unuseful modules.
4. Valid XHTML. Is the sample template XHTML valid?
5. Documentation. How is the module documented? Is the help splitted into the standard sections like "What does it do?", "How do I use it?" etc. Are there undocumented actions or parameters?

Very important is:

does the module deliver language-files ?

Avantart wrote: Very important is:

does the module deliver language-files ?

The module does not "deliver" any language files except of the original language file, usually en_US.php. All the translations of the module (if it is configured correctly) can be made through CMS Made Simple Translation Center, so that the module developer does not need to care about them. It cannot be a quality criteria for the module if the developer is not able to translate it into 20 languages

Privet Sonya,

the moduledeveloper must check in the module into the translation center.

The modules are not into the translation center per se.

That's why I think the moduledeveloper must take care of that

All,

This is interresting material!
I was working on a module as well for the ratings (and stopped). In front of that I read some information on what others do (why invent the wheel again?).
I came accross the following:
http://drupalmodules.com/module-rating-guidelines
It helps put the ideas mentioned in metrics.

Duketown

Duketown, that's a great link - that covers a lot of aspects that we are looking at.

Thanks,
Alan

Sonya wrote: Some objective criterias:
-----------------------------------------------
1. Quality of code. Enable $config['debug'] and see if and how much warnings are shown. Too much warnings means the code is not clean enough. The developer does not debug his code. It might be buggy and inefficient.
2. Bug report of the module. Are the bugs are handled or just ignored? Too much old unhandled and unclosed bugs can indicate that the module developer is not interested in the quality of the module or has left the development.
3. Translations. Can the module be found in the Translation Center? If not, it indicates that the develper has not follow the instructions on how the module translations can be realized. On the other hand, the larger the number of translations files for the module is, the more popular is the module, as the translation are not made for buggy or unuseful modules.
4. Valid XHTML. Is the sample template XHTML valid?
5. Documentation. How is the module documented? Is the help splitted into the standard sections like "What does it do?", "How do I use it?" etc. Are there undocumented actions or parameters?

This is all relevant, and from a developer standpoint these need to be included. Perhaps when a rating is being made,we can offer two categories under which to rate it.

A: End User
B: Developer



This was originally intended to increase the end user experience with modules and have people waste less time on poorly written modules, I think the focus needs to stay there for the moment. 

thoughts on this?

Avantart wrote: Privet Sonya,

the moduledeveloper must check in the module into the translation center.

The modules are not into the translation center per se.

That's why I think the moduledeveloper must take care of that

The language support is slightly off topic here, but it has been noted that it is a factor.