Module Ratings and Appraisals - Opinions Sought

Talk about writing modules and plugins for CMS Made Simple, or about specific core functionality. This board is for PHP programmers that are contributing to CMSMS not for site developers
User avatar
aln_cms
Forum Members
Forum Members
Posts: 88
Joined: Mon Jan 08, 2007 7:09 pm
Location: Ireland

Module Ratings and Appraisals - Opinions Sought

Post by aln_cms »

Hi,

A discussion started on the IRC channel last week concerning the standard of modules and the possibility of having community based peer reviews of each module.  I have started to develop an extension to the core and some additions to ModuleManager which will allow the administrator of each site to provide a rating of each installed module. The reviews will be available from the ModuleManager and also (hopefully) on the individual project pages in the Forge.

The idea behind this is that new users of CMSMS will be not be distracted or put off by installing modules which are in beta mode, not fully developed, buggy  etc...

So far, I have implemented the following in my local version - I will submit this to Ted & co. when it is complete.

Each module is rated (1 to 5 stars) according to
  • Does it function as described?
  • Is it feature rich ?
  • Is it easy to Use / Configure?
  • Is documentation abundant and clear?
  • Is it easy to customise the look and feel of the module output?
  • From a developer standpoint does the code conform to CMSMS standard?
A rating is per install, that is each module can only be rated once per domain name. Suggestions as to how to improve this are welcome - we want to avoid the situation where a module can be bumped unfairly.

I should have a beta version of this early next week, all comments regarding this matter are welcome and  I will implement those that we reach concenus on.

Alan
--
Alan Ryan
Lead Developer
CodeCrunchers Internet Software Development - Ireland
http://www.codecrunchers.ie
Peciura

Re: Module Ratings and Appraisals - Opinions Sought

Post by Peciura »

I think vote for module from particular domain should expire in 2 years. I hope modules would improve during that time.
User avatar
aln_cms
Forum Members
Forum Members
Posts: 88
Joined: Mon Jan 08, 2007 7:09 pm
Location: Ireland

Re: Module Ratings and Appraisals - Opinions Sought

Post by aln_cms »

Hi,

That's a good point, how about if once a new version of the module is installed, a vote can be cast again.

This way the vote is always a combination of: by  domain, by  module, by  version

Alan
--
Alan Ryan
Lead Developer
CodeCrunchers Internet Software Development - Ireland
http://www.codecrunchers.ie
milehigh

Re: Module Ratings and Appraisals - Opinions Sought

Post by milehigh »

Quality of dependencies is another factor as well and would be good to note in overall ratings. For example...if a module was made 2 years ago and relied on modules A and B there is a good chance modules A and B have changed and may have affected the module attempted to be used (See the forge for XSPF Player for an example).

This isn't a one man job, email me if I can assist you in any way as this takes shape. CMSMS NEEDS this - (quoting another topic.."The forge is a mess")

Other items:
Module security (are params all known and properly cleaned)
Does the module follow tightly to the documented API?
User avatar
aln_cms
Forum Members
Forum Members
Posts: 88
Joined: Mon Jan 08, 2007 7:09 pm
Location: Ireland

Re: Module Ratings and Appraisals - Opinions Sought

Post by aln_cms »

Thanks,
It starting to strike me that two levels of appraisal may be needed.

1. From a user perspective
2. From a developer perspective.

SjG has offered to implment the forge / data storage side of things

Alan
Last edited by aln_cms on Wed Nov 11, 2009 3:49 pm, edited 1 time in total.
--
Alan Ryan
Lead Developer
CodeCrunchers Internet Software Development - Ireland
http://www.codecrunchers.ie
Sonya

Re: Module Ratings and Appraisals - Opinions Sought

Post by Sonya »

For me is every rating very subjective.

Example 1: I am a developer and have enough experience with CMS Made Simple, I have no problem in configuring any module = 5 stars
Example 2: I am not a developer, but pretend to be one. I am not able to configure the module due to the lack of knowledge = 1 star
Example 3: I am developer and have no expereince with CMS Made Simple. I have not understand the general logic behind modules = 2 stars

For me, it is more important to know:
-------------------------------------------------------
1. What does the module do? It is not enough to read some sentences in description of the module given in the forge. I prefer full text search through the module help sections.
2. I would like to see download statistics for the modules. Not the one I can see in forge now, it is a lifetime statistic and new modules do not have any chance to appear there. I would like to see statistic broken down to the actual month or the last 30-90 days.
3. I would like to have RSS Feed for the intial releases of the modules. There are a lot of modules that are announced and never developed. I do not need the list of new empty projects, but the list of the modules that have at least one release.
4. I would appreciate if the module developers better mark-up alpha, beta and stable releases. There are a lot of alpha modules, that work like a sharm and "stable" releases that screw the admin panel right after installation.
Last edited by Sonya on Thu Nov 12, 2009 11:26 am, edited 1 time in total.
Sonya

Re: Module Ratings and Appraisals - Opinions Sought

Post by Sonya »

Some objective criterias:
-----------------------------------------------
1. Quality of code. Enable $config['debug'] and see if and how much warnings are shown. Too much warnings means the code is not clean enough. The developer does not debug his code. It might be buggy and inefficient.
2. Bug report of the module. Are the bugs are handled or just ignored? Too much old unhandled and unclosed bugs can indicate that the module developer is not interested in the quality of the module or has left the development.
3. Translations. Can the module be found in the Translation Center? If not, it indicates that the develper has not follow the instructions on how the module translations can be realized. On the other hand, the larger the number of translations files for the module is, the more popular is the module, as the translation are not made for buggy or unuseful modules.
4. Valid XHTML. Is the sample template XHTML valid?
5. Documentation. How is the module documented? Is the help splitted into the standard sections like "What does it do?", "How do I use it?" etc. Are there undocumented actions or parameters?
Avantart

Re: Module Ratings and Appraisals - Opinions Sought

Post by Avantart »

Very important is:

does the module deliver language-files ?
Sonya

Re: Module Ratings and Appraisals - Opinions Sought

Post by Sonya »

Avantart wrote: Very important is:

does the module deliver language-files ?

The module does not "deliver" any language files except of the original language file, usually en_US.php. All the translations of the module (if it is configured correctly) can be made through CMS Made Simple Translation Center, so that the module developer does not need to care about them. It cannot be a quality criteria for the module if the developer is not able to translate it into 20 languages :)
Avantart

Re: Module Ratings and Appraisals - Opinions Sought

Post by Avantart »

Privet Sonya,

the moduledeveloper must check in the module into the translation center.

The modules are not into the translation center per se.

That's why I think the moduledeveloper must take care of that
Duketown

Re: Module Ratings and Appraisals - Opinions Sought

Post by Duketown »

All,

This is interresting material!
I was working on a module as well for the ratings (and stopped). In front of that I read some information on what others do (why invent the wheel again?).
I came accross the following:
http://drupalmodules.com/module-rating-guidelines
It helps put the ideas mentioned in metrics.

Duketown
User avatar
aln_cms
Forum Members
Forum Members
Posts: 88
Joined: Mon Jan 08, 2007 7:09 pm
Location: Ireland

Re: Module Ratings and Appraisals - Opinions Sought

Post by aln_cms »

Duketown, that's a great link - that covers a lot of aspects that we are looking at.

Thanks,
Alan
--
Alan Ryan
Lead Developer
CodeCrunchers Internet Software Development - Ireland
http://www.codecrunchers.ie
User avatar
aln_cms
Forum Members
Forum Members
Posts: 88
Joined: Mon Jan 08, 2007 7:09 pm
Location: Ireland

Re: Module Ratings and Appraisals - Opinions Sought

Post by aln_cms »

Sonya wrote: Some objective criterias:
-----------------------------------------------
1. Quality of code. Enable $config['debug'] and see if and how much warnings are shown. Too much warnings means the code is not clean enough. The developer does not debug his code. It might be buggy and inefficient.
2. Bug report of the module. Are the bugs are handled or just ignored? Too much old unhandled and unclosed bugs can indicate that the module developer is not interested in the quality of the module or has left the development.
3. Translations. Can the module be found in the Translation Center? If not, it indicates that the develper has not follow the instructions on how the module translations can be realized. On the other hand, the larger the number of translations files for the module is, the more popular is the module, as the translation are not made for buggy or unuseful modules.
4. Valid XHTML. Is the sample template XHTML valid?
5. Documentation. How is the module documented? Is the help splitted into the standard sections like "What does it do?", "How do I use it?" etc. Are there undocumented actions or parameters?

This is all relevant, and from a developer standpoint these need to be included. Perhaps when a rating is being made,we can offer two categories under which to rate it.

A: End User
B: Developer



This was originally intended to increase the end user experience with modules and have people waste less time on poorly written modules, I think the focus needs to stay there for the moment. 

thoughts on this?
--
Alan Ryan
Lead Developer
CodeCrunchers Internet Software Development - Ireland
http://www.codecrunchers.ie
User avatar
aln_cms
Forum Members
Forum Members
Posts: 88
Joined: Mon Jan 08, 2007 7:09 pm
Location: Ireland

Re: Module Ratings and Appraisals - Opinions Sought

Post by aln_cms »

Avantart wrote: Privet Sonya,

the moduledeveloper must check in the module into the translation center.

The modules are not into the translation center per se.

That's why I think the moduledeveloper must take care of that
The language support is slightly off topic here, but it has been noted that it is a factor.
--
Alan Ryan
Lead Developer
CodeCrunchers Internet Software Development - Ireland
http://www.codecrunchers.ie
Post Reply

Return to “Developers Discussion”