Module Ratings and Appraisals - Opinions Sought

[Sonya]

This was originally intended to increase the end user experience with modules and have people waste less time on poorly written modules, I think the focus needs to stay there for the moment.  

Well, I am a developer and a teacher (that is much more awful) I have problems with ratings made from the questions like:
1. Are the features useful? Well, I cannot imagine that a developer spends his time by implementing "unuseful features" . If they are unuseful why should anybody use the module?
2. Is it feature rich ? If I need only one feature from a very simple and basic module and my requirements are solved with the module, is it a point to say, the module is bad because it is not feature rich?
3. Is the module reliable? An end user cannot appraise if the module is reliable. If end user does not see the issue it does not mean that the module has no bugs or no security holes.
4. Is there enough documentation? Enough from what point of view? For beginners or for experts or for end users?

I am probably not the right person for this rating . I am too technical and do not like any soft factors, as in my daily work I have to appraise code and projects with clear defined criterion.

[calguy1000]

You people all have to remember why these modules get written, or enhanced

There is only one reason.... and that is to satisfy the needs of one person, or one paying customer.  The software is released to the public in the hopes that somebody else may find it useful.  If anybody else can use the module then it is purely a bonus. 

The software is released purely on an 'as is' basis.  The software is free, and there is, and can be no expectation of features, functionality, documentation, or translation on the part of some anonymous web developer that wants to use the software.

Some times developers will fix bugs, or provide support for a piece of code purely out of the goodness of their heart... Other times, they won't.

I'm personally, very tired of this discussion.

[aln_cms]

Hi calguy,

Sorry to hear that you're very tired of the discussion; we're just trying to improve the end user experience. It almost seems like you feel that we are judging the developer on their output - this is by no means what this intends to do.  I want a tool that mitigates the negative experience of loading a module which is incomplete. 

Perhaps, rather than an all damning fire and brimstone statment such as you just posted,  you could tell us where you stand on having something like this added into the core.

Alan

[tyman00]

I think it wouldn't take much to figure out what he thinks about having it in the core judging by his post.

Ratings/Rankings of modules are too subjective to be valid and beneficial. Why? Well you have CMSMS Beginners, Web Development Beginners, CMSMS Experts, Web Dev Experts with no CMS experience, End Users all voting on the same piece of software.

So when Web Dev newbie comes in and wants to make a photo gallery but gets frustrated in the first 10 minutes because he can't create a gallery as quickly as he can on Flickr so he gives it a 1 out of 5.

Then I come in to build a photo gallery and I can't get enough of the exact same module because it is so flexible that I can implement what ever JS plugin I want within minutes with limited effort. I can then customize the look with CSS and more JS and have a nice professional looking Photo Gallery within minutes all managed inside of the CMSMS admin... so I give it a 5 out of 5.


So there you have two completely different people voting on the same piece of software with completely opposite opinions. How are two exact opposite rankings going to be useful to future viewers... they aren't. It becomes a disservice to everyone.

Then what happens when the module first comes out and 5 people give it a 1 because it is missing a critical feature (in their opinion) which is slated for future releases? When the feature is added 2 weeks later the module now has 5 strikes against it for a no longer valid issue and it no longer get's download traffic because it has a 1 star rating. HUGE disservice to the developer, module and community.

We can't even agree on a good set of criteria, if that can't be accomplished how can we expect something as subjective as rating to work... let alone ensure everyone is following the criteria and not basing their rating off of something completely irrelevant.

Why can't people just take a couple minutes to try a module out and review the FR/BR in the forge and see what buzz is out there on the forum for the module. 10 minutes of research has proven more useful for me over the last 4  years than a 4 star rating from 15 people that I have no idea if their caliber of experience is on the same playing field as mine, be it higher or lower.

[aln_cms]

Hi tyman,

That's an excellent post, I've really had a change of heart about the ratings add-on.  You put that so succinctly that I'm wondiering how I didn't see this problem in the first place, especially given the wide diversity of the rating criteria...

I'm officialy holding on doing any further development until I think of a fool proof system

Regards,
Alan

[jmcgin51]

Something I think would help is to have the module's Help file available for viewing in the Forge.  The Help file is often the best source of documentation for a module (as it should be), and there are many times when I would like a chance to read the Help before actually downloading/installing the mod.

Assuming the developer did a good job with documentation, this would allow me to see what parameters are available in the module, if there are any dependencies, any known limitations, etc.

Currently there are Release Notes and Changelog links in the Forge - how about a Documentation link as well, which would open the module Help file?

[tyman00]

Alan - Under no circumstances do I want to block creativity. I would just sure hate to see lots of time and effort put into a system that is ham-stringed by a diversity (because this community is very diverse in many areas).

Feel free to bounce your ideas around in here. I too have tried to think of a way that we can properly provide a good QA system for the 3rd party modules because I think it is important too. Just not an easy black and white job, unfortunately


jmcgin - I personally think that wouldn't hurt any and could help. It will definitely help those willing to take time to research instead of expecting everything.




***Completely Off-Topic*** Alan, I get a kick out of your Avatar 

[aln_cms]

tyman,

Just not an easy black and white job, unfortunately

I think this sums it up, it's needed and it would be a great feature - but if it's wrong it would be worse than the 'no ratings' status quo  .. I'm not giving up, just going to think about it harder and longer.  Starting now to wonder if someone who has a Forge account would be presented with a differenct evaluation system than others, however that doesn't really solve the other issues/scenarios you correctly raised.

Currently sourcing much inspiration from Coders At Work http://www.codersatwork.com/, maybe I'll be back tomorrow with a kick ass solution, or maybe I won't 

[replytomk3]

http://forum.cmsmadesimple.org/index.php/topic,39095.new.html#new

[Sonya]

tyman00, I like what you write. I was trying to say it in my first post on this thread but it apparently it was not so clear as yours  ;D

But, still I can understand all efforts to give the development of 3rd party modules more control. There are 631 hosted projects; I estimate that 5% of them are usable. More than a half do not have any releases. A lot of them are abandoned. I think that any developer (not end user) who makes his first steps with CMS Made Simple is stuck in the forge and spend a lot of time to figure out that a lot of modules are just, well, "not so good". How many of them get frustrated after installing of 4th or 5th module that is delivered "as is" some years ago and had no signs of live since then?

You can appraise the modules quickly and efficiently yourself, because you are a developer with very deep knowledge of CMS Made Simple and with the good background knowledge of how the forge is organized. This knowledge is yours; it is not stated anywhere else. The new developer must spend a lot of time to figure it out.

I think that any efforts made "outside" of forge are useless, as they have to be kept actual and synchronized and no one can afford it continuously. It means that any rating or "list of best modules" will die in a short or long period of time. My suggestions for the forge:

1. Include the help section of the modules in the search feature of forge, not just a description. People search for functionality, not for module names.
2. Hide the modules with no releases from the projects' list. They do not have any yield for the community.
3. Replace the lifetime download statistic with statistic of the last month, so that new modules do have chances to appear in the list.
4. Replace the list of the new created projects with the list of the initial releases. Nobody is interested in what projects are announced to be developed some day. But the most of us are interested in the initial versions.
5. Add additional field to the project where the developer can place the link to his demo (= his website).

Advantages:
--------------------------
1. The search time for the functionality will be extremely shortened, not only for end users, but for dev team as well. Help section includes much more information about the module than three line description fields, modules without releases are not shown, so that nobody must spend his time to click through project names without a line of code.

2. The number of support requests for “dead” modules in forum will go down. Support and dev team can invest more time in core development. Questions like “what module should I take to realize that and this” will go down.

3. Actual download statistics gives the module developer the feeling how useful his module is. It encourages developing the module further if you see that you make something useful. Initial releases are exciting for the developers, they pine for feedback, but at the same time they are often overlooked. It is just impossible to track all of new announced projects. Announcing initial release will give the developer instant feedback from the community. Not announcing encourage the developer to leave it, so that it soon abandonend.  Dev team can profit from new useful modules as well.

4. If developers could link to the demo, a lot them would have done it. It is like own advertising, there are a lot of developers who do make good jobs, so why not to give them the opportunity to present themselves instead of treating them as a "number" or anonymous name in forge? I mean active link.

5. Dev team can save time on participation in such “3rd party modules quality” discussions. 

[tyman00]

tyman00, I like what you write. I was trying to say it in my first post on this thread but it apparently it was not so clear as yours  ;D

But, still I can understand all efforts to give the development of 3rd party modules more control. There are 631 hosted projects; I estimate that 5% of them are usable. More than a half do not have any releases. A lot of them are abandoned. I think that any developer (not end user) who makes his first steps with CMS Made Simple is stuck in the forge and spend a lot of time to figure out that a lot of modules are just, well, "not so good". How many of them get frustrated after installing of 4th or 5th module that is delivered "as is" some years ago and had no signs of live since then?

You can appraise the modules quickly and efficiently yourself, because you are a developer with very deep knowledge of CMS Made Simple and with the good background knowledge of how the forge is organized. This knowledge is yours; it is not stated anywhere else. The new developer must spend a lot of time to figure it out.

I agree that there are far too many useless modules on the Forge. There are plans in place to help out the situation. I am also sympathetic to the fact that I am indeed familiar with CMSMS and what modules are/aren't good (though I am surprised by one once and awhile). But I became knowledgeable in the system and the modules by doing exactly as I described. Downloading, playing, researching, repeating

I think that any efforts made "outside" of forge are useless, as they have to be kept actual and synchronized and no one can afford it continuously. It means that any rating or "list of best modules" will die in a short or long period of time.

I share the same fears.

My suggestions for the forge:

1. Include the help section of the modules in the search feature of forge, not just a description. People search for functionality, not for module names.
2. Hide the modules with no releases from the projects' list. They do not have any yield for the community.
3. Replace the lifetime download statistic with statistic of the last month, so that new modules do have chances to appear in the list.
4. Replace the list of the new created projects with the list of the initial releases. Nobody is interested in what projects are announced to be developed some day. But the most of us are interested in the initial versions.
5. Add additional field to the project where the developer can place the link to his demo (= his website).

1. Ajprog is actually doing some work to Module Manager for 2.0 and it will include search capabilities and the help is in there too. Though I still think it would be nice to be able to pull up the Help in the forge. However the help is part of the language files so it may be a little more difficult to put it in the forge.

2. It has been discussed and the thoughts are to put them into an "inactive" state. So default searches won't include them, but you can "Show Inactive" projects doing an advanced search. This would also be used for not only fileless projects but also projects that haven't seen any development for X amount of time.

3. I am not sure how the download statistics are currently handled so I can't speak for Ted. But it would be cool even for the module developer to see more detailed stats on their module. Monthly, Annual, All-Time, etc.

4. I see your point... but I like watching what projects get created. Even if they don't have development/releases for weeks. I think this is where the request to expand the "Recent Releases" list would be bonified.

5. This would have to be thought through... but I don't entirely disagree. My main concern would be QA again. From my experience most developers work locally and don't have their modules on sites with demos or don't update them each time. But for those who want to share... it would be neat. I'm torn 

Advantages:
--------------------------
1. The search time for the functionality will be extremely shortened, not only for end users, but for dev team as well. Help section includes much more information about the module than three line description fields, modules without releases are not shown, so that nobody must spend his time to click through project names without a line of code.

2. The number of support requests for “dead” modules in forum will go down. Support and dev team can invest more time in core development. Questions like “what module should I take to realize that and this” will go down.

3. Actual download statistics gives the module developer the feeling how useful his module is. It encourages developing the module further if you see that you make something useful. Initial releases are exciting for the developers, they pine for feedback, but at the same time they are often overlooked. It is just impossible to track all of new announced projects. Announcing initial release will give the developer instant feedback from the community. Not announcing encourage the developer to leave it, so that it soon abandonend.  Dev team can profit from new useful modules as well.

4. If developers could link to the demo, a lot them would have done it. It is like own advertising, there are a lot of developers who do make good jobs, so why not to give them the opportunity to present themselves instead of treating them as a "number" or anonymous name in forge? I mean active link.

5. Dev team can save time on participation in such “3rd party modules quality” discussions. 

I like this list... especially #5. Just yesterday I was thinking the same thing. 

[Peciura]

I think it is possible to sort votes according user status received from CMSms forum or Forge .
Just an idea to have dev team, dev (module admins could be separated from module and plugin devs ), power poster, peanut votes.
While system is under development all votes could go to peanut section.