Page 1 of 1
Checking for tokenizer functions Failed!
Posted: Sun May 21, 2006 10:35 am
by heltinde
Step 1 in installation process - Checking for tokenizer functions Failed!
What does it mean, and what should I do about it?
I'm trying to install v. 0.13 on the hotserv.dk server.
Thanks
Re: Checking for tokenizer functions Failed!
Posted: Mon May 22, 2006 7:25 am
by lexan
Hi Heltinde
I dont know what that function does, but it means that php is not compiled with that function - webhost B-ONE is the same, annoying !
Just to try it out, I disabled that check in install/index.php, and I was able to install fine, no problems.
Maybe someone here could tell us if that specific function is vital or a securityhole if disabled or what...
regards lexan
Re: Checking for tokenizer functions Failed!
Posted: Mon May 22, 2006 7:45 am
by tsw
tokenizer is used for page preview in admin
Not really sure why it would be disabled but hostin companies usually have some reason (or they have forgot it)
You can ask your hosting why it doesnt have tokenizer enabled
Re: Checking for tokenizer functions Failed!
Posted: Mon May 22, 2006 8:03 am
by heltinde
OK - I'm gonna ask them. If they reply, I'll post the answer here.
Thanx for your feedback.
Re: Checking for tokenizer functions Failed!
Posted: Mon May 22, 2006 8:41 am
by heltinde
--> lexan
Ehh, here's what happend after I took your advise:
Fatal error: Call to undefined function token_get_all() in /var/www/hotserv.dk/users/cmsimple/lib/smarty/internals/core.write_compiled_include.php on line 45
I was able to install just fine, but not to use the system

(cmsimple.hotserv.dk)
Re: Checking for tokenizer functions Failed!
Posted: Mon May 22, 2006 9:30 am
by Ted
Smarty (the templating system we use) uses it pretty extensively. What's strange is that the tokenizer is compiled into php by default, so it seems strange to me that people don't have it. However, I did put that check into it to curb these problems before hand.
Re: Checking for tokenizer functions Failed!
Posted: Mon May 22, 2006 9:55 am
by Piratos
token_get_all() parses the given source string into PHP language tokens using the Zend engine's lexical scanner.
This function needs php >=4.2.0, PHP 5.
But your provider can disable this function.
CMSMS 0.13 cannot installed if this function does not exists ->
Code: Select all
echo "<tr class=\"row2\"><td>Checking for tokenizer functions</td><td>";
if (function_exists("token_get_all"))
{
echo "Success!";
}
else
{
echo "Failed!";
$continueon = false;
}
Re: Checking for tokenizer functions Failed!
Posted: Mon May 22, 2006 11:59 am
by heltinde
So it's easy for my provider to just enable the function again?
Re: Checking for tokenizer functions Failed!
Posted: Mon May 22, 2006 3:45 pm
by Piratos
So it's easy for my provider to just enable the function again?
yes , enable and server restart.
Re: Checking for tokenizer functions Failed!
Posted: Fri May 26, 2006 5:01 am
by heltinde
The end of story:
The people at
www.hotserv.dk are not going to enable tokenizer functions. They will have to recompile php and it's going to give too long downtime, they say.
Sigh, I'll have to look for yet another webspace.
Any suggestions? Frac.dk is still not open to new members - they endure a problem.
Re: Checking for tokenizer functions Failed!
Posted: Fri May 26, 2006 7:14 am
by tsw
ugh, last time I did something like that downtime was something around one minute

(you dont have to even reboot the machine)
well maybe its just me
there is a thread about hosting companies:
http://forum.cmsmadesimple.org/index.ph ... 157.0.html
Re: Checking for tokenizer functions Failed!
Posted: Fri May 26, 2006 7:23 pm
by tambaqui
Hi all,
I would have the tokenizer functions with my web hosting company, but not on my local server. CMSms is the fourth (!) cms I'm going to test during the last week (the others weren't so good), so I'd really like to install it locally first.
I'm using php 4.2.2, but there's not a trace of a tokenizer option in the php.ini. How do I activate it? Or do I need a newer php version?
Thanks for an answer!
Rainer
Re: Checking for tokenizer functions Failed!
Posted: Fri May 26, 2006 10:13 pm
by tsw
You may need to recompile php
check out this page
http://php.net/tokenizer
Beginning with PHP 4.3.0 these functions are enabled by default. For older versions you have to configure and compile PHP with --enable-tokenizer. You can disable tokenizer support with --disable-tokenizer.
Re: Checking for tokenizer functions Failed!
Posted: Sat May 27, 2006 7:48 am
by tambaqui
Well, testing CMSms online may then be the easier option...
Thanks for your quick answer,
Rainer