TikiURLt

LATEST NEWS

3rd December 2006

I've moved all the sites I'm responsible for to Drupal, so am not doing any active work on Tiki (or URLt) any more. However, if you use this mod and want to ask a question, feel free - I'll do my best to help you.

22nd May, 2005

Tiki URLt Version 1.0 is now ready to go! Download it here

It's had lots of work done to it, and has the following main features:

• Super-fast HTML parsing and URL translation
  
• Advanced cached database reads for greater efficiency (configurable!)
  
• Handles errors gracefully (eg. Page Not Found, Translation Error)
  
• Can handle non-PHP content (configurable MIME types)
  
• Works on any web server (Apache, IIS, etc) or can be used with Apache mod_rewrite
  
• Written to the latest Tiki guidelines (ie. ADODB, PHPdoc, etc)
  
• Tested and working on Tiki 1.8.5 and 1.9.0 (but should also work with other versions)
  

Now it just needs to be part of the Tiki distribution ;-)

See TikiURLtInstall for install details.

Reference sites:

• http://www.coofercat.com (this site!)
  
• http://www.pre-emptive.net

What is Tiki URLt? How does it work?

Tiki URLt is the name given to the process of transforming URLs within the application. The intention is to make URLs look different from the way they would normally look. For example, by default, Tiki would have a URL for this page that looks like this:

http://www.coofercat.com/tiki-index.php?page=TikIURLt

However, in fact it looks like this:

http://www.coofercat.com/wiki/TikiURLt

This is because the URLt process has changed it. This change, or transformation, is a two part process. The first is to change the URL on it's way out of Tiki. That is, the Tiki application embeds URLs in the HTML it sends to the browser. The URLt process intercepts this HTML and transforms the embedded URLs. Thus, the links the user sees on a page are of the transformed type, not the default type.

The second part of the problem is when the user clicks a transformed URL. Clearly the transformed URL doesn't actually exist on the server, so the URLt process has to intercept the request as it comes into the server and transform it into the default URL style. The Tiki application then takes over, processing the request as it normally would.

Okay, so how does it actually do it?

Tiki URLt is three main pieces of code. These are:

• Outgoing URLt: A Smarty Output Filter
  
• Incoming URLt: A PHP wrapper
  
• Administration: A few forms that allow configuration of URLt

Outgoing URLt: A Smarty Output Filter

Tiki uses the Smarty template engine to control it's output. Smarty can have a number of "output filters" installed. These filters run after the HTML has been created and is about to be sent to the browser. The filters have access to the HTML, so can modify it as they wish.

The URLt filter contains an optimised (cut down) HTML parser that looks for HTML links embedded in the HTML (ie. within a...href, form...action, img...src tags, for example). A cut-down parser is used because it is much faster then a full-blown HTML parser (of which there are a few for PHP). As links are found, they are passed through the transformation process, the result of which is put back into the HTML in place of the original URL. (see below for more on the transformation process)

Incoming URLt: A PHP wrapper

Incoming URLt presents somewhat of a problem: How to intercept requests from browsers? There are a number of solutions to this problem, most notably using Apache mod_rewrite rules to change the incoming request into the real location of resources. This is of course Apache specific, and requires access to the Apache configuration, which precludes shared hosting users from using this faclity.

Tiki URLt uses a PHP wrapper instead. Here, the web server must pass all requests to a single location (the PHP wrapper). This can be achieved in a number of way (see below), but all web servers can do this in some way, and in most cases access to the server configuration is not required.

Since the PHP wrapper is called for every request, it effectively takes over the role of the web server. That is, it has to transform the URL and then locate the document that is required. In the case of a PHP document, it has to execute this PHP, but make the environment "look" as if the PHP had been called directly. Thus, the PHP that actually handles the request is unaware that it has been called by URLt.

In the case of non-PHP documents (eg. images, CSS, etc) the PHP wrapper has to look up the MIME type of the document. It then has to construct suitable HTTP headers and return the document. Thus, the client is unaware that URLt has taken place - the response is exactly as if the document had been retrieved directly.

Administration

Administration of the URLt process is performed by three forms. The first is general configuration, which covers the following:

• Is incoming and/or outgoing URLt enabled?
  
• URLt database read cache times
  
• Default MIME type
  
• Default index page name

The remaining two forms are almost identical (one for incoming, one for outgoing). They allow regular expressions to be entered that will perform transformations on URLs. Each expression is called a "rule". Each rule can have an optional textual comment added as an aid to memory and ease of administration (regular expressions can be notoriously hard to understand!). Each rule can also have a number of "modes". For example, the URLt process (see below) can continue processing after using the rule, or stop processing further rules. Rules can also be disabled for testing purposes.

The URL Transformation Process

As URLs are found (either in outgoing HTML or in incoming requests) they are passed to a transformer. The transformer works on "rules". Each rule is a regular expression substitution, with optional regular expression flags (e.g. i = case insensitive) and a "mode". The mode can be one of:

• Continue
  
• Stop on match
  
• Stop on no match
  
• Disabled rule
  

If the mode is "continue" then rule processing proceeds to the next rule, regardless of the outcome of this rule. Alternatively, the transformer can stop procssing further rules if either a match is found, or if no match is found. This allows for significant optimisation of transformer execution. Finally, the transformer will skip over any disabled rules which can be useful for testing and development of rules.

With careful use of rules and modes, complex transformations can be performed on URLs. The use of "continue" means that a single URL can undergo numerous changes, stage by stage, depending on it's exact makeup. The ability to stop transformation means that careful rule ordering can not only optimise transformation, but also perform rudimentary exception handling.

Web Server Configuration

To use incoming URLt, the web server has to be configured to direct all requests to a single location (outgoing URLt does not require any web server configuration). Web server configuration is of course server specific, and access to the configuration is site specific. Readers are advised to check with their systems administrators or providers for details.

Apache server users can direct all requests for an area of their site to a single location in a number of ways. Here are two, the first using mod_rewrite, the second using core Apache functionality:

Example 1 - mod_rewrite

RewriteEngine on

 ReWriteCond %{REQUEST_URI} ^\/wiki\/<br />
 ReWriteRule \/wiki\/(.*)$                      /tiki/tiki-urlt.php [[L]71

Here, any requests to /wiki/ will be directed to the file /tiki/tiki-urlt.php (below the DocumentRoot). This is possibly the best way to achieve the required functionality

Example 2 - ErrorDocument

<Location /wiki/>

  ErrorDocument 404 /tiki/tiki-urlt.php<br />
 &lt;/Location&gt;77

Here, the /wiki directory has to be created inside the DocumentRoot (but left empty). Since no requests can be satisfied, they will all result in a 404 error, which the web server will call /tiki/tiki-urlt.php to handle. This method is less than ideal because the web server has to perform slightly more work than the mod_rewrite method. Also, the web server will log a "page not found" message in the error_log for each request. However, this is a quick way to achieve the required functionality and can usually be used in shared hosting environments (by using a .htaccess file in the /wiki directory).

Current Known Issues

(1) The HTML parser in the output filter does not handle JavaScript links correctly. This is not ideal, but it is unlikely the parser will ever really be able to translate links embeded in JavaScript.

(2) Smarty seems to submit the same template information multiple times. This means that URLs are double-translated, so the "stop on match" facility is somewhat diminished in value. A Tiki bug has been raised for the problem, although it may be a broader Smarty issue.

(3) Tiki programs that return HTTP redirects (ie. "302 Location:" headers) are not available to the output filter. As a result, the redirects do not bear translated URLs. It is unclear how to solve this issue (are outgoing headers available to PHP4? If so, how!?)

(4) Numerous Tiki programs do not output consistent URLs. For example, a blog page post is identified as "tiki-view_blog_post.php?blogId=1&PostId=1". However, the link "back to blog" is "tiki-view_blog_post.php?find=&blogId=1&PostId=1...". Whilst only a slight difference in ordering, and no doubt easily done, this causes immense problems for URL translations. It is unclear how this can be overcome in a sensible Tiki-wide way.

(5) Blogs are a real pain in the neck for SEFURL work. I'm not sure what the solution is, short of some major blog hackery.

(6) Actually making regex transformations work is a really difficult process. Users require intimate knowledge of regexes, and how web pages are delivered and how requests are made. As a result, this is a very advanced feature, which only the very cunning should consider!

Legal Stuff

Do what you want to use this information? Let me know if there's anything wrong with it, but by all means use it for (real!) Tiki documentation if URLt ever makes it into Tiki for real.