finally solved a terrible bug

two weeks ago i encountered a *terrible* but that marked my entire rendering system void. basically (without going into all the upsetting details), any item delivered from my cache was causing errors in MSIE (Switch from current encoding to specified encoding not supported.") and forced FF/Safari to render the plain XML instead of the resulting web document. *crap!*

i struggled with various work arounds trying to isolate the problem, but could *not* find the answer. i am using a mix of XML from the database, XML from a template document and XSL transforms. add to that a sprinkling of caching the generated document to disk for quick reply and i was having a devil of a time sorting it all out.  in fact, after several days of no luck, i just put the thing aside for about a week. 

well, i started in again last night and - although i got almost no where last night, the solution finally hit me this afternoon. it *was* an encoding issue, but not where i thought it was. not in the database data, not in the template xml, not in the cache file itself, but in the xsl transform!  i noticed that a few of my templates worked just fine. but one set (my blog templates) did not. the answer? my transforms for the blog site was **missing the xml version/encoding declaration at the top of the page**!

yep - bonehead play. i was too quick to build my blog templates (in a single night about three weeks ago) and did not fully test them at the time. they worked while i had caching turned off (i do this while developing). it was only after i tunred cahcing back on that the problem came up. apparently, the encoding declaration in the transform is critical to the way the final document is output and (in my case) stored to disk.

once i sorted this out, things are working quite well (sigh).  of course, now i will need to do some serious testing to make sure i reach into all the corners, but it's much better now.

man - what a bonehead!

refining the new dispatch model

i made some additional progress on the new dispatch model for the framework. i now have the templateHandler in the original Exyus assembly. i also have a separate Exyus.Blog assembly to handle the blogging details. i had to tweak some regex string in order to fix a bug with the arg-handler, too.

now, i just need to press the edges some more over the next few days to make sure it's all cool. all looks good. but i could proly get a lot more out of this setup if i had better regex chops.

next step - implement the other template pages for the blogging app.

two big breakthroughs this weekend

i made two critical breakthroughs this weekend on the framework:

- support for passing arguments in xml documents

- simplified dispatch pattern for the handlers



argument replacements in xml templates

first, i can now support a simple argument pattern within xml documents that are treated as templates. for example:



<x:include href="/xcs/data/weblogs/?category=$category$" />



will match the ?category=1 from the request url and pass it along to any $category$ with the value of "1."



i also added a $_qs$ argument which represents the *entire* query string. so far, this has worked well for passing filters to the data-oriented requests. i'll continue to beat this up over the next several days to make sure it's all working as expected and without nasty side effects.



simple dispatch pattern

thanks to some serious inspiration from brad wilson, i now have a much more scalable dispatch pattern using a custom attribute and reflection. now, instead of having to craft a special handler factory to respond to all xcs calls, i have a single factory that can san all loaded assemblies for eligible handlers/factories that can respond to requests.



it works by decorating my handler (or factory) class with a new attribute:



[UrlPattern("/xcs/data/weblogs/(.*).xcs")]



and there's a single factory class that scans all loaded assemblies and catalogs all classes with this attribute. then, upon each request, the collection is checked and, if a match is found, the associated class is loaded and executed for that request.



again, i need to beat it up a bit to make sure it covers all the bases, but - so far - this is working quite well.  hopefully, it will scale well as the number of handlers increases for a single web app - time will tell.



so, improved dispatching and solid arg-passing for xml templates (btw - i have solid xml templates!). that covers another set of important features.



i'm now very close to completing the templates for the blog. once that is done, i need to put together the editor tools.

improved the date filtering for the blog

i added a number of new 'reserved' words to date queries for the blogging app.



today

thisweek

thismonth

thisyear

yesterday

lastweek

lastmonth

lastyear

ayearago



now you can query the data like this:



/data/weblogs/?thisweek



and see all the posts on file for this week



not bad



i also finally sorted out support for secured templates. i've created a new space /templates/ that requires http auth - right now only the site admin has it. and the runtime uses siteadmin rights to load anything via includes. that means that any direct calls by clients will prompt for user auth. any *internal* calls via includes, etc will always be decorated with siteadmin user/pass and will work without trouble.



pretty slick!



i plan on adding a robots file to prevent any attempts to crawl that space, too.



i also did some work a a simple template this past week. it's pretty clean. i think it will work for starters. now i just need to sort out the final details for the display side and implement it locally. proly take a day.



then i need to add the editing UI. i think i need to step up and use the dhtml components i've been work on at the office (html edit, calendar control, sort/page table, tabs, etc.) this should make the ui more interesting with less hackery, too.



so i'll proly spend a few days getting those all working smoothly, too.



it's inching along.

the value of 3xx

in my reading of the rest-discuss list and digging around in general, i am starting to realize that the 3xx response code can be very valuable in dealing with proper handling of situations.



301 - moved permanently

does iis handle this at all?



302 - moved temporarily

shuttle the request to another uri for this moment - iis handles this one (not always well)



303 - see other

commonly used to redirect POST to another page - not using this at all, might be quite handy



304 - not modified

the resulting request has not changed since you last asked - i use this alot now



gotta get my hands into the 303!

stlled, but learning

i'm kinda stalled on upgrading my rest-ian framework. the initial templating system is working, but i need to iron out and sand off the rough edges on my security implementation for embeded templates (templates calling templates...).  it should be working fine in a few days.



in the meantime, i helped jesse implement another feature on his web site. although he is not using the 'full-blown' rest-ian model, most of the recent changes i've implemented for him use rest-like patterns and lots of xml/xslt work. he's actually getting the hang of working with xsl transformations, too.



finally, i stumbled upon a couple related items on a rest email list this week and it's got me thinking:





Megginson Technologies: Quoderat

Lost Update Problem. This doesn’t have to do with REST per se as much as with the choice not to use resource locking, but since REST people tend to like their protocols lightweight, the odds are that you won’t see exclusive locks on RESTful resources all that often (it also applies to some kinds of POST updates as well as PUT).

Reliable delivery in HTTP

Achieving reliable delivery in HTTP is not difficult. It takes a little bit of understanding of how HTTP works and what reliable delivery means.

both of these items point out common problems with any remoting/async system. and they both have very simple solutions!  i really like the version id solution for the lost update. i am still working on the details of reliable delivery using POST (or RPOST).



but they bothremind me that there are a number of additional things to consider before i claim a 'complete' framework!

more on hypermedia and the engine of app state...

been running into several other posts that relate to this issue (see posts below)



one thing that is starting to sink in is that the hypermedia aspect relates closely media types.  in a nutshell, if replies from the server express media types and links, i should be able to figure out what kind of representation is allowed/expected at each link (end-point) along the way.  if a system (client) understands a collection of media-types, it stands to reason that the client can make GET to the server to retrieve a list of links and media types and then be able to compose a PUT/POST based on the media-type and pass it to one or more of the links.  i think i'm getting it...



An Opinion? Well, if you ask...: D'oh! REST already had contracts.

The contracts and protocol exist solely in the data. A client that understand a media type can then navigate the web of links based on knowledge gleaned from documents conforming to that media type. (This is exactly hypermedia as engine of application state).

Joe Gregorio | BitWorking | Do we need WADL?

Everybody's atwitter about WADL, a description file for REST services, and since it's supposed to be RESTful I regularly get questioned about it.

Shopping - SimpleWebServices

In the REST world, shopping revolves around application state - transitioning as required.

hypermedia as the engine of application state

i don't yet 'grok' the last one in the list:



REST is defined by four interface constraints:

- identification of resources

- manipulation of resources through representations

- self-descriptive messages

- hypermedia as the engine of application state





i'm learning, but i'm not quite there yet.

big leap forward today

i made a big leap forward in the rest-server framework today. i added support for using xml server-side templates that support x-include and other good stuff. that means it's possible to define an xml template that includes several calls to the /data/ portion of the system and then apply a transform to it on the server. it generates clean xhtml and sends it to the client (optionally caching it on the server, too).



all very sweet.



i ran into a few (not unexpected) hurdles along the way. first, since the /data/ part of the system is secured, templates calling into the /data/ area always failed rudely unless the user had already authenticated. i decided to implement a server-side system-user account that can access the data quietly as needed. this is the road of least-friction. i also plan on implementing a version that will force the user to log in first. that will take a bit of doing. i can easily write an x-include fallback, but i also need a clean way to pass credentials (i think) once the user has logged in. it's on my list of things to check out.



another stinger was the whole xml:base resolution deal. i decided to keep my xml templates in a folder not directly accessible to external users (prevents hacking). but that meant for each request, i needed to 'adjust' the internal path to point to the template pile. that - in turn - broke the xml:base for the x-includes. as a result, i now diddle the url as it comes in to make sure it matches the one for the intial template. now it all seems to work fine. this may not be the correct solution, but it works.



i also added suport for xhtml validating of user input on the server. i use the built-in validator for xml, but using a DTD/Schema instead. works great. now i can make sure saved data does not end up hosing the xml/xsl templates. i did run into a bit of trouble, tho. seems my escapeHTML javascript hack converts the input to HTML (not xhtml) each time i *read* it in. this is a bummer. i need to check into that since it means each time i edit an existing record, all my cool xhtml changes are removed.



finally, i did a bunch of code clean-up and some object renaming. i've been meaning to do that for quite a while, but just never took the time. now it's much cleaner (references to amundsen and REST are now gone) and i've moved a few things from local vars into const and such.



so, i have a solid templating solution started. i need to beat it up a bit before i will rest, tho [grin]. i figure building the public side of the weblog app will gfive me enough chance to expose weaknesses in the model.



onward and upward!