next steps for exyus

now that things are pretty stable (codebase-wise), i'm putting together a list of 'next step's for the framework. here's what i have so far:

• add support for feeds
  this will include publishing a feed from the blog (or other data source) as well as the ability to consume feeds from other sources. consuming feeds will also mean support for caching the feed locally and transforming it as a standard data source. 
  
• add support for external editors
  most likely this will be in the form of the metaWeblog API. i figure the ATOM format is really more 'geek-friendly' but the metaWeblog API has been around for several years and there's lots of support for it 'in the wild.' I'm thinking of using MSFT's Live Writer as a test-bed for my implementation, too.
  
• add support for handheld posting
  this would allow me to post from my moto-q phone or any other html-aware small form-factor device. i think this means a pure html approach (no javascript). shouldn't be too much of a problem. the authentication might take some thinking (a simple user/password screen that does the real basic auth on the server?), tho. also, it will be a post-only pattern (no editing existing posts, deleting, etc.) that does not support markup (no p-tags, et al).
  
• add support for DIGEST auth
  this has been on my list for a while. it'll be a bit gnarly (my BASIC auth pattern is kinda buried into the code now) so i've put it off for a while. but i have a working example of DIGEST support for C#/ASP.NET already. so it'll be a relatively boring (i hope) day or two of hacking up the code and getting all the bugs out.
  
• move users and auth to db
  current the user list (small) and permissions list (kinda growing) is stored in two XML files. this works well for now and is easy to work with (and cached, too). eventually, this should be moved to the SQL-DB, tho. not a big rush on this one. it's another example of something that will happen behind the scenes.
  
• add SSL support
  first, i need to post the code to a public server. second, i need to get a simple SSL cert installed. finally, i need to make sure my existing code works well with SSL. should be no big deal. the only 'magic' might be getting the exyus framework to force SSL for certain URLs. not sure if that's needed, but i can see benefits. 

well, that should keep me busy for a while!

sweetening exyus

i did more work this week to sweeten the exyus framework.

304 fixed

first, i fixed my broken 304 handling. when i was storing all cached data on disk, the 304 handler would compare the file timestamps against the if-modified-since and etag headers (serialized) to determine if 304 was the proper response. when i moved the cache into memory, my 304 checking was broken. i would return data from the cache, but always return false for 304 checking (no file data anymore!). to fix this, i resurrected a simple cache object that has timestamp, etag, and payload properties. now all resource entries in the memory cache are cache objects and my 304 checking is working again. sweet!

support for HTTP HEAD

second, i added support for the HTTP HEAD method. actually, it was embarrassingly easy. i just call the GET method and then toss out the response content. this allows for all the usual good stuff (304 checking, marking the item w/ last-modified, and etag, etc.), but doesn't return a body. now i have solid HEAD support!

cache invalidation implemented

my other big task was to implement a simple cache invalidation pattern. i did thsi by implementing support for the cache-control:no-cache header in my GET requests. that means anyone can issue a GET request to a URI with the cache-control:no-cache header and that will force the system to invalidate any existing cached version of that resource and return a fresh one. of course, along the way, this fresh version will get placed in the cache for future calls! *and* (to make it all really nice), you can even use HEAD instead of GET!

simplify data object declaration

finally, i tightened up the initial declaration for data object. no longer do you need to declare each verb XSD/XLST set. you just supply a single directory location that holds the [verb].xsd and/or [verb].xsl files. this emphasizes the standard interface concept for HTTP/REST and simplifies/shortens the declaration code. now, a data object declaration looks like this:

    [UrlPattern("/data/websites/(.*).xcs")]
    public class websiteData : SqlXmlHandler
    {
        public websiteData()
        {
            this.ConnectionString = "mamund_personal_db";
            this.UrlPattern = @"/websites/(.*).xcs\??(.*)$";
            this.DocumentsFolder = "~/documents/websites/";
            this.XHtmlNodes = new string[] { "//description" };
            this.ClearCacheUri = new string[] 
                    { "/blogging/", 
                      "/blogging/websites/", 
                       "/blogging/websites/{@id}" 
                    };
        }

getting to the next level

it's been a while since i posted here, but that's just 'cuz i been bizzy!

finally getting the exyus framework distilled to the important bits. i now have a single pageHandler class that handles standard HTML GET activity. it can be sub-classed and declaring a new page in the site looks like this (C#):

    [UrlPattern(@"/blogging/.xcs\??(.*)$")]
    public class home : pageHandler
    {
        public home()
        {
            this.TemplateXml = "/blogging/index.xml";
            this.TemplateXsl = "/blogging/index.xsl";
            this.MaxAge = 600;
        }
    }

note i only need an xml document (that supports x:include, btw) and a transform (that supports exslt, etc.) and i'm rockin'

i also added a dataHandler class today. this can be used to produce XML resources that respond to the standard GET POST PUT DELETE interface. again, i distilled it down to a small bit of code that leverages XML,XSD, XSLT and a dash of TSQL. here's a typical class:

    [UrlPattern("/data/weblogs/(.*).xcs")]
    public class weblogData : dataHandler
    {
        public weblogData()
        {
            this.ConnectionString = "mamund_personal_db";
            this.UrlPattern = @"/data/weblogs/(.*).xcs\??(.*)$";
            this.XHtmlNodes = "//body";

            this.PostXsdFile = "~/documents/weblog/weblog-post.xsd";
            this.PostXslFile = "~/documents/weblog/weblog-post.xsl";
            this.PutXsdFile = "~/documents/weblog/weblog-put.xsd";
            this.PutXslFile = "~/documents/weblog/weblog-put.xsl";
            this.DeleteXslFile = "~/documents/weblog/weblog-delete.xsl";
            this.GetXslFile = "~/documents/weblog/weblog-get.xsl";
        }
    }

again, it's all about creating the proper schema and transform files. of course, there' some T-SQL in the background, but that's another story.

the point here is that it's now pretty simple to create a web site that supports all the standard (X)HTML stuff as well as acts as a good HTTP netizen. most of this is following the REST model (with liberties along the way).

finally, i implemented a no-cache pattern last week. now, if you do a GET using the cache-control:no-cache header, exyus will ignore any cached value and rebuild the request (caching it along the way, if called for).  using the header is a bit cumbersome for html clieints, but works fine for scripting and ajax work.

i need to exercise the new upper-level classes (dataHandler mostly) but they should be solid. next, i want to work more on the requestor class (to make internal http requests) and then look at wrapping things up for the first post to a public server!

finally solved a terrible bug

two weeks ago i encountered a *terrible* but that marked my entire rendering system void. basically (without going into all the upsetting details), any item delivered from my cache was causing errors in MSIE (Switch from current encoding to specified encoding not supported.") and forced FF/Safari to render the plain XML instead of the resulting web document. *crap!*

i struggled with various work arounds trying to isolate the problem, but could *not* find the answer. i am using a mix of XML from the database, XML from a template document and XSL transforms. add to that a sprinkling of caching the generated document to disk for quick reply and i was having a devil of a time sorting it all out.  in fact, after several days of no luck, i just put the thing aside for about a week. 

well, i started in again last night and - although i got almost no where last night, the solution finally hit me this afternoon. it *was* an encoding issue, but not where i thought it was. not in the database data, not in the template xml, not in the cache file itself, but in the xsl transform!  i noticed that a few of my templates worked just fine. but one set (my blog templates) did not. the answer? my transforms for the blog site was **missing the xml version/encoding declaration at the top of the page**!

yep - bonehead play. i was too quick to build my blog templates (in a single night about three weeks ago) and did not fully test them at the time. they worked while i had caching turned off (i do this while developing). it was only after i tunred cahcing back on that the problem came up. apparently, the encoding declaration in the transform is critical to the way the final document is output and (in my case) stored to disk.

once i sorted this out, things are working quite well (sigh).  of course, now i will need to do some serious testing to make sure i reach into all the corners, but it's much better now.

man - what a bonehead!

refining the new dispatch model

i made some additional progress on the new dispatch model for the framework. i now have the templateHandler in the original Exyus assembly. i also have a separate Exyus.Blog assembly to handle the blogging details. i had to tweak some regex string in order to fix a bug with the arg-handler, too.

now, i just need to press the edges some more over the next few days to make sure it's all cool. all looks good. but i could proly get a lot more out of this setup if i had better regex chops.

next step - implement the other template pages for the blogging app.

two big breakthroughs this weekend

i made two critical breakthroughs this weekend on the framework:

- support for passing arguments in xml documents

- simplified dispatch pattern for the handlers



argument replacements in xml templates

first, i can now support a simple argument pattern within xml documents that are treated as templates. for example:



<x:include href="/xcs/data/weblogs/?category=$category$" />



will match the ?category=1 from the request url and pass it along to any $category$ with the value of "1."



i also added a $_qs$ argument which represents the *entire* query string. so far, this has worked well for passing filters to the data-oriented requests. i'll continue to beat this up over the next several days to make sure it's all working as expected and without nasty side effects.



simple dispatch pattern

thanks to some serious inspiration from brad wilson, i now have a much more scalable dispatch pattern using a custom attribute and reflection. now, instead of having to craft a special handler factory to respond to all xcs calls, i have a single factory that can san all loaded assemblies for eligible handlers/factories that can respond to requests.



it works by decorating my handler (or factory) class with a new attribute:



[UrlPattern("/xcs/data/weblogs/(.*).xcs")]



and there's a single factory class that scans all loaded assemblies and catalogs all classes with this attribute. then, upon each request, the collection is checked and, if a match is found, the associated class is loaded and executed for that request.



again, i need to beat it up a bit to make sure it covers all the bases, but - so far - this is working quite well.  hopefully, it will scale well as the number of handlers increases for a single web app - time will tell.



so, improved dispatching and solid arg-passing for xml templates (btw - i have solid xml templates!). that covers another set of important features.



i'm now very close to completing the templates for the blog. once that is done, i need to put together the editor tools.

improved the date filtering for the blog

i added a number of new 'reserved' words to date queries for the blogging app.



today

thisweek

thismonth

thisyear

yesterday

lastweek

lastmonth

lastyear

ayearago



now you can query the data like this:



/data/weblogs/?thisweek



and see all the posts on file for this week



not bad



i also finally sorted out support for secured templates. i've created a new space /templates/ that requires http auth - right now only the site admin has it. and the runtime uses siteadmin rights to load anything via includes. that means that any direct calls by clients will prompt for user auth. any *internal* calls via includes, etc will always be decorated with siteadmin user/pass and will work without trouble.



pretty slick!



i plan on adding a robots file to prevent any attempts to crawl that space, too.



i also did some work a a simple template this past week. it's pretty clean. i think it will work for starters. now i just need to sort out the final details for the display side and implement it locally. proly take a day.



then i need to add the editing UI. i think i need to step up and use the dhtml components i've been work on at the office (html edit, calendar control, sort/page table, tabs, etc.) this should make the ui more interesting with less hackery, too.



so i'll proly spend a few days getting those all working smoothly, too.



it's inching along.