somewhat daily mutterings

/Programming/Projects/Gemcast Gemcast 0.1.0 Available

I've done some major refactorings in gemcast, the Ruby-based weblog engine that runs this site. I've rewritten some pieces, and added new features here and there.

The two main new features are a "Top" entry for the Categories sidebar, and the ability to display a single weblog entry, like this. This latter feature I'll be able to use for "real" permalinks (although they'll only be permanent if I don't move or rename my blog entries), rather than the anchor hack that I was using before (and that stopped working as soon as the entry scrolled off the bottom of the blog). As a side-effect of the latter, I've also started down the road of making gemcast's URLs compatible with blosxom's URLs, which is reasonable, given that gemcast is a ruby-based knock-off of blosxom.

Read more about gemcast here.

Download gemcast 0.1.0

Posted: Sun Aug 14 17:25:27 -0700 2005

/Programming/Projects/Gemcast New Dependency Graph, Produced with Dot and Graphviz

You may be aware that I recently was turned on to Graphviz and the 'dot' graph representation language (see my SpringViz page). Well, tonight I turned my attention to graphing the dependencies among the Ruby modules that make up gemcast (the weblog engine that I wrote for this site).

gemcast Dependency Graph, rendered by Graphviz

For now, I'm using a simple Unix script to generate the dot file (that's the way I almost always start this sort of thing, and it literally took me about 5 minutes to put together), but I'll convert it to Ruby soon in order to add more powerful rendering. Here's the script, in case you're interested:

1 echo "digraph {"
2 echo "  node [ shape = box ]"
3 grep require *.rb | sed 's/\:/ /g;s/"//g;s/\.rb//g;s/require/->/g'
4 echo "}"

Posted: Sun Oct 03 17:10:05 -0700 2004

/Programming/Projects/Gemcast What's Cool about a Blosxom-style Weblog...

... like mine (gemcast, which is written in Ruby), is that it's filesystem-based. Doesn't sound very impressive, but because of the filesystem approach, it took me only about 10 minutes to hack together a little cron job to produce the "SpringViz Download Count" sidebar box you see at the right (when you're at the top of the weblog). It updates every 15 minutes. Sweet. The Google referrals item was also done this way.

As a side note, it blows me away that I've been running my blog with gemcast for a year and a half now, and haven't felt a burning need to make do more. I guess I'd really like to have a "single-entry" view, but it hasn't been important enough to get me back into hack mode.

Posted: Wed Sep 15 19:58:22 -0700 2004

/Programming/Projects/Gemcast Gemcast 0.0.5 Available

I did some minor refactorings in "gemcast", the Ruby-based weblog engine that runs this site. I didn't do anything major - I hadn't visited the code in some time, so I basically just adjusted and simplified a few things, and improved the documentation, to reacquaint myself with the code.

Read more about gemcast here.

Download gemcast 0.0.5

Posted: Thu Jan 01 20:30:57 -0800 2004

/Programming/Projects/Gemcast FYI: Lines of Code for gemcast v0.0.4

Stats produced by the 'loc' script included with the gemcast distro:

Including def/end
Excluding def/end

Posted: Sun Jun 22 00:25:54 -0700 2003

/Programming/Projects/Gemcast New gemcast Features!

OK, I'm excited. I finally got around to adding a couple of new features to gemcast. First, each entry's Category line is now a link so you can jump to that particular category and read only that category's content. I started implementing this in code when I realized that I could do it via the template if I'd only publish the site's base URL as an template replacement attribute. So, it still took an additional one line of code, but obviously nothing fancy.

Secondly, I finally got around to "fixing" my Category sidebar box. Rather than show only the categories that are under the current category (which I think was a bit confusing), it now always shows all the categories available on the site, regardless of your location on the site. The result is that the site's structure much easier to see. This mainly took a little adjustment of the FileStore#getAllCategories method. Maybe one day I'll fix the display so it will look like a pretty hierarchy, instead of a filesystem dump (which is basically what it is).

These changes comprise gemcast v0.0.4. You can download the source here.

Posted: Sat Jun 21 22:02:49 -0700 2003

/Programming/Projects/Gemcast Major gemcast Refactoring

I've been meaning to add new features to gemcast (the engine that runs this weblog) for some time now. But in order to add the features, I've really needed to revisit the overall structure of the current code. That bit of work has held me back -- until today. I stayed home with a funky back (and therefore really wasn't up to driving), but later in the day I felt pretty OK, and decided to crack open gemcast again and see what progress I could make toward future enhancements.

It turned into a pretty major refactoring, with the introduction of a couple of new "modules" (not in the formal ruby sense, but in the sense of .rb files containing related classes - very loosely analagous to Java packages), and the deletion of a couple more, all in the interest of cohesion and dependency management. Over time, I've reflected on some problems that the original gemcast had (and still has, to some degree), which allowed me to make some pretty decent changes. This release has no new functionality, but is cleaner than the previous release, and hopefully gives me some room to do what I want in the future. Version 0.0.3 is available here.

Here's a dependency graph of the new version of gemcast:


Posted: Wed Jun 11 19:44:04 -0700 2003

/Programming/Projects/Gemcast Fix to "Recent Googles" Code

Apparently, google has some other formats for queries in its URL parameters. Sometimes you get an 'as_q=' or 'es_q=' (or even both, apparently). I assume that's from someone filling in the advanced search form. The result of attempting to process queries like this is a blank entry in my list. Because regular expressions aren't great for dealing with n occurences of a given string (or maybe more because my regular expression skills aren't up to it), I decided to put in the following hack:

grep '\.google.*search.*[?&]q=' /var/apache/logs/referer_log

This bit of hackery forces the grep to find only searches that don't use the special query parameters. Cheesy but effective.

Posted: Sun Apr 27 22:16:45 -0700 2003

/Programming/Projects/Gemcast Implementing the "Recent Googles" Sidebar Box with a Shell Script

The other day I turned on referer logging on my Apache instance. Almost immediately, I noticed Google searches that had led to my site. I though to myself, "wouldn't it be cool if I could display a sidebar box on my site containing the last n Googles that had led to my site?" "Yes", myself replied.

gemcast, the weblogging software I wrote for this site, has a feature that looks for '.box' files in its content directories. When it finds a '.box' file, it creates a sidebar box for the page it's building. Simple. So, about a half-hour later I had a shell script that generates a list of the last ten Google searches, along with the content that makes the output good for a .box entry. Then I created a cron job run this script every 15 minutes and send its output to my root gemcast content directory, so it would display on the "top" page.

Here is the script:

1  echo "10 Recent Googles Leading to"
2  grep '\.google.*search' /var/apache/logs/referer_log |
3    awk {'print $2'} |
4      sed 's/http.*q=/<li>/;s/%22/"/g;s/\&.*$//' |
5        uniq |
6          tail -10
7  echo "Generated on `date`"

For those of you who aren't Unix script hackers, a line-by-line explanation is in order:

  1. Print the title of the box.
  2. Search the apache referer log for lines containing Google search URLs.
  3. Use 'awk' to extract the 2nd field, which is the URL. Given that I'm just using awk to extract a field, I could have used 'cut' as well, but I'm much more familiar with awk's syntax.
  4. Use 'sed' to replace the first bit of the URL, up to the 'q=' query string, with HTML list item markup, replace '%22' with a '"' character, and eat off everything after the query parameter. The result is the encoded query parameter, prefixed with an '<li>' HTML element, like so: "<li>this+is+the+query".
  5. Strip out duplicates.
  6. Only show the 10 most recent queries.
  7. Print the time that the script was run.

Note that lines 2-6 constitute a pipeline -- each command's result is fed into the next command as input. In reality I could have used awk to the entire script. However, that would have required me to write a much more sophisticated awk script. I'd rather string together Unix commands that do a single job (or few jobs) well. To me, it makes the script more obvious, and since I know the Unix commands pretty well, I was definitely done more quickly than if I'd have to hack out and debug an awk script. Also, if I was going to do an awk script that complicated, I might as well use Ruby.

Posted: Fri Apr 25 05:29:01 -0700 2003

/Programming/Projects/Gemcast Permalinks Added

I've modified gemcast to generate "permalinks" for each entry. It was pretty easy, but it's not perfect -- if the item to which a permalink refers has "scrolled off" the weblog, then the item isn't displayed. To fix this I would like to introduce a "single entry" view. I tried to get that working tonight, but it turns out that it will require some refactoring that I'm just not ready to take on at the moment.

Posted: Fri Mar 07 21:32:12 -0800 2003

/Programming/Projects/Gemcast gemcast Now has a Max Entries Setting

My pages were getting a bit long because gemcast had no way to configure the maximum log entries to render for a given category. So, I quickly hacked this feature in. I used a range against the returned set of log entries to select off only the first 'n' entries. This is a quick and dirty, inefficient hack because gemcast currently reads in every entry, even those that won't be rendered. At the moment I have to do it this way, because I'm sorting all the entry objects after they're collected, rather than pre-sorting the list of entry files and processing only the top 'n'. I'll get around to fixing it sometime.

Posted: Wed Mar 05 21:42:34 -0800 2003

/Programming/Projects/Gemcast gemcast needs RSS Support

Some time ago, I whined that I felt ignorant for not knowing what RSS is. I've done something about that, and now I realized that gemcast must be modified to provide an RSS feed. Somehow, it strikes me as profoundly arrogant to assume that anyone would syndicate my own personal RSS feed. But gemcast, to be taken seriously, still needs the capability so that others can make feeds available for their important stuff. Dang, maybe it's profoundly arrogant to assume that anyone will ever run gemcast, too. I can't win.

Here's what I need to do to get there:

  • Generalize the template stuff so that you can have "named groups" of templates. These map to what blosxom calls "flavours", and I think I'll stick with that nomenclature (and anglo spelling). The default flavour will be configurable, but most folks will make that their web-browser presentation (HTML) templates. RSS will be an alternate flavour, meaning an alternative set of templates, and of course you'll be able to support different versions of RSS (not to mention different HTML, etc. layouts) with different flavours.
  • Take the CSS references out of the gemcast code, and move them into the HTML-based template(s). This is because files generated by gemcast are no longer always HTML.
  • Get permalinks (or at least single-entry views) working in gemcast so that RSS entries can contain links to the original entry, to be viewed in context.

Posted: Fri Feb 28 23:29:02 -0800 2003

/Programming/Projects/Gemcast Organizing by File Dates Sucks

So, this morning I decided to move some of my files around to better fit my category taxonomy. I had written some stuff about the iPod, and decided to move those items from my main category down to the Music category. Simple, I thought, I'll just do the following:

cp `grep iPod *.txt` Music

I did a 'cp', rather than a move, in order to double-check that the results of the command worked. After that, I 'rm'd the files from the original location. Cool, no problem, I thought. I thought wrong. The files I copied now all had today's date, therefore, they sorted incorrectly into my weblog. Dufus!

Fortunately this could be fixed with a bit of Ruby magic:

ruby -e 'File.utime(Time.local(2003, "Feb", 16, 0, 0, 0), \
         Time.local(2003, "Feb", 16, 0, 0, 0), \

Luckily, I'd written the entries fairly recently, so I could make a reasonable guess as to when they were composed.

Posted: Tue Feb 25 07:39:49 -0800 2003

/Programming/Projects/Gemcast Deploying and configuring gemcast

Deploying gemcast is pretty simple, really. It requires only that you have Ruby installed and a web server that can run CGI programs.

  • Create a directory for the gemcast CGI application and copy the *.rb files there.
  • Ensure that gemcast.rb has the proper execute privileges.
  • Configure your web server with a "script mapping" to the directory under which you've put the gemcast script files.
  • Create a data directory somewhere where gemcast will be able to see it. It's probably easiest to just copy the data directory included with the gemcast distro to wherever you want. The data directory should have, at the least, a templates and a css directory, each with their appropriate content (read further).
  • Configure gemcast with settings appropriate to your environment. This is done in the config.rb file. The file is fairly well-commented, so you shouldn't have much trouble there.
  • Configure your templates. There are currently three templates: body, entry, and box. Each relates to an element of your website's layout. Body represents the overall layout of your site's page(s). Entry and box represent the layout of individual weblog entries and sidebar boxes. Variables placed in the templates are replaced by gemcast when a given page is rendered.

    The following variables are available in your body template:

    • #{log_entry_content} (this is log content after the entry content is generated and the entry template is applied)
    • #{category_content}
    • #{sidebar_content} (this is sidebar content after the sidebar content is generated and the sidebar template is applied)
    • #{breadcrumb_content}
    • #{weblog_title}
    • #{current_day}
    • #{current_mon}
    • #{current_yr}

    The following variables are available in your log entry template:

    • #{entry_title}
    • #{entry_text}
    • #{entry_date}

    The following variables are available in your sidebar box template:

    • #{entry_title}
    • #{entry_text}
    • #{entry_date}

Posted: Thu Feb 20 16:44:25 -0800 2003

Thanks for visiting! Send comments to Mike Thomas.