Blogging using org-mode and Ruby: Managing dependencies




On the previous post we layed down the work for writing posts in org-mode and then publishing them using Ruby.

Now we need to set up a system to organize and publish them when running rake.

Initial setup

The first thing to do is to do some setup up front.

We define a constant for our posts and exports directories. Those two directories will be needed several times and it's a good idea to be able to change them if we decide to.

Right after that we make sure they'll exist when we run rake by creating them if they don't.

  POSTS_DIR="./posts"
  EXPORTS_DIR="./html"

  FileUtils.mkdir_p POSTS_DIR
  FileUtils.mkdir_p EXPORTS_DIR

Then we want to gather the list of all the org files in the posts directory, but we exclude the ones that begin with WIP (remember that this was the strategy we landed for drafting episodes and ideas).

To accomplish this, we use a FileList that searches for all files with the .org extension and then we exclude the drafts.

From this file list, we can now extract all the post names by removing their extension and the folder part of the path. So, for example, if we have a file called "./posts/my-cool-post.org", the corresponding post name would be "my-cool-post".

We also extract the destination files, which need to be stored in the exports directory and have the .html extension.

  org_files = FileList.new("#{POSTS_DIR}/*.org").exclude(/\/WIP[_-]/i)

  post_names = org_files.ext("").sub("#{POSTS_DIR}/", "")
  html_files = org_files.ext(".html").sub("#{POSTS_DIR}/", "#{EXPORTS_DIR}/")

It's important to understand that at this point, those file and post names are just the names of the tasks and files we'll need to manage, but we haven't done any work yet.

Dependency management

Now that we have all the required data, we set up the dependency management system.

We declare the default task (the one that will run when we invoke rake without any parameters) to be the list of post names. This means that for each post name, we'll need to have a task defined.

To do so, we iterate on the list of post names and, for each one, we define a task. This task will be named after the post name and depend on the html files to be exported.

Finally, we need a rule for exporting the org files into HTML. Here, we need to take into account that the input and output files need to go on different directories (that's the meaning of the proc)

  task :default => post_names

  post_names.each do |post_name|
    post post_name => html_files do |t|
      # ...
    end
  end

  rule %r[/html/.+\.html$] => proc { |t| t.pathmap("posts/%n.org") } do |t|
    # ...
  end


Edit: the code above is wrong =(, read till the end for a follow up on this

Now let's take a look at each individual part of this management system.

Creating HTML content from org files

To convert the org files into it's HTML counterpart, we create a new OrgPost object passing in the source file name (.org) as an argument to it's constructor and save the html render on a variable. Then we write the generated content to the target file (.html).

  rule %r[/html/.+\.html$] => proc {|t| t.pathmap("posts/%n.org")} do |t|
    puts "Converting #{t.name}"

    html = OrgPost.new(t.source).html

    File.write(t.name, html)
  end

And that's it.

If we want to customize this export in some way, the OrgPost class is the perfect container to do so. We have isolated the content export into a single place and it should be easy to do so.

Publishing and updating posts

Publishing and updating is also a simple process.

We want to create a task for each post name on our list. For this, we iterate over all our post_names and invoke the post method we created on the previous post.

For each one of them we:

First we get create an OrgPost file using the post name, directory and extension.

After this, we gather some default options:

  • We set the post status to publish because all the posts on the list are meant to be published.
  • The post content gets read out of the source file
  • The post name is the name of the current task
  • We get the post title from the org_post we created in the previous step. If we haven't defined one on the org file, we default to the task's post_title method, which we defined on the PostTask file. Remember that this method will extract the name from the file name.
  • Finally, we obtain the list of categories also from the org_post

After gathering the options, we decide whether to create or update our post on the server by asking if the post exists there.

If it exists, we invoke the editPost method on the WordPress client and pass it the post id as an argument and the options we defined as another under the content keyword.

If it doesn't exist, we invoke the newPost method instead passing just the options (there's no id assigned to the post yet).

  post_names.each do |post_name|
    post post_name => html_files do |t|
      org_post = OrgPost.new("#{POSTS_DIR}/#{t.name}.org")

      options = {
        post_status: "publish",
        post_content: File.read(t.source),
        post_name: t.name,
        post_title: org_post.title || t.post_title,
        terms_names: {
          category:  org_post.categories
        }
      }

      if t.post_exist?
        puts "Updating post #{t.name} ...".green
        WpClient.editPost(post_id: t.post_id, content: options)
      else
        puts "Creating new post #{t.name} ...".green
        WpClient.newPost(content: options)
      end
    end
  end

Default task

Now we want to set a default task. Ideally, we'd like to run by default all the post tasks we defined, we want all our posts published after all.

Is that possible to do? It's not only possible, but extremely easy. The only thing we need to do is to define a :default task and set it's dependencies to be all the posts names we've gathered. And that's it

  task :default => post_names

This shows how powerful Rake is and how simple it is to use.

Conclusion

    I've started this small project with three objectives:
  • Start writing in my blog again.
  • Writing in org-mode
  • Learning Rake more in depth

So far I've accomplished all three of them. Having a project to play on is an excellent way to learn.

Also writing during the process was highly beneficial. I've found and corrected bugs and modified heavily the design due to things I discovered along the way (and I have some more work to do because I didn't fix everything in order to not procrastinate writing).

You should definitely try it out. I hope you found this posts useful.

Saluti

Edit: Aaaand I messed up

After publishing this article, I realized that the contents where incorrect, which means that there was something wrong.

As a result, I fixed the bug and realized I could substantially simplify the dependency management portion of the project.

I'll write a separate article explaining exactly what went wrong and how I solved it.

See you then.