Knowboard Updates (literally)

Knowboard now supports executing queries to update data via SPARQL. I’m working on some finishing touches and documentation before it’s released, but wanted to talk a bit about the design.

My goal with Knowboard has been to better manage my personal productivity workflows. This started with visual planning boards, then with the pivot to the current language server-based design this was mostly through auto-complete and some external scripting layers. Now, Knowboard supports applying updates through the editor for building personalized workflows.

Play

SPARQL Updates

The first critical step was allowing Knowboard to update the contents of the workspace. In SPARQL, an “update” query consists of DELETE and/or INSERT statements. There’s no UPDATE statement like in SQL since in RDF an “update” really consists of removing one tuple of information to replace it with another, rather than directly modifying the value of a column in a table.

PREFIX schema: <http://schema.org/>
DELETE { GRAPH ?graph { <> schema:creativeWorkStatus ?oldStatus } }
INSERT { GRAPH ?graph { <> schema:creativeWorkStatus "Published" } }
WHERE { GRAPH ?graph { <> schema:creativeWorkStatus ?oldStatus } }

Knowboard supports .rq or .sparql files, as well as sparql code blocks in Markdown. For update queris Knowboard will add a Code Lens to “Apply Update” which executes the query, updating the affected files.

Internally Knowboard keeps your workspace cached in an Oxigraph data store for fast queries, but to make updates it needs to save changes back to Markdown or the RDF text files. To support this, it uses the Oxigraph internals to apply a query inside a temporary transaction, tracking the data that was added or removed. Any affected files will be re-written with the updated content. A nice aspect of the LSP protocol is that these updates are sent back to the editor with workspace/applyEdit. Instead of immediately writing the files to disk, editors will typcially show these files as “modified” with unsaved changes which makes it easy to review and undo the changes if needed.

One caveat with this approach to re-writing the updated files is that I don’t currently have a good solution to preserve the original formatting of the file. The updates are re-formatted to the best “pretty” output available using the Sophia Rust RDF library. This generally looks “good”, and for Markdown properties, having the YAML normalized and sorted has seemed reasonable. However, when using Turtle for some larger data files, they often have more deliberate formatting and comments which get lost if the re-writing process. So far my typical uses for this are only to update the Markdown docs since that’s where most of my daily work happens. In other situations, using Turtle purely for “data” I may not care too much about the formatting. I mostly care about the formatting of Turtle for schema files where grouping and comments are more helpful in being able to read the document. There may be scenarios like renaming a property where I’d want to do a bulk update across the workspace that having a better format-preserving edit would help. It’s something I’d like to revisit, but would require building a new library to integrate more directly with the parsing layer to handle this.

“Portable” queries

It can be helpful to run queries which are “relative” in respect to the document containing them. To support this, Knowboard includes a BASE IRI for the current document, allowing for patterns like this to refer to the current document:

BIND(<> as ?this)

Here, <> is a blank “relative” IRI, and thus refers to the BASE IRI itself, allowing you to copy the query to different documents and have it run relative to wherever it is placed.

Resource Actions

While the portable query pattern is nice, copying them to each new document where you want to apply an update would still be cumbersome. Ideally I wanted a way to define updates more centrally which applied to specific types of entries. This was the key problem I wanted a solution for before adding updates.

I’ve been using the DASH Data Shapes specifications for describing how to preview documents, and in exploring more I read up on their “Suggestions” spec, which seemed similar to what I was looking for.

Suggestions solves a slightly different problem of providing an update that is mean to resolve a validation error. This is something I’ll probably add to Knowboard later since it would be quite useful for adding “quick fix” suggestions in the editor, but was not as general as I wanted for applying “workflow” updates when the document was already valid.

However, this led me to TopQuadrant’s dash:ModifyAction which was even closer to what I wanted. With this you can describe a modification to associate with a relevant rdfs:Class.

The Knowboard implementation follows dash:ModifyAction closely, though differs in a couple ways:

First, Knowboard’s kb:resourceAction is applied to a shape definition instead of directly to the class. Currently it still only looks at the class the action should apply to, but by using a shape I plan to allow further refinement based on properties as well. This would be useful for limiting some actions that are only relevant to specific states of an entry.

Second, dash:ModifyAction only defines updates via JavaScript. This would add complexity to recreate TopQuadrant’s JavaScript API. On the other hand, DASH Suggestions includes both JavaScript and SPARQL as options to define the updates. So, I chose to reuse sh:update from DASH Suggestions instead to define updates via SPARQL.

:ArticleShape a sh:NodeShape ;
  sh:targetClass schema:Article ;

  kb:resourceAction [
    a kb:ModifyAction ;
    rdfs:label "Completed" ;
    sh:update """
      PREFIX schema: <http://schema.org/>
      DELETE { GRAPH ?graph { <> schema:status ?oldStatus } }
      INSERT { GRAPH ?graph { <> schema:status "Read" } }
      WHERE { GRAPH ?graph { <> schema:status ?oldStatus } }
    """ ;
  ] .

Query parameters

Something I’ve punted on for now is the ability to specify parameters to use in the update.

In the dash:ModifyAction spec, they allow actions to contain parameters which the user will be prompted for. This seems useful, though the LSP protocol doesn’t have a direct way to ask for user input to provide the values. This could be possible using custom commands and additional editor-specific hooks, but it was not something I wanted to include yet.

Instead, I’ve been experimenting with patterns for adding temporary properties to a document that would be used to specify a value. I’ve been testing a “Do Later” action to postpone the date on a task. It defaults to 1 day, but by adding defer: P10D (using the dayTimeDuration syntax) to the document, it postpones it by the specified amount. While not perfect, it does offer a workaround.

Future

As mentioned earlier, I think one big potential improvement here is to add state-based targeting of actions. Instead of adding every possible action based on the type, actions could be relevant to a specific status, or maybe if there’s a specific relationship between entries.

Since this does have some risk by editing your files (instead of just reading them) I’m planning to guard it behind a configuration option to help ensure users understand the implications.

I’m continuing to test it in my own workspace, and want to prepare some documentation for it before releasing the update, but it’s been an excellent addition to be able to quickly create new tasks, or update their state right in the editor, and I’m regularly adding new actions as I come up with new uses for them.