Orion API - Build
Joe Schaefer
Build System
This document covers the Build System APIs.
Basically, the build system is governed by two user-supplied Perl modules: lib/path.pm and lib/view.pm.
The former’s job is to do three things:
- load
lib/facts.ymlandlib/acl.yml, - construct
@path::patterns, and - opportunistically walk the
content/tree to seed%path::dependenciesand@path::aclsfrom markdown/yaml file header metadata.
The latter’s job is to provide invokable view-based $methods for the matching entries in @path::patterns (as a stringified method name in the second slot of each arrayref entry), invoked by the build scripting:
for my $p (@$patterns) {
my ($re, $method, $args) = @$p;
next unless $path =~ $re;
if ($args->{headers}) {
my Data::Dumper $d;
$d = $d->new([$args->{headers}], ['$args->{headers}']);
$d->Deepcopy(1);
$d->Purity(1);
eval $d->Dump;
}
my $s = $method_cache{$method} //= view->can($method) or die "Can't locate method: $method\n";
my $start_call = [gettimeofday];
no warnings 'once';
$view::path = $path;
my ($content, $ext, undef, @new_sources) = $s->(nonce => rand, website => $ENV{WEBSITE}, repos => $ENV{REPOS}, path => $path, lang => $lang, %$args);
my $elapsed = tv_interval($start_call);
if ($$args{compress}) {
$lang .= ".gz";
if (defined $content) {
utf8::encode($content) if utf8::is_utf8 $content;
gzip \($content, my $compressed);
$content = $compressed;
}
}
if (defined $content) {
my $dest = "$target_base/$target_file.$ext$lang";
my $encoding = $$args{encoding} // ($$args{compress} ? "raw" : "utf8");
my $mtime;
#$mtime = $_->mtime for map stat $_, "content/$path";
open my $fh, ">:$encoding", $dest
or die "Can't open $dest: $!\n";
print $fh $content;
close $fh;
#utime $mtime, $mtime, $dest if $mtime;
}
syswrite_all "Built to $target_base/$target_file.$ext$lang in ${elapsed}s.\n";
return @new_sources;
}
COPY:
my ($dest, $copied) = copy_if_newer $file, "$target_base/$file";
syswrite_all "Copied to $dest.\n" if $copied;
Many views are meant to be stacked as “filters” to preprocess (or postprocess) aspects of the file in $path that are novel, like external code snippets or asymptote-fenced markdown blocks. You can see an example of that here.
SunStarSys::View — base class for lib/view.pm
The most popular (and sophisticated) view
This view incorporates automated processing of files located within the $path‘s attachment directory. In other words, if $path = "/foo.md.en", then the files stored within the /foo.page/ directory associated with the “.en” language extension will be incorporated into the template addressable arguments for that $path regardless of the preprocess argument’s setting — which if true, would also make that material accessible by the page’s content itself.
Mandatory Arguments:
templatepathlang
Optional Argmuents:
deps— overrides normalfetch_depsprocessingquick_deps— internal deps-processing optimization setting; best left unsetpreprocess— enables template processing within the$pathcontent itself,archive_root— files in “archived” status are “copied” and tracked by year/month subfolders to this content-rooted location viassi,category_root— items in the “categories” header are “copied” over into appropriately named categories folders in this content-rooted location viassi.
For building index.html and sitemap.html pages
Locale-specific, sorted index of dependencies.
Mandatory Arguments:
pathlang
Optional Arguments:
quick_depsnestedpreprocess
Builds and caches Asymptote triple-backquoted-code blocks for HTML5-WebGL-canvas-enabled vector graphics
Mandatory Arguments:
viewlangpath
Don’t build these at all.
Instead, build the associated generated source files (e.g. .bib\$lang \$base.page/bibliography.yml\$lang) to be built on a secondary build system run.
Convert YAML files, typically into JSON.
Optional Arguments:
extdefaults tojsonfilterdefaults tojson_rawtemplateoverridesfilterdefault expression
Convert CSV files, typically into JSON.
Optional Arguments:
extdefaults tojsonfilterdefaults tojson_rawtemplateoverridesfilterdefault expression
Refactors $data argument hashref as a timestamp ordered arrayref of 2-element arrayrefs.
The first entry in each 2-element arrayref is the file path-name, second element is the resultant read_text_file hashref for that path-name.
Returns a list of resulting new source files if $quick > 2.
Mandatory Arguments:
pathdata- input as hashref; stores resulting arrayref of deps upon returnquick- defaults to 2
Processes external code snippet lines, normally importing their source locations on GitHub into programming-language-specific fenced markdown blocks. Example here.
Adds a title string to markdown links. Normally should be used under the offline view to avoid pulling in titles from remote sites, which can put a significant drag on page build times.
SunStarSys::Util — utility library for lib/path.pm and lib/view.pm
Orion’s universal text file processor
Parses headers+content of UTF-8 encoded file $file and stores results in hashref $out. $content_lines is the (optional) maximum number of content lines to read.
Returns actual number of lines read (including headers).
$file may be a reference to a raw string, representing the full contents of a file. The results in $out will still be UTF-8 encoded.
Copies $src to $dest if the former’s modification timestamp is newer than the latter’s.
On copy, additionally gzip-compresses the $dest file if it’s a text file, and adds “.gz” extension to the name. GZIP compression can be disabled by setting $ENV{NO_COPY_COMPRESS}=1.
Sorts Markdown Tables in $content according to each table’s column spec.
Exactly one column may be sorted per table, optionally numerically n, in either descending v or ascending ^ order.
fixup_code($prefix, $type, @_)
Strips $prefix from every arg in @_.
The purpose of the $type argument is implementation specific, but is mainly used to seed the editor.md "mode" for processing this content in @_.
normalize_svn_path(@_)
Normalizes all paths in @_ for safe use as raw arguments to SVN::Client commands.
sanitize_relative_path(@_)
Secures paths in @_ for use as pure relative paths in Dotiac::DTL (Django Template) path-specific commands.
Wrapper around File::Basename::fileparse. Without arguments, uses $_ as the filename to parse.
Conditionally walks the ./content tree of the build system checkout; first normalizing $_ as the formal content-rooted subpath, and then invoking $code->() on each item in the tree-walk.
For most builds, the walk never happens — instead the build relies on the cached data from prior builds.
The only way to force a walk is by setting $path::use_cache to a false value in your user-supplied modules. Otherwise this behavior is expertly-managed by the incremental build technology.
Returns 1 if the walk actually proceeded, instead of relying on cached data. Otherwise returns a false value.
Flags each Status: archived $path (in a natural-language-specific way). Uses $_ if no args are passed.
Archiving Markdown files is a natural way to tell Orion to “stop paying attention to this file’s permalinked location”, unless it is updated again, in which case the archival location will be refreshed. In particular, archived files do not appear in directory listings within the Orion CMS itself; you have to navigate to the live page itself to be able to edit it again online.
Securely updates %path::dependencies for this $path, based on its dependencies header glob(s).
Defaults to using $_ as the path if no args are passed.
Safely updates @path::acl for this $path, based on its acl header spec.
Defaults to using $_ as the path if no args are passed.