Engineer the web, together.

Symbiote news

Keep in touch with what we're up to.

RSS available too if that's your thing!

Cleaning up after yourself

Marcus Nyeholt

Posted 6 Mar 2018


Ever taken a backup of a site database and gone to copy it somewhere, and been surprised how long it's taking to do so? Tried copying a site database locally to debug an issue and wonder why it fails restoring? 

We have a few modules we use regularly - Queued Jobs and Data Change Tracker are probably the most guilty - that can silently fill a database table with millions of records depending on your usage. Consider the following job

class RegularJob extends AbstractQueuedJob {

    public function process() {

        $nextJob = new RegularJob();

        // run every five minutes except between midnight and 7am
        $nextTime = date('H') < 7 ? date('Y-m-d 07:00:00') : date('Y-m-d H:i:s', time() + 300);

        $this->queuedJobService->runJob($nextJob, nextTime);

Pretty straight-forward; every 2 minutes, run some expensive processing code unless it's between midnight and 7am. The side effect of this being that every day, there's over 500 records being added to the QueuedJobDescriptor table, on top of any other jobs running. Not a huge deal, but over a year this is over 180000 records added. With other processing jobs happening, in a very interactive site this can lead to a multi-million row table. 

DataChange Tracker is another that has this slow creep of content buildup, exacerbated by the fact that in its most aggressive mode, it will capture data object contents (before + after) and request variables. 

Solving it with ... more jobs.

Luckily, both these modules come with cleanup jobs to ease the process. 

CleanupJob - Removes all jobs that have completed and are older than 30 days (you can configure this value from .yml). This can be created from the Jobs admin section of the CMS. 

PruneChangesBeforeJob - Create via the Jobs admin section with a single constructor argument being a strtotime compatible string representing how long ago to prune data changes before. For example, "-1 month" will remove all tracked changes from before 1 month. We find that -3 months works pretty well here. 

Using previous search data

Marcus Nyeholt

Posted 29 Jan 2018


We're back! Most of us have taken a good break over the Christmas / New Year period, and just getting back into 5 day work weeks. To get back into things, we're looking at how search suggestions can be used from the Extensible Search module.

The module provides an out-of-the-box search page implementation that allows CMS users to configure how content is searched in a SilverStripe CMS website, rather than needing everything to be defined in code. One of its nifty features is the ability for it to record the searches executed in the site, which can be subsequently used to prompt users for known-good search.

While ExtensibleSearchPage will handle this for you, if you have your own search implementation you can still make use of the search suggestion capabilities by directly working with the API. Recording a search is as simple as

            $term, $totalResults, $queryTime, 'SearchEngineType', $page->ID

Under the covers, this leads to the recording of two bits of information;

  • An ExtensibleSearch object, which records the specific term and result count
  • If there were results for that term, a "suggested" search term object,

search analytics

The ExtensibleSearchSuggestion object records the total number of times a term has been searched on, and contains a flag indicating whether the term is an "approved" suggestion, allowing CMS authors the ability to indicate whether a term should be used on the site.

Making use of the suggestions is straightforward; for example, say you're wanting to display a list of the most frequently searched terms on your site as a prompt for users

$suggestions = ExtensibleSearchSuggestion::get()->filter('Approved', true)->sort('Frequency DESC')->limit(5);

and in your template

    <% loop $Suggestions %>
    <li><a href="search?term=$Term.ATT">$Term</a></li>
    <% end_loop %>

Alternatively, from the frontend of the site in an autocomplete scenario, you can call //{term}&amp;page={searchPageID} to retrieve suggestions that partially match the input term.

Note: If you have enabled search logging, you might want to look into using the ExtensibleSearchArchiveTask to ensure that you don't end up with several hundred thousand records of search history. This task ensures the history is periodically pruned of the raw records and keeps an archive of the terms/frequencies.

YAML-eage May Vary

Nathan Glasl

Posted 18 Dec 2017


So you're trying to make your YAML configuration be applied after another module, the following is what you want to use, right?

Name: after-multisites
Before: multisites
    - CMSMainFilterExtension

You'd think this would make it come before, but perhaps that's for another blog post.

You flush the cache, and the following appears..

Fatal error: Based on their before & after rules two fragments both need to be before/after each other in /framework/core/manifest/ConfigManifest.php on line 373

What? Having a further look into what's going on, there doesn't seem to be anything obvious, so what's actually happening?

Well, turns out that when figuring out the before/after logic, every file and fragment within those files are considered, meaning the rules defined in other config files may be conflicting as the Before statement is affecting all files.

So, you just needed to be more specific;

Name: after-multisites
Before: multisites/extensions#extensions
    - CMSMainFilterExtension

Don't you love a good error message.