Searching a public write.as or writefreely blog is easy to do, compared to searching private blogs or anonymous posts. It just takes adding an html iframe snippet to a blog page's source code, no javascript or CSS is needed. The blog must be public, if it's a private blog it won't work. For an example, take a look at this blog page: https://plaintextproject.online/articles.html. Then view the source code for that page (typically, right-click on the browser page and select View Source). You'll find the following snippet embedded in that code:

<iframe src="https://duckduckgo.com/search.html?width=250&site=plaintextproject.online&prefill=Search The Plain Text Project" style="overflow:hidden;margin:0;padding:0;width:308px;height:40px;border-radius: 15px 50px 30px 5px;" frameborder="0"></iframe>

The DuckDuckGo search engine is used. Replace two parts of that code:

1. Replace plaintextproject.online with either your personal domain name or your <blog>.writeas.com blog.

2. Replace Search The Plain Text Project with your custom search message.

A couple points:

1. You can search other public blogs or web sites if you want. It doesn't have to be your own blog.

2. Since this snippet is html with no javascript or CSS, this will work in an write.as/writefreely anonymous post also. However, you would have to search a public blog or site. Anonymous posts and private write.as blogs are not searchable with this technique (though see point #4 for a different way to search them).

3. If your blog's url is in the form write.as/blogname rather than blogname.writeas.com, that might work as well. I quickly did a test and it appeared to work.

4. If your write.as/writefreely blog is private or if you want to search your anonymous posts, you can still do it using, for example, a PHP program in combination with data downloaded from the write.as API. This technique is outlined here. The technique described in that post requires that you have a web server with PHP support. If you don't have that, CJ Eller, community manager for write.as, wrote a glitch.com app that will do the search on your blog(s) and anonymous data downloaded via the write.as and/or writefreely API.

5. Of course, this html iframe snippet will also work on other web site pages besides just those created on write.as or writefreely.

Thanks Plain Text Project for giving an example of how this can be done.

Recently I began to look into the API features of Write.as and WriteFreely, specifically to use the API to get the content of all my blog posts and anonymous posts in JSON format for the purpose of doing compound searches on this data.

My intention was to periodically download the data to my website using the API and to code one or a series of custom PHP programs to query it from a web page.

The code below is an example showing the API command along with a simple program to do a compound search of the data.

The PHP program illustrates navigating the JSON data and can be used to query one or multiple user instances. The data from a user's instances may include multiple blogs (whether private or published) and any number of anonymous posts.

The program output lists only those posts that have a match to your input keywords. The post title is a hyperlink that takes you directly to the post. There is also a link to the individual blog or anonymous posts page.

This code is freely available, as is, to anyone who may wish to use or modify it, but no support is provided for it. I am not an expert-level PHP programmer and this was a quick coding job, so I don't consider it to be particularly elegant coding. But hey, it works. I have this installed on my web server (Debian 10, Apache 2.4, and PHP 7.3). It could be installed on your local machine as well, and I've tested it in Windows 10 with XAMPP. It probably would also work on MacOS provided you have a web server with PHP.

The JSON files generated by the API need to be downloaded for each user Write.as or WriteFreely instance before running this program. Here's a curl command example for getting all user posts via the API:

curl "https://write.as/api/me/posts" \
   -H "Authorization: Token 00000000-1111-2222-3333-444444444444" \
   -H "Content-Type: application/json" \
   -X GET  >writeas_api_posts.json

If you don't yet have an authorization token, see this to get one: https://developers.write.as/docs/api/#authenticate-a-user

The beginning of the PHP program has a Required Input section where you need to specify your data files, the urls to your Write.as and/or WriteFreely instance(s)¹, and a flag involving markdown output. See the code for more info. You don't need to specify your individual blogs. Based on your auth code, the API will take care of downloading all your blogs and anonymous posts. The urls are used only to construct the hyperlink to the matched posts – they are not used to retrieve data, the data comes only from the API.

¹ As an example, I included two WriteFreely services, https://qua.name and https://wordsmith.social. Note that if you have custom domain names for your instances, you would of course use those.

Here's an image clip of an input form and output example:

Here is the PHP code. For display convenience this includes all the HTML, CSS, and PHP code in one file. A more complex example would use separate files.

Note: If you have a Write.as and/or WriteFreely blog and want want to try this out but don't have access to a server running PHP, it can be run on glitch.com. Instructions are provided here by CJ Eller, who has packaged this into a convenient, easy-to-use glitch app. General glitch.com Help topics are here. For info regarding public vs private visibility of the JSON data files in the glitch app, see this post.

<?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);

/*======================================================================================
  REQUIRED INPUT:
========================================================================================
  Specify the filenames for your posts API data, your write.as and/or writefreely blogs, 
    and for the $md_ext value enter either '' or '.md' depending on what's needed for the 
    anonymous post files to be rendered in markdown (write.as anon posts typically need 
    the '.md' whereas writefreely anon posts typically do not).
  Modify this associative array for your situation. Include a trailing forward slash, /,
    in the $href element. Add a folder path prefix to the $fn filename if needed.
  The API data files should be in the same folder as this program.  
========================================================================================*/
$blog[0] = array("fn" => "wa-posts.json", "href" => "https://write.as/", "md_ext" => '.md');
$blog[1] = array("fn" => "qua-posts.json", "href" => "https://my.writefreely.instance/", "md_ext" => '');
$blog[2] = array("fn" => "wordsmith-posts.json", "href" => "https://wordsmith.social/", "md_ext" => '');
/*======================================================================================*/
    
/*======================================================================================
  The API data JSON files need to be downloaded before running this program. Here's a 
  curl command example for getting all user posts via the write.as or writefreely API:
    
  curl "https://write.as/api/me/posts" \
    -H "Authorization: Token 00000000-1111-2222-3333-444444444444" \
    -H "Content-Type: application/json" \
    -X GET  >writeas_api_posts.json
    
  If you don't yet have an auth token, see this to get one:
  https://developers.write.as/docs/api/#authenticate-a-user
======================================================================================*/

// IMPORTANT NOTES:
//
// For anonymous posts, in order for the Title field to be populated in the API data,
// the first line of the post MUST start with a single # and a space. Otherwise, the 
// Title field will be blank.
//
// For the names of the elements provided by the API data, see bottom of this file
?>

<!DOCTYPE HTML>
<html>
<head>
  <meta http-equiv="content-type" content="text/html; charset=UTF-8">
  <title>WaWf API Data Search</title>
  <style type="text/css">
    body {
      font-family: sans-serif, helvetica;
      font-size: 15px;
      line-height: 1.3;
      margin: 15px;
    }
    a {
       color: blue;
       text-decoration: none;
     }
     a:hover {
       color: #AA08AA; 
       font-weight: bold;
       text-decoration: underline;
     }
  </style>
</head>
<body> 
<?php
//==========================================
// Main part of program
//==========================================
$msg = 'Search these files: ';
$i = 0;
foreach($blog as $item) {
  $fn = $item['fn'];
  if (!file_exists($fn)) {
    echo "<span style=\"color:red;\"><b>Error: File $fn not found; please check and either correct filename or remove it from the list (i.e., the relevant " . '$blog[] entry in this program)</b></span>';
    exit;
  } else {
  $msg .= $fn . ", ";
  }
  $href = $item['href'];
  if (substr($href, -1) !== '/') $blog[$i]['href'] = $href . '/';
}  
echo substr($msg, 0, -2) . "<br><br>";
$search1=''; $search2=''; $search3=''; $butnot=''; $bOption=''; $nq='';

if (isset($_POST['submit'])) {
  $search1 = $_POST['searchkey1'];
  $search2 = $_POST['searchkey2'];
  $search3 = $_POST['searchkey3'];
  $butnot =  $_POST['butnot'];
  $bOption = $_POST['bOption'];
  $uri = $_SERVER['REQUEST_URI'];
  $nq = '<a href="' . $uri . '">New Query</a><br>';
}

print '<form name="myform" action="' . $_SERVER['PHP_SELF'] . '" method="post">
	Keyword1: (&nbsp;<input name="searchkey1" size="15" autofocus value="' . $search1 . '">&nbsp;<b>AND</b>
	Keyword2: <input name="searchkey2" size="15" value="' . $search2 . '">&nbsp;)&nbsp;
 	<select name="bOption">';
if ($bOption == 'OR') {
  print	'<option value="AND">AND</option><option value="OR" selected>OR</option>';
} else {
  print	'<option value="AND" selected>AND</option><option value="OR">OR</option>';
}  
print	'</select> 
	Keyword3: <input name="searchkey3" size="15" value="' . $search3 . '">&nbsp;
	But Not: <input name="butnot" size="15" value="' . $butnot . '">&nbsp;
	<input type="submit" name="submit" value="Go">&nbsp;&nbsp;' . $nq . '</form>';
if (!isset($_POST['submit'])) exit;

// The "submit" variable exists, so the form has been submitted - process form input and display results

// The API data file can have multiple blogs along with anonymous posts and they may be all mixed together with no 
// separation and no sort order. Therefore, if you want to have all anon posts and each blog's posts together in grouped
// sections, a sorting mechanism is needed. We sort on the url (which groups each blog's posts together') and to handle 
// the anon posts, we add a class="anon" to the url so they'll be grouped together when sorted.

if ($search1 == "" && $search2 == "") {
  echo '<br><b><span style="color:red;">You must input at least one search keyword in the first 2 input boxes</span></b>';
  exit;
}

$iMatches = 0;
foreach($blog as $item) {
  $fn = $item['fn'];
  $href = $item['href'];
  $md_ext = $item['md_ext'];
  $json = file_get_contents($fn);
  $array = json_decode($json, true);
  $blogposts = $array['data'];
  $lines='';
  
  foreach ($blogposts as $post) {  
    $id    = $post['id'];
    $slug  = $post['slug'];
    $title = $post['title'];
    // title might be blank; if so, substitute the slug if it's not also blank, otherwise use the post id
    if (trim($title) == '') {
      if (trim($slug) !== '') {
        $title = $slug;
      } else {
        $title = $id;
      }
    }
    $tags  = $post['tags'];  // note $tags is an array
    $body  = $post['body'];
    $url = '<a href="' . $href . $id . '.md" target="_blank">' . "<b>$title | $slug</b></a><br>\n"; 
    //Check title, tags, and body of blog post for keyword matches - entire post vs individual lines?
    $contents = $title . '||' . implode(" ", $tags) . '||' . $body; //use || separator so don't have false match possibility due to a run-on 
    if (isMatched($contents)) {
      // Matched
      if (isset($post['collection'])) {
        // Blog post
        $coll = $post['collection'];
        $cAlias = $coll['alias'];
        $blogUrl = '<a href="' . $href .  $coll['alias'] . '" target="_blank">' . $cAlias . "</a>"; 
        $url = '<a href="' . $href . $cAlias . '/' . $slug . '" target="_blank">' . "<b>$title</b></a>"; 
        $lines .=  substr($cAlias,0,3) . "$blogUrl | $url<br>\n";
      } else {
        // Anonymous posts
        // They typically have a null slug (possible exception is if they've been moved from blog post to anonymous post)
        $url = '<a href="' . $href . $id . $md_ext . '" target="_blank">' . "<b>$title</b></a><br>\n";
        $lines .= 'ano' . '<a class="anon" href="' . $href . 'me/posts">Anonymous</a>' . " | $url<br>\n";   //class="anon" needed for sort
      }
      $iMatches += 1;
    }  
  } // end foreach blog post
  
  if ($iMatches == 0) {
    echo "<br>No matches were found for the search criteria";
    exit;
  }
  
  // $lines contains each blog post. Sort them so all anon posts and separate blog posts group together.
  $posts = explode("\n", $lines);
  sort($posts, SORT_NATURAL | SORT_FLAG_CASE);
  $prev = '';
  $i = 1;
  $itot = 0;
  foreach($posts as $p) {
    if(strlen($p) > 4) {
      if (substr($p,0,3) !== $prev) {
        echo "<br>\n";
        $i = 1;
      }  
      echo str_pad($i, 2, '0', STR_PAD_LEFT) . ": " . substr($p,3) . "\n";
      $i++;
      $itot += 1;
      $prev = substr($p, 0,3);
    }
  }
} // end foreach blog

echo "<br>$iMatches matches were found for the search criteria";

function isMatched($contents) {
  global $search1;
  global $search2;
  global $search3;
  global $butnot;
  global $bOption;  //applies solely to the 3rd searchkey input

  if (stripos($contents, $butnot) !== false) return false;  
  if ($bOption == 'OR' && $search3 > '' && stripos($contents, $search3) !== false) return true;
 
  // The ButNot and OR criteria were handled above, so now we only need to check search 
  // term 1 by itself if it's the only input or in cobmination with any term 2 and 3 input
   if ($search1 == "" && $search2 !== "") {
  	$search1 = $search2;
  	$search2 = "";
  }	
   if ($search2 == "" && $search3 !== "" && $bOption == "AND") {
  	$search2 = $search3;
  	$search3 = "";
  }	
  // By the program logic, search1 is guaranteed to be be populated
  // So if there is no search1 match and since only AND logic now applies here, return false
  if (stripos($contents,$search1) === false) return false;
  // We now know that there is a $search1 match
  if ($search2 == '' && $search3 == '') return true;
  if ($search2 > '' && stripos($contents, $search2) !== false  && $search3 == '') return true;
  if ($search2 > '' && stripos($contents, $search2) !== false  && $search3 > '' && stripos($contents, $search3) !== false) return true;
  return false;
}

/*
Example elements in the JSON API data
  "id": "xxxxxxxxxx",
  "slug": "confidence-level-contours",
  "appearance": "sans",
  "language": "en",
  "rtl": false,
  "created": "2019-11-03T15:46:58Z",
  "updated": "2019-11-03T16:09:50Z",
  "title": "Confidence level contours",
  "body": "Some body text",
  "tags": [],
  "images": [
     "https://i.snap.as/xxxxxxx.jpg",
     "https://i.snap.as/yyyyyyy.jpg",
   ],
  "views": 0,
  "collection": {
     "alias": "MyBlogName",
     "title": "My Blog Title",
     "description": "Primarily About this, that, and the other",
     "style_sheet": "",
     "public": false,
     "views": 22,
     "total_posts": 0
     }
*/

?>
</body>
</html>

Some preparatory very helpful tips that I found are that (1) GCE instance boot disks can be attached and reattached, (2) a new persistent disk can be made from a saved snapshot of an instance's boot disk and then attached to the instance (basically swapping out the original boot disk with the new one created from the snapshot, and (3) a debian.org article on how to upgrade from version 9 to 10.

With this info, I felt I had two upgrade choices:

(1) Create a new GCE instance installing Debian 10, update it to match the configurations, apps, and data of the original v9 install, then detach that persistent disk and attach it as a new boot disk on the original instance. After verification that all was properly working, the second VM instance could be deleted along with the original boot disk with the old v9 install.

(2) Make a snapshot of the current v9 disk, use it to create a new persistent disk, attach this new disk as the boot disk of the original instance, then try an upgrade and see if that would work ok. This would avoid having to create a second VM instance.

I preferred the second option (the first one seemed like more work and I wanted to avoid starting over with reconfiguring a new install), though I found no write-ups online from anyone's experience trying that. But even if the upgrade in-place didn't work, I would still have the original v9 boot disk and could swap that back in if needed. So I went with the in-place upgrade.

In-Place Upgrade Steps So here's the basic steps I used and they worked without incident! (however, do note that this was a simple single-user instance with few apps).

I basically followed the steps in the debian.org doc on how to upgrade from version 9 to 10. Below I'll just mention any differences from those procedures.

In addition to the backups suggested in Section 4.1.1, I backed up the entire webroot (/var/www/html/) folders and some other data and config files (e.g., in my home folder) to a Cloud Storage Bucket. I also would have the detached original boot disk available if needed.

Section 4.1.6. Verify network interface name support was noted but I did nothing in response to it. I did have an eth0 network interface both before and after the upgrade, and there were no problems with it.

The non-Debian sources I had were the sury.org packages, Docker, and the Google Cloud guest packages for Debian. As noted in the Debian upgrade doc, I updated all references from stretch to buster in /etc/apt/sources.source.list and to the files in /etc/apt/sources.list.d. Those were the only changes I made to these files. I noted that the /etc/apt/truteed.gpg.d folder included gpg keys for Buster, sury.org, and Google Cloud.

I didn't record the session as recommended in Section 4.4.1, though in retrospect that would have been a prudent thing to do.

As recommended in Section 4.4, I first did the 'minimal upgrade'. This went without incident and 384 packages were updated. I then proceeded immediately (did not reboot) to the full-upgrade. 146 packages were upgraded, 69 newly installed, 1 was removed, and 0 not upgraded. There was a message that a number of packages were no longer required and could be removed with sudo apt autoremove, which I subsequently did after verifying that the upgrade was successful.

There were only two prompts during the full-upgrade. One was whether to install a new /etc/ntp.conf or keep the old. After looking at the diffs and verifying that they were due to new statements and that I had not customized anything in the old, I selected the new version. The second prompt was whether to install the new /etc/sshd_config. I kept the old as I had made customizations and there were no new statements in the v10 version that weren't in v9.

I then rebooted the VM, and everything worked, no glitches were encountered. The new kernel is vmlinuz-4.19.0-6-amd64.

As recommended in Section 4.7.1 Purging Removed Packages, I issued those commands. There were only 4 packages to be purged.

There are 410 packages files in /var/cache/apt/archives/, most of them probably due to the upgrade. I will probably delete them at some point with apt clean.

Updated Info Edits After working with the new install for a while, there might be additional useful comments. If so, this post will be updated with a timestamp for the update added below.

Original post version: 03-Dec-2019

Hashtags: #Debian #GoogleComputeEngine #Sysadmin #VPS