Add SEO documentation - how to

docs/content/seo.md

@@ -0,0 +1,200 @@
+title = "Search Engine Optimization (SEO)"
+date = "2022-08-09T14:05:02.118466Z"
+
+[extra]
+url = "https://github.com/fermyon/bartholomew/blob/main/docs/content/seo.md"
+---
+
+## Google verification using Bartholomew
+
+Your Bartholomew implementation can be <a href="https://support.google.com/webmasters/answer/9008080?hl=en" target="_blank">officially verify with Google</a>. This is an important first step because the verification process provides you with access to the <a href="https://g.co/kgs/UcBemE" target="_blank">Google Search Console</a>. The Google Search Console (formlerly Google Webmaster Tools) allows webmasters to check indexing status, search queries, crawling errors and also optimize visibility of their site.
+
+Let's take a look at how the verification process is accomplished using Bartholomew. 
+
+### Markdown
+
+The first step in the Google verification process is where Google provides you (the owner) of the specific website with a specially named file i.e. `abcdefg.html`. Google now wants you, the owner, to make this file available on your site, so that Google can fetch it as proof that you are the site's owner which has access control to the site. This is a really simple task. First you create a Markdown file (in Bartholomew's `content` directory) called `abcdefg.md` (the name of this file just has to match the name of the file which Google provided). **Note: We are creating an `.md` file here not an `.html` file**.
+
+Bartholomew uses templating so you just have to be explicit about a couple of things inside that new `.md` file. Specifically,  Makes sure that there is a template name (we will create the template next) and that the content type (of this file) is rendered as `text/html`. This is shown in the source code of the new `abcdefg.md` file below.
+
+```
+title = "Google Verification"
+description = "Google verification file which provides us with access to Google Search Console"
+date = "2022-07-11T00:01:01Z"
+template = "google_verification"
+content_type = "text/html"
+---
+
+This is the abcdefg.html file that Google can see openly at the root of the website.
+```
+
+### Template
+
+Bartholomew uses <a href="https://handlebarsjs.com/" target="_blank">Handlebars</a> templating. Therefore the next step is for you to go ahead and create a new `google_verification.hbs` file in Bartholomew's template directory. Once the file is created, populate it with the content which Google requested, below is just an example.
+
+```
+{{! 
+For info on what can be placed here, see https://support.google.com/webmasters/answer/9008080#html_verification&zippy=%2Chtml-file-upload
+}}
+google-site-verification: abcdefg.html
+```
+
+At this point the verification button in the Google dashboard can be pressed; Google fetches the file from your site and the verification is complete!
+
+## Search Engine Optimization(SEO) using Bartholomew
+
+In addition to just verifying the ownership of a site/domain, You can see that there are <a href="https://developers.google.com/search/docs/beginner/seo-starter-guide" target="_blank">specific SEO requirements</a> in relation to how the <a href="https://g.co/kgs/p6qtQs" target="_blank">Googlebot</a> indexes content. Googlebot is the web crawler software used by Google.
+
+Let's take a look at how the SEO compliance (i.e. sitemap and robots.txt) is accomplished using Bartholomew.
+
+### Generating a sitemap
+
+Google <a href="https://developers.google.com/search/docs/advanced/sitemaps/build-sitemap" target="_blank">expects the standard sitemap protocol to be implemented</a>. Thankfully, Bartholomew automatically builds a sitemap file based on the entire set of content in the CMS. The heavy lifting of the work is performed using the <a href="https://rhai.rs/" target="_blank">Rhai</a> scripting language. Here is an example of the `sitemap.rhai` file that you would store in Bartholomew's scripts directory.
+
+```
+// This function lists all of the posts, filtering a few.
+//
+// It returns an array of objects of the form:
+//  [
+//    #{ uri: "path/to/page", page: PageObject }
+// ]
+
+// These should be skipped.
+let disallow = [
+    "/sitemap", // Don't list self.
+    // "/tag", // tag will list all of the tags on a site. If you prefer this not be indexed, uncomment this line.
+    "/index", // This is a duplicate of /
+    "/atom",
+    "/robots",
+];
+
+// Param 1 should be `site.pages`
+let pages = params[0];
+
+let site_pages = [];
+let keys = pages.keys();
+for item in keys {
+    let path = item.sub_string(8);
+    let page = pages[item];
+
+    path = path.sub_string(0, path.index_of(".md"));
+    if !disallow.contains(path) {
+        site_pages.push(#{
+            uri: path,
+            page: page,
+            priority: prioritize(path),
+            frequency: "weekly",
+        });
+    }
+
+}
+
+// This is an example of how we could prioritize based on information about the page.
+//
+// Specifically, here we use path to boost docs and blogs while reducing the priority
+// of author pages and features.
+fn prioritize(path) {
+    let boost = ["/blog/", "/docs/"];
+    for sub in boost {
+        if path.contains(sub) {
+            return 0.8
+        }
+    }
+    let nerf = ["/author/", "/features/"];
+    for sub in nerf {
+        if path.contains(sub) {
+            return 0.3
+        }
+    }
+    0.5
+}
+
+// Return the blogs sorted newest to oldest
+fn sort_by_date(a, b) {
+    if a.page.head.date < b.page.head.date {
+        1
+    } else {
+        -1
+    }
+}
+
+// Sort by the value of the page date.
+site_pages.sort(Fn("sort_by_date"));
+
+site_pages
+```
+
+In conjunction to the above scripting, the aforementioned <a href="https://handlebarsjs.com/" target="_blank">Handlebars</a> templating assists in this work being performed dynamically (using variables common between the script and the template); as shown in the `sitemap.hbs` file's contents below.
+
+```
+<?xml version="1.0" encoding="UTF-8" ?>
+{{!
+    For sitemap.xml, see https://www.sitemaps.org/protocol.html
+    For date/time format, see https://www.w3.org/TR/NOTE-datetime
+}}
+<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
+    <url>
+        <loc>{{site.info.base_url}}/</loc>
+        <changefreq>daily</changefreq>
+        <priority>0.8</priority>
+    </url>
+    {{#each (sitemap site.pages) }}
+    <url>
+        <loc>{{../site.info.base_url}}{{uri}}</loc>
+        {{#if page.head.date }}<lastmod>{{date_format "%Y-%m-%dT%H:%M:%SZ" page.head.date}}</lastmod>{{/if}}
+        <changefreq>{{frequency}}</changefreq>
+        <priority>{{priority}}</priority>
+    </url>
+    {{/each}}
+</urlset>
+```
+
+**If you ever need assistance with any of the scripting, templating or Markdown mentioned here, please go ahead and jump into our <a href="https://discord.gg/AAFNfS7NGf" target="_blank">Discord</a> server. We are here to assist and would love to see what you are building. Alternatively, place [an issue in GitHub](https://github.com/fermyon/bartholomew/issues) and ask for help.**
+
+From a display point of view we again just use Markdown (create a `sitemap.md` file in the site's content directory, correctly references the name of the template (`sitemap`) and then ensures that the content type is set to `text/xml`). The above process will generate an XML sitemap called `sitemap.xml` at the root of the site. Perfect!
+
+```
+title = "Sitemap XML file"
+description = "This is the sitemap.xml file"
+date = "2021-12-29T22:36:33Z"
+template = "sitemap"
+content_type = "text/xml"
+---
+
+This is the autogenerated sitemap. Note that the suffix .xml is replaced with .md by Bartholomew.
+```
+
+### Creating a Robots file
+
+You can actuall control the Googlebot and tell it which files it may access on the site. This is done via the use of <a href="https://developers.google.com/search/docs/advanced/robots/create-robots-txt" target="_blank">a robots.txt file</a>.
+
+Similarly to the process above, you create a `robots.md` Markdown file in the content directory and also a `robots.hbs` in the template directory. These are shown below (in that order).
+
+```
+title = "Robots"
+description = "This is the robots.txt file"
+date = "2021-12-30T03:17:26Z"
+template = "robots"
+content_type = "text/plain"
+---
+
+This is the robots.txt file. It is autogenerated.
+```
+
+```
+{{! 
+For info on what can be placed here, see http://www.robotstxt.org/
+See also: https://developers.google.com/search/docs/advanced/robots/intro
+}}
+User-agent: *
+Sitemap: {{site.info.base_url}}/sitemap.xml
+Disallow: /index
+```
+
+## Google Search Console
+
+The above steps of a) verifying and b) complying to the SEO requirements will give you great control over what is indexed by the Googlebot, and other Web Crawlers also. From a Google Search Console perspective specifically, you can now enjoy specific features and benefits such as on-demand page indexing i.e. allowing Google to go ahead and index specific content (like a new blog post) and much more.
+
+## Google Search Analytics
+
+<a href="https://g.co/kgs/xZqj9L" target="_blank">Google Analytics</a> tracks and reports web site traffic. Showing not only where users are visiting from but how long they are staying and which pages they are reading and so forth. Once Google Analytics is configured you can even see how many users are on the site in real time. It is recommended that Google Analytics be used in conjunction with the above SEO.

docs/templates/content_sidebar.hbs

@@ -18,6 +18,7 @@
         </p>
         <ul class="menu-list">
             <li><a href="{{site.info.base_url}}/contributing">Contribution guide</a></li>
+            <li><a href="{{site.info.base_url}}/seo">Search Engine Optimization (SEO)</a></li>
             <li><a href="{{{page.head.extra.url}}}" class="button is-round is-outlined is-size-6 is-medium">
                 <svg width="0.825em" height="0.825em" viewBox="0 0 32 32" fill="none" xmlns="http://www.w3.org/2000/svg"><path fill-rule="evenodd" clip-rule="evenodd" d="M16 0C7.16 0 0 7.16 0 16C0 23.08 4.58 29.06 10.94 31.18C11.74 31.32 12.04 30.84 12.04 30.42C12.04 30.04 12.02 28.78 12.02 27.44C8 28.18 6.96 26.46 6.64 25.56C6.46 25.1 5.68 23.68 5 23.3C4.44 23 3.64 22.26 4.98 22.24C6.24 22.22 7.14 23.4 7.44 23.88C8.88 26.3 11.18 25.62 12.1 25.2C12.24 24.16 12.66 23.46 13.12 23.06C9.56 22.66 5.84 21.28 5.84 15.16C5.84 13.42 6.46 11.98 7.48 10.86C7.32 10.46 6.76 8.82 7.64 6.62C7.64 6.62 8.98 6.2 12.04 8.26C13.32 7.9 14.68 7.72 16.04 7.72C17.4 7.72 18.76 7.9 20.04 8.26C23.1 6.18 24.44 6.62 24.44 6.62C25.32 8.82 24.76 10.46 24.6 10.86C25.62 11.98 26.24 13.4 26.24 15.16C26.24 21.3 22.5 22.66 18.94 23.06C19.52 23.56 20.02 24.52 20.02 26.02C20.02 28.16 20 29.88 20 30.42C20 30.84 20.3 31.34 21.1 31.18C27.42 29.06 32 23.06 32 16C32 7.16 24.84 0 16 0V0Z" fill="#24292E"/></svg>
                 Edit on GitHub