Hugo and Static Files

Hugo and Static Files

Byrichard

Yesterday I was experimenting with the Static folder in Hugo. Hugo and other static site generators has a folder where you can usually put content that you don’t want to have changed. You can add php, css, js and more. By making this an option it is possible to have your blog as markdown files that are updated and published every time you make changes while other files remain intact.

PHP Files

I spent time converting Roman civilisation content to PHP, as well as the geography section. Now that I have invested that time in learning to use PHP in production I don’t want to lose that content by adding it as markdown pages. With PHP and HTML pages I have full creative control on how content looks and behaves.

Legacy Content

Back in the 90s and the Zeros (2000s, but I like saying zeros), we would share photos of an event by creating a gallery and sharing that. These galleries are simple html with a page for each image, and plenty of pages to update and navigate between. These pages can be brought into the 21st century but by parking them in the static folder in Hugo we keep access to them, until we have written the json or other file type to display these photo galleries as a single page app.

Hugo’s Behaviour

There is an explanation of what Hugo does to Static files. When files and folders are added to the static folder they are added to the watch list, if Hugo is running. This means that static content will automatically be updated and available on the local instance.

Static Files and the Public Folder

When you add static files to a Hugo sight Hugo will track changes and show them via the web server but they are not added to the public folder. It is up to us, to make sure that the files that are static are updated and uploaded to the server if and when we make changes to them.

That static files are not added to the publish folder is useful. I was worried that by adding thousands of files to the static folder thousands of files would be uploaded every time I wrote a blog post and published it. This isn’t the case. I do not need to take a break from blogging with Hugo, whilst I experiment with the static part, and prepare a new folder architecture.

Looking Forward

To avoid confusion, as I experiment I put all the static files in a sub-folder. In so doing I avoid the risk of duplicate file names or file names with the same names, but different extensions, confusing Hugo. I aim to keep it clean, until I have decided on what I want to achieve with my experimentation.

And Finally

I have been struggling with Eleventy and Hugo whilst it’s easy to setup one stream of content, it is not easy to have two or more in parallel. For this reason using the Static folder allows me to keep the part that already works, separate from the part that I am still working on. It enables me to put my experiment into production sooner, and to stop going around in circles.

Blogging with VS Code and Hugo

Blogging with VS Code and Hugo

Byrichard

Yesterday I had no inspiration. In the end I did write about something but only after hours of staring at a hypothetical blank page. When I did start writing I used Frontmatter to generate the page but I forgot to open terminal and write the blog post using VIM. I used VSCode and Markdown. Whilst this might sound ordinary to most I did this because I like writing blog posts with VIM as it gets me to learn, over time, how to use it, automatically, rather than by struggling.

Very Fast

I really like blogging with hugo, VIM and VS Code because it’s quick and efficient. With Frontmatter I can create the front matter I need for the Markdown document, and then I can start writing, until inspiration dries up, or I have nothing left to say.

Git, Git FTP and VIM

I then add the blog post to git, type hugo, generate the new page, and then git add all the publish files before going to a second directory and writing git FTP publish. If I wanted to be more of a purist I could do everything from within one or two terminal windows. One terminal window is to keep a live preview of the page, and the second is for git, git ftp and VIM.

The hugo Command and FTP

Although my site is large it is still faster to run hugo, to update and then ftp the site. Although FTP is seen as old fashioned it works well. I did add a safety feature. I created an FTP account just for the blog and I specified the directory. This means that if people do get access to that FTP account they can only change what is within the blog directory, nothing else. I’m applying the “minimum privileges to get things done” principle to keep the site secure.

Slow WordPress

When you go from VSCode, vim and git to keep a site up to date to WordPress Wordpress feels slow and clunky. WordPress requires you to be online, to navigate to the right page, and the create content, before eventually saving, and checking that it works. The 26 seconds that it takes to generate the latest page and associated pages still feels faster than WordPress.

And Finally

The beauty of using Hugo rather than WordPress is that it’s fast. It’s almost instant to navigate to all pages. The let down is that you do need to spend some time reading the fantastic manuals to learn how to do specific things. With WordPress it’s built in and simplified.

Static Web Generators and CMSes

Static Web Generators and CMSes

Byrichard

For weeks, or even months, by now I have been playing/experimenting with Hugo, 11ty and other solutions. I really like that with Hugo I can use FrontMatter as a CMS to create new posts, add the appropriate meta data, and keep track of what is published and what is in draft form. It allows me to create posts with the right metadata in seconds, rather than having to write the date, time, draft status and more by hand. It also generates the right file title for good archival practices.

Decap CMS

As I was looking for a CMS tool to make managing 11ty content easier I came across Decap CMS and it seemed interesting. I installed a version locally, and then I started to look at the code manually, rather than using the CMS tool. It felt complicated so I did some more research. Eventually I learned that in order to play with Decap CMS you need to setup a netlify account, a github account and then expose yourself to accidental charges when playing with a static website generator. I was struck by the paradox. Why would you use a CMS tool that requires you to commit to an external hosting tool? Why not use ClassicPress or WordPress and cut out the middle man. Of course the short answer is “because you still generate a static tool, but the interface is intuitive for non coders.

Yet Another Service

By requiring us to set things up via Netlify we’re forced to use yet another service, which is fine, when you’re using the service in the first place. I am not.

FrontMatter.Code

Within a few minutes frontmatter.codes could be setup locally do do what I want, to manage documents and frontmatter for an 11ty site. In so doing I keep development on the local machine, only connecting to the external server when I’m uploading site changes. I can use the same workflow as I have for Hugo, once I set it up.

ClassicPress and WordPress

It’s easier, for me to setup a ClassicPress or WordPress CMS and use that. ClassicPress feels very fast and I can use markdown or html for pages that I am creating, or that already exist. Within a short amount of time I can do what Decap CMS does, anywhere I want.

For WordPress you can use this method/tutorial or with the free playground option. Within seconds you can have a wordpress instance running on azure, up and ready for a new site and content.

In particular, while App Service F1 will not generate any cost, database usage is chargeable for “pay as you go” plans or when the usage limit of 750 hours per month for 12 months is exceeded. So, in order to ensure they will not pay for the WordPress playground, developers should monitor and track their database usage.

With this tool a wordpress instance is prepared for you, and for a month you can see what the cost would be, before jumping into a financial commitment.

And Finally

If I am experimenting with a Static website generator like Hugo or 11ty I want to have local versions to play with, rather than remote ones that may cost something if I am not careful. If I’m reading it correctly the basic plan I’m experimenting with is 3 CHF per month for a server in Northern Switzerland. With this “playground” I have the opportunity to experiment, and see whether that is the case.

The testing options are cheap, but for production Azure and other cloud solutions are expensive, which is why we use other cloud solutions, especially for personal sites. I will spend time experimenting with Frontmatter, set up for 11ty, following this learning experience.

Hugo and Upload Time

Hugo and Upload Time

Byrichard

I like blogging with Hugo. I like writing my daily article in VIM and then exporting it to WordPress, before publishing it both with Hugo and WordPress. The build time for this build was 14 seconds but it can take up to 20 seconds. Although this i slow this isn’t the bottle neck.

New Files and Hugo

Every time you create a blog post, and every time you use a keyword that’s tag page has to be regenerated. If you use two, three or more tags then three or four pages need to be updated. As well as these pages needing to be regenerated so does the site’s index page, and others. If you include related pages at the bottom of the page, then every article with the tag also needs to be updated. The result is that tens, or even hundreds of pages need to updated at a time.

FTP Compare

Every day I write a blog post and every day I use Hugo to rebuild the website. All the files that have been updated need to be uploaded to the server. I don’t want to overwrite every page, when it makes more sense just to overwrite the updated pages. This usually takes time. I experimented with Filezilla and Cyberduck. Both take at least an hour or two every single day, to work through and upload the new pages. This is time consuming and inefficient. With WordPress, Drupal and other solutions, you write the post, publish and you’re done. The same is true of Hugo with smaller websites.

When a website has thousands of articles and tags Hugo becomes slow, at the upload stage. It’s at this point that the switch to a faster solution makes sense.

And Finally

I will continue to experiment with Hugo and my blog but I will not publish on a regular basis, since uploading takes time. I will upload once a week to save time and resources. I have come up with a workflow and it works. I have learned at least the basics of Hugo and now I can move on to carry out more experimentation.

YQ, FrontMatter and Jekyll

YQ, FrontMatter and Jekyll

Byrichard

For two days I have been trying to understand how to use Jekyll, a Ruby version of Hugo. It is a Static website generator that is similar to Hugo but rather than being written in Go, it is written in Ruby. I find that it renders sites faster than Hugo, but that it has less assistance for creating pages, giving them layouts and the rest. That’s why I experimented with YQ


a lightweight and portable command-line YAML, JSON and XML processor. YQ uses jq like syntax but works with yaml files as well as json, xml, properties, csv and tsv. It doesn’t yet support everything jq does – but it does support the most common operations and functions, and more is being added continuously.


I read about it in this article. You can install YQ with a brew command brew install yq or you can install it in a number of other ways, depending on whether you’re using MacOS, Linux or Windows.


The Use case


Exporting a WordPress blog to MD files can be done with a multitude of tools but they don’t add all the FrontMatter that you need for a blog, especially Jekyll. I imported the most recent blog posts, that I created for Hugo, with a year-month-date-title.md name to the posts folder but I created a second one for the archive. As the file title was different Jekyll did not like the files. The problem was the file name format, so that’s why I created a separate directory.


Side Note on Reading the Archives MD Files


Bard’s help: I provide this to help you understand Jekyll logic, as well as Bard help.


Getting It To Work On MacOS


In the article about using YQ to edit FrontMatter for Hugo pages the first part worked fine, but it’s written for Linux rather than macOS so some commands failed. This is where Bard comes in.You can ask Bard to explain what parts of a line of code or a command does and it will explain it to you, and you can eventually transfer the command that is understood by one OS and get it to work with the other.


I tried this find -name “*.md” -exec yq –front-matter=”process” ‘.updated_at = now’ {} \; but it failed to work. Bard was kind enough to explain the options for me to figure out which options to use. I added the correct path, and specified that I was looking for the files with the ‘type f’ flag.


find ./_posts -type f -name “*.md” -exec yq ‘.title’ {} \; To see the title of .md files 


I wanted to add these two lines to the FrontMatter of the files in the relevant folder.


I tested find ../unprocessed “*.md” -exec yq –front-matter=”process” ‘.updated_at = now’ {} \; and this works but does not make changes.


This works and updates the pages to add the updated at field to the metadata with the keyword now. The -i tells it to write to the file.


The full command is find ../unprocessed -name “*.md” -exec yq e –front-matter=”process” ‘.updated_at = now’ -i {} \; works to add updated now tag .


Implementing the Required Change


The lines that I wanted to add to the front matter were ‘layout: post’ and ‘categories: [archives]’. After the trial and error described above I tried:


find ../unprocessed -name “*.md” -exec yq e –front-matter=”process” ‘.updated_at = now’ -i {} \; works to add updated now tag


find ../unprocessed -name “*.md” -exec yq e –front-matter=”process” ‘.layout = post’ -i {} \; works to add updated now tag


Finally the command to make a permanent change was:


find ../_archives -name “*.md” -exec yq e –front-matter=”process” ‘.layout = “post”‘ -i {} \; 


find ../_archives -name “*.md” -exec yq e –front-matter=”process” ‘.categories = “[categories]”‘ -i {} \; 


  • A Quick explanation. The ../archives needs to be replaced with the name of the folder your files are in, when terminal i in the same folder. .layout and .categories are the names of the field to add and the “text” part is the content that you want to add to the front matter.

  • I have unprocessed and archives folders because I tested the code on two files to start with, and when I saw that it worked, then I moved to the main archives folder. I also used git to have a backup of the latest versions. If something had gone wrong I could have “stashed” the mistake, and been back to normal.


The first line ads the layout information to the FrontMatter post and the second one adds the categories information to the posts. Instead of spending hours doing something manually I was able to find the right tools to do what I wanted within seconds.


The Process


I knew what I wanted to do. I started with a google search to see if I could find a tool with some instructions and I did. The instructions were useful, but of limited use for my setup so I used the app’s documentation to try one thing, establishing that this worked, before moving on to accomplish the task that I had set for myself-. Google Bard was able to provide me with some help, but so did chatGPT.


The value of AI tools, in my eyes, is to help us understand the code we’re looking at, but also to help us tweak it for the OS we’re using, when we get error messages. I appreciate that with AI we can ask “I’m getting this error message, why?” and it will help find an answer to the question.


If a web article had provided me with a solution that worked I would have stuck to the web page and I would have little to write about. It is because of trial and error, and using four sources to achieve the goal that I wanted to achieve, that it becomes blog worthy.

Playing with Front Matter

Playing with Front Matter

Byrichard

For those who want to use Hugo, but find post creation and management complicated there is a FrontMatter plugin that should help. This is a plugin that makes it easy to create new posts in the category of your choice. It automatically creates the title, description field, that you need to populate, date, preview, draft status, tags and categories. It displays each post as a block and you can see within seconds whether a post is published, or just a draft.


Server Spin Up


If you want to check how the page that you’re working on works you can spin up an instance and every time you stop typing the layout will be updated to show your changes. It’s quick and easy to use. Instead of remembering the commands to spin up an instance, close it and more, the plugin does that.


File Managment is Great


Although your old file names,from importing to Markdown from Wordpress might be a mess, every new post is called year-month-date-title. This means that as you write new posts they are ordered in chronological order. Finding the most recent posts is quick and easy.


The Draw Back


The key draw back that I have come across so far, is that algough it plays very well with markdown, it ignores html files. This limits the app to people creating markdown sites, rather than HTML ones. This is a shame, since I really wanted to use it for my static site, rather than just this blog.


Within VS Code


The App runs within VS Code so you get all of the plugins and functionality that you are used to, within VS Code. If you are used to using VIM and commands this interface is much easier, for those unfamiliar with VS Code. It’s quick, it’s efficient, it’s clean. You need to spin up the server to see a preview of the page you’re working on.


Intuitive


This is a quick and intuitive way to write content for a Hugo website, without the need for a website running a CMS. It requires a shorter learning curve than my previous workflow.


Tagging


The Plugin generates a json file and within this file it keeps a collection of all the tags that you have used. When you create a page you can then type a few characters and choose the relevant tags. This helps keep tags consistent. When tagging it’s easy to write “Switzerland” in one place, and “switzerland” in another. By keeping track of all tags you can avoid having too many similar tags. Each tag is a page, so duplicate tags split the results page in two.


Just the Surface


I have only skimmed the surface, for now. I eill explore and learn more about how to use FrontMatter.

How to Audit Hugo HTML

How to Audit Hugo HTML

Byrichard

With Hugo you can generate entire websites within milliseconds if they’re small and seconds if they’re large. Within a very short amount of time thousands of pages are generated. If you went through and checked each page then this could take hours, or even weeks, depending on the size of the site. To save time Hugo does have a command to check for html in Markdown in seconds. The next step is to see the title of the markdown pages with an issue and fix them individually.


You can run an html audit on Hugo by running the command: 


HUGO_MINIFY_TDEWOLFF_HTML_KEEPCOMMENTS=true HUGO_ENABLEMISSINGTRANSLATIONPLACEHOLDERS=true hugo && grep -inorE "<\!-- raw HTML omitted -->|ZgotmplZ|\[i18n\]|\(\)|(<nil>)|hahahugo" public/ .


It will go through your website and check that all html is working properly.


When to Use It


The best time to run an audit is when you have imported a wordpress blog to md, or from some other source to markdown. Markdown recognises some HTML but not all. I found that it is especially at comments that markdown was confused. It saw the time and date tags and detected an error. It also generates errors when it comes across html tags. If you want to display code with md you need to use back example code for the code to be ignored. \ Does not work in markdown.


How it Helps


Auditing Hugo before building helps to detect flaws that are not visible unless you check the source code for each page individually. With this simple solution you can check the entire website when you build. The first time you run it you will find errors, but over time those errors will increase. In the tutorial they say that you will run it once to check, and a second time for the official build. I ran it more often than that. The errors I got were intuitive to fix. Look for html code in markdown and you’ll find the problems. Markdown does not like smileys so if comments are converted to markdown, you need to remove the smileys.


The article is here if you want to learn more.


And Finally


If you want to learn more about Hugo you can read the tutorials pages. I also use markdownlint in Visual Studio for help with checking markdown and ensuring that it is correct.

Playing With Hugo – Continued

Playing With Hugo – Continued

Byrichard

Yesterday I decided to write a blog post using Hugo and HTML rather than markdown and it worked fine. I was able to write the post, checked that everything was displaying properly, and then noticed that with HTML the theme I am using does not detect section headings.


Markdown Behaviour


With Markdown when you write a blog post you can add headers with # and it will give the heading the weight and position in the list that it needs. If you look at the top of the page you can click on a heading and you will be taken to it. No need to scroll, as this is done for you.


HTML Behaviour


When you write html rather than markdown you tell it whether you want one type of heading or another, and everything diplays well on the page. The drawback is that if you use the article map at the top of a post then that map is empty. The result is that it makes more sense to write markdown, unless you want to add embeded content. If you want to embed youtube videos you need to create an HTML page and add the code for the embed. With Markdown you do not have this freedom.


Markdown is Quick and Light


I find that I like markdown and Hugo because I can write blog posts using Terminal and Vim. With Wordpress I was writing the blog post in Day One and then copying it over to the WordPress Blog. I did this for two reasons. The first is that I don’t trust a browser to keep what I am writing safe. The second is that I found that Day One can sometimes be sluggish or crash. I don’t want to use an app that crashes.


An Unrelated Change


The computer I am using six years old, or so, and it was feeling sluggish recently so I took the time to delete a few apps that were running in the background without being used. Since I removed those apps I have spare Ram and the computer is behaving better than before. I removed all the Jetbrain Apps, as well as Boinc and other apps. I enjoyed contributing to grid computing but my computer is old now, and I’d rather keep it running well for what I want to do.


And Finally


Now that I have the workflow setup to write Hugo Blog posts I may stop updating the WordPress blog. WordPress feels slow, and bloated now. So many features have been added, that it has become slow and clunky. I want to write a post, check grammar, and then publish. With WordPress I can do this but it’s noisy. The experience requires ignoring a lot of information. I like how clean and elegant Hugo is as a simple blogging option.

Transitioning from WordPress to Hugo

Transitioning from WordPress to Hugo

Byrichard

Transitioning from WordPress to Hugo is tempting because I don’t need an entire CMS for what I’m doing. What I need is a centralised system that checks for tags, titles and the theme, and updates the navigation as I add new pages. You don’t need a CMS for that.


The Good Old Days


If you go through the meta data for many of my static pages you will see that they were created with dreamweaver, frontpage 2000 and other solutions. The aim of these was to create an application that would prepare the html and navigation for pages, so we could concentrate on content creation. This was before the age of the CMS.


A short Learning Curve


If you want to create a simple blog, without changing much the learning curve for WordPress and Hugo is short, compared to what is required for Laravel, Angular and other solutions. You create content and there are tools that will take care of almost everything else.


With Hugo and WordPress you can, theoretically, start creating content and sharing it within minutes, or even seconds, depending on how many times you have set things up before.


HTML or Markdown


With Hugo if you want to create a simple page you can use markdown and save a lot of time and effort. You focus on headings, images, lists and hyperlinks. If you want to have more flexibility you can generate an HTML page, and with this you can do anything you want, and have the experience, to do.


  • hugo new blogs/big-timber.html


  • hugo new posts/hugo-and-wordpress.md


A Huge Mess


I need to find a solution to a problem. I have a blog with thousands of articles so I have thousands of markdown pages in a single directory. I considered using the creation date with year-m-d-article-title.md but if I do this then Hugo will use that as the headline for new blog posts. I would need to edit the Title of new posts, to remove the date information. For the public folder, once the blog is exported the files are tidy.


Thousands of Files and Directories


Hugo likes to create a folder and file for every single page. The result is that every page has its own folder. That’s fine, when everything is working well, but chaotic every time you make changes. At the moment of writing every change generates 6000 files and folders. That’s 10-20 seconds per export, and this doesn’t include the upload time.


Minor Changes With A Big Cost


When you’re using a CMS like wordpress changing a theme takes seconds. You find a theme, apply it, and a few seconds later you’re ready to role. With Hugo if you change the theme you need to regenerate and then reupload the website in its entirety. Small changes become big changes. Having said this, with a static website, managed by hand, rather than Hugo, this would take hours, rather than seconds.


Very Fast, Flexible


I see that Hugo is fast and flexible. It provides the organisational features of a CMS with the flexibility of HTML pages, when that is desired. This makes it easy to circumvent the limitations that are imposed by markdown, for those that want to embed videos or other features into a specific web page or blog post.


And Finally


Hugo requires no back end. Anywhere that you can upload html, can display a Hugo website, or individual pages. You have the ability to tell Hugo where the files will be on a server, so that links are built to play nicely. Both the static part of my website, and the blog may soon run via Hugo.

Converting This Blog To Hugo
|

Converting This Blog To Hugo

Byrichard

On Saturday I converted my blog from WordPress to Hugo as an experiment and it went quite well. I downloaded the xml file and then I converted that xml with blog2md to go from xml to md pages. A page was created for every single blog post. This took a while. I now have my blog both as static pages, and as a wordpress blog.


Two challenges


There are two challenges. The first is that Hugo has to generate over 5000 pages each time I generate the site, or at least it has to check 5000 pages per change. That’s several minutes of work, every time I tell Hugo to build the site. The next concern is that if I make a change I don’t know whether I need to re-generate the entire site to reflect that change or whether it will just generate one page. I can run tests using Git Version control to see what’s happening.


Is Hugo or WordPress Lighter


I don’t know whether Hugo is heavier, because of every individual file, or whether WordPress is heavier, due to all the functionality that I barely ever use. WordPress often feels bloated and slow, especially in the admin console. It would be nice to play either with markdown, as I’m doing for this post, or html, as I could do, if I want more flexibility


Images Still Load


One of the surprises I came acrosss is that images still load, whether I am in the blog directory I created for testing purposes on the website, or within the wordpress blog folder, as I did by mistake. It’s pretty easy to make the change, as long as you keep the assets where they’re expected. I might consider deleting most of the wordpress install except for the media assets. That will be decided lateri.


What I Like About Hugo


What I like about Hugo is that the blog is generated and links between pages are taken care of automatically. As long as the tags and categories are set ahead of time, the generator will take care of the rest. As I browsed through the blog generated by Hugo I liked the look and feel. That’s before I spend time tweaking the theme and trying to add more.


An Alternative Approach


The blog I use now has been around for 20 years or so. This means that if I make a big change, like migrate from WordPress to Hugo I will kill all the hyperlinks and hierarchy and it will take time for search engines and links to be written to make the new version visible. Having said this the traffic to the site is low and there is a chance that the change will not be noticed. It’s by updating and linking to pages that I will make the entire site more visible.


And Finally


I have written the last two blog posts using VIM and I like the interface. It’s limited in functionality, because I haven’t spent enough time playing with VIM but it feels fast, and effective as a blogging tool. If I use Day One it takes time to load, I theoretically need to pay a subscription and then I need to cut and paste the content from there, to the blog. With this I could press esc, :wq, save, and write hugo and it will upload automatically. My reason for not doing that, for now, is that I don’t want to risk sharing the configuration for connecting to the server. Time will decide.