Post-Release Portfolio Blog
archivedThis content has been split into a six-part Portfolio Design series: 1. Portfolio Design #01: Introduction, Hosting, & Architecture Overview (Edge) 2. Portfolio Design #02: User Interface & Experience (Frontend) 3. Portfolio Design #03: Content Processing & Auth Flow (API) 4. Portfolio Design #04: Content Management (Database, Caching, and Storage) 5. Portfolio Design #05: Monitoring & Analytics 6. Portfolio Design #06: GitOps & Going Live
Context
After a week of “just two more things, and I’ll be done,” I am finally done with my initial version of my new portfolio + blog website.
==Note: I will be updating this as time progresses if I make any new discoveries and major changes.==
Requirements
There were two sets of requirements – aside from the usual functional and non-functional requirements: realistic and ideal. For functional, it’s the same for both.
Functional Requirements
- Users can access my portfolio (
/) - Users can view blog posts listing + individual blog posts.
- Users should be able to filter to find relevant content.
- Users should only view published and archived posts.
- Blog posts are quick to display (i.e. no processing client-side)
- An admin (I) can have a secured admin view.
- An admin (I) can update page content without making manual DB queries.
- An admin (I) can get view page analytics to get insights on performance.
- An admin (I) can upload Markdown content, which would display as HTML content for users.
- It should cost < $5/mo.
Scale
- Daily-Active Users
- Realistic: 5 DAU
- Ideal: 1000 DAU
- Posts: 1 Per-Week
Non-Functional Requirements
- High-Availability > Consistency
- I cared more about reader-access, it’s okay if blog posts take a while (1 day max) to propagate CRUDs.
- < 250ms latency
- Caching should alleviate database load.
- Since we’ll be doing server-side processing of posts, we’ll want to ensure we cache efficiently to avoid this constant load whenever a user accesses a blog post.
- The system should be non-blocking when an admin makes content updates.
Architecture
Hosting
I already owned the https://fabianswe.com domain name through AWS Route53, so I was heavily leaning towards AWS services.
Between Two Methods
Initially, I was looking into EKS since I have Kubernetes experience. In my opinion, it would have been easy to set up a micro-service architecture and dogfeed kGateway to handle ingress traffic, or even Istio Ambient Mesh to handle inner-cluster traffic security + resiliency. But, alas, my limit on costs was noticeably lesser on a simple VPS. Because of this, I settled with using Docker Compose to spin up my service(s) and host it on a VPS.
Pricing
EKS Pricing (based on AWS calculator): \(70/mo **Lightsail Pricing (based on AWS calculator)**: ~\)40/mo
When I dug more into this, since I was being cheap, I settled for a medium instance.
Specs: 1 GB Memory, 2 vCPUs Processing, 40 GB SSD Storage, 2 TB Transfer.
Overall, the simple Lightsail instance is costing $7/mo, and I get 90 days free.
For DoS and bot protection as well as some distribution benefits (as I’m not hosting this in a distributed manner), I used Cloudflare’s free CDN tier.
The price went \(2 above my ideal, but I knew I couldn't be _too_ cheap. I was primarily trying to drop a previous host that cost around \)8-$15/mo, since it was a hybrid cluster + serverless approach. For a simple portfolio and self-managed blog, this was overkill.
Deployment Configuration Overview
As I brought up above, I use Docker Compose to set up my project with all its requirements and deploy it in the VPS. Below is my configuration setting up three networks for isolate as-needed.
Networks
- edge: edge proxy
- backend: communication between apis + databases
- monitoring: analytics-related services
services:
# tsdb for easy storage of analytics data which we scrape in api
prometheus:
image: prom/prometheus:v3.8.1
container_name: portfolio-prometheus
volumes:
- ./prometheus/:/etc/prometheus/
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/usr/share/prometheus/console_libraries'
- '--web.console.templates=/usr/share/prometheus/consoles'
- '--storage.tsdb.retention.time=7d'
- '--storage.tsdb.retention.size=5GB'
networks:
- monitoring
- backend
depends_on:
node-exporter:
condition: service_started
cadvisor:
condition: service_started
restart: always
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
# getting analytics from node
# note: technically not validated outside of dev environment YET
node-exporter:
image: prom/node-exporter:v1.10.2
container_name: portfolio-node-exporter
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
- /:/host:ro
command:
- '--path.rootfs=/host'
- '--path.procfs=/host/proc'
- '--path.sysfs=/host/sys'
- --collector.filesystem.ignored-mount-points
- "^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)"
networks:
- monitoring
restart: always
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
# getting analytics from containers
cadvisor:
# image: ghcr.io/google/cadvisor:0.55.1 <- fails to do docker registration
image: infocus7/cadvisor:0.55.1-grace # <- gracefull fallback for when containerd is not available, my local changes
container_name: portfolio-cadvisor
volumes:
- /:/rootfs:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /sys:/sys:ro
- /sys/fs/cgroup:/sys/fs/cgroup:ro
- /var/lib/docker:/var/lib/docker:ro
- /dev/disk/:/dev/disk:ro
privileged: true
networks:
- monitoring
command:
- '--docker_only=true'
restart: always
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
# relational db for content
postgres:
image: postgres:16-alpine
container_name: portfolio-postgres
environment:
POSTGRES_USER: ${POSTGRES_USER:-portfolio}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-portfolio}
POSTGRES_DB: ${POSTGRES_DB:-portfolio}
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-portfolio}"]
interval: 10s
timeout: 5s
retries: 5
networks:
- backend
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
# app, which also exposes frontend + handles in-memory caching
api:
build:
context: .
dockerfile: Dockerfile
image: ${DOCKERHUB_USERNAME}/portfolio:${VERSION:-latest}
container_name: portfolio-api
environment:
DATABASE_URL: postgresql://${POSTGRES_USER:-portfolio}:${POSTGRES_PASSWORD:-portfolio}@postgres:5432/${POSTGRES_DB:-portfolio}?sslmode=disable
PORT: 8080
SESSION_SECRET: ${SESSION_SECRET:?"SESSION_SECRET must be set in production"}
ADMIN_SETUP_TOKEN: ${ADMIN_SETUP_TOKEN:?"ADMIN_SETUP_TOKEN must be set in production"}
ENV: ${ENV:-production}
BASE_URL: ${BASE_URL:-https://fabianswe.com}
expose:
- "9091"
ports:
- "8080:8080"
depends_on:
postgres:
condition: service_healthy
prometheus:
condition: service_started
networks:
- monitoring
- backend
restart: unless-stopped
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/api/v0/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
# light reverse proxy
caddy:
image: caddy:2-alpine
container_name: portfolio-caddy
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
- caddy_config:/config
depends_on:
- api
networks:
- edge
- backend
- monitoring
restart: unless-stopped
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
volumes:
postgres_data:
caddy_data:
caddy_config:
prometheus_data:
networks:
edge:
backend:
monitoring:
Ingress Traffic
I have a simple Caddy reverse proxy running that handles simple reverse-proxying. This acts as the only exposed service port to avoid bad actors.
www.fabianswe.com {
redir https://fabianswe.com{uri}
}
fabianswe.com {
reverse_proxy api:8080
encode gzip
tls {
protocols tls1.2 tls1.3
}
# Security
# CSP header is set by the application with dynamic nonces for better security and dynamic script loading -- more on that in the backend/frontend section
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
X-Content-Type-Options "nosniff"
X-Frame-Options "DENY"
X-XSS-Protection "1; mode=block"
}
# Logging
log {
output file /var/log/caddy/access.log
format json
}
}
Database
Storing my Blog Posts
I needed a database for blog posts and user (my) data. Something that could withstand server restarts, easy to migrate, secure (because my login is in there), and one which I can set defined structure within.
It must also allow for easy configuration of performant content searching. For instance, if a user wanted to search “go” for go articles.
SQL vs NoSQL
For my self-hosted blog, there wasn’t much “relational” requirements, such as cross-table querying – I own all the blogs. I wanted to keep performance top-notch, so I also wanted to self-host my choice.
While a distributed NoSQL database (e.g. MongoDB, which has historically been my go-to) would have been easiest to set up, I knew I was going to host this in a simple single-server environment, so I would not see the benefit of a NoSQL database or hit a drawbacks of SQL databases (distributive configurations). Having a strong, structured Post: {id, author, excerpt, content, ...} that I could easily model alongside my API was most-ideal.
In the end, I went with PostgreSQL as it’s open-source (woo!), is structured, has support of trigrams (for text search), and I had previous experience using it. I was able to easily set up initial migration files so when spinning up the application, we have the Postgres database in the rightful state required.
Post Mortem Note: A mistake I made was instantly going with Postgres without delving into other lighter RDB’s like MySQL or SQLite. I would realistically only have about ~60 posts a year, so Postgres’s high performance is likely overkill. From some research during this post, I found that SQLite supports trigrams – which can help with fuzzy searching – as well.
Table Designs
Posts Table
The posts table was the first table I created. It needed to house post information (title, excerpt, generated slug, content-as-markdown), and over time grew to house additional features I thought of while working on the blog (i.e. readtime and is_featured).
| id | title | slug | is_featured | content | readtime | excerpt | created_at | updated_at | authors | tags | status |
|---|---|---|---|---|---|---|---|---|---|---|---|
| SERIAL | VARCHAR(255) | VARCHAR(255) | BOOL | TEXT | INT | TEXT | TIMESTAMP | TIMESTAMP | JSONB | TEXT[] | VARCHAR(50) |
Efficient Querying of Posts
Posts are the most important artifact stored in the database, which calls efficient searching and fetching.
B-Tree Indices (Regular) INDEX
slug- Finding exact slug match is key for loading
/blog/{slug}.
- Finding exact slug match is key for loading
status- When listing posts, we want to avoid listing drafted posts, and I wanted to keep it open for an optional ‘deleted’ status in case I want some for historical purposes.
- Index on status allows for an efficient
WHERE status IN ('archived', 'published')
tags- I will remove this. It was an initial implementation to allow tag searching before I looked into GIN (below.)
Generalized Inverted Indices (GIN) INDEX ... USING GIN
GIN is designed for handling cases where the items to be indexed are composite values, and the queries to be handled by the index need to search for element values that appear within the composite items - PostgreSQL GIN
tags- For efficient tag filtering.
- Allows efficient search criteria (e.g.
WHERE tags @> ARRAY['go','tutorial']).
Trigram Indices INDEX ... USING GIN (... gin_trgm_ops)
The
pg_trgmmodule provides functions and operators for determining the similarity of alphanumeric text based on trigram matching, as well as index operator classes that support fast searching for similar strings. - PostgreSQL PG_TRGM
I set up trigram indices on the below two fields to allow for fuzzy searching blog posts.
- title
- excerpt
Post-Mortem Thoughts: When querying unique tags, I’m currently querying for unique tags with with SELECT DISTINCT unnest(tags) FROM posts ..., which is not resource-friendly. Luckily, it’s a query that gets worse based on cardinality, so this should be no issue with my limited data. Relevant Read: Please Stop Using Distinct!!. To combat this, I could move to a Materialized View which comes with its own set of drawbacks, or better yet, create an extra in-memory cache for tags. The latter makes most sense, since it would reduce load on my DB, reduce latency, and will only realistically require ~(16 bytes + 8 bytes) * 50 => 1.2Kb, where 16bytes for string headers, 8bytes as the likely largest tag (“tutorial”), and 50 tags.
Calculating Estimated Blog Post Read-Time
For read-time, I followed something similar to Medium’s Read Time and You article:
Read time is based on the average reading speed of an adult (roughly 275 WPM). We take the total word count of a post and translate it into minutes. Then, we add 12 seconds for each inline image.
I took into account 3 things: words, images, and code lines. I also used a different average reading speed which is more forgiving. We walk the Abstract Syntax Tree (AST) of the Markdown content using goldmark library to count instances.
import (
"github.com/yuin/goldmark"
"github.com/yuin/goldmark/ast"
"github.com/yuin/goldmark/text"
)
const (
wordsPerMinute = 238
imagesPerMinute = 5 // 12 seconds per image
codeLinesPerMinute = 60 // 1 line per minute
)
type artifactCounter struct {
images int
codeLines int
words int
}
func CalculateWPM(markdown string) (int, error) {
// ast parse the markdown
parser := goldmark.New().Parser()
astNode := parser.Parse(text.NewReader([]byte(markdown)))
counter := &artifactCounter{}
ast.Walk(astNode, func() {...})
return ((counter.words / wordsPerMinute) + (counter.images / imagesPerMinute) + (counter.codeLines / codeLinesPerMinute))
}
While on a larger-scale system, having this on the same service app as everything else could lead to performance issues during high-post times, it is only me CrUD-ing posts, therefore this being done about once a week is negligible.
Experience Table
For my portfolio /, aside from personal information, I also needed a table to easily manage my experience: jobs, projects, and education. This and the posts table are the only two instances of multiple rows.
| id | name | location | type | role | description | start_date | end_date | created_at | updated_at | url | stack |
|---|---|---|---|---|---|---|---|---|---|---|---|
| SERIAL | VARCHAR(255) | VARCHAR(255) | VARCHAR(255) | VARCHAR(255) | TEXT | DATE | DATE | TIMESTAMP | TIMESTAMP | VARCHAR(255) | TEXT[] |
Efficient Querying of Experiences
B-Tree Index (Regular) INDEX
type- Allows for easy querying based on the distinction of
type(e.g.job,education,project)
- Allows for easy querying based on the distinction of
name- Allows for sorting by name – used in admin panel.
start_date- When returning experiences to display, sorting by start date is always done, and there is also sorting in the admin panel.
Profile Table
The profile table stores my personal information used to display in the hero page.
| id | name | phone | address | bio | skills | social_media | created_at | updated_at | |
|---|---|---|---|---|---|---|---|---|---|
| SERIAL | VARCHAR(255) | VARCHAR(255) | VARCHAR(255) | TEXT | TEXT | TEXT[] | HSTORE | TIMESTAMP | TIMESTAMP |
Note: There are no indices, as there is a single row (me), and is queried by $id.
Admin Table
| id | username | password_hash | recovery_codes | created_at | updated_at |
|---|---|---|---|---|---|
| SERIAL | VARCHAR(255) | VARCHAR(255) | TEXT[] | TIMESTAMP | TIMESTAMP |
Note: There are no indices, as there is a single row (me, admin), and the username is UNIQUE so it’s already indexed.
Ensuring Low Latency Through Well-Defined Caches
I defined a few in-memory caches to alleviate database load as well as resource (cpu) usage. I decided on managing the cache within the go service because adding an additional dependency (e.g. Redis) would add to the resource consumption in the database. I had simple caching use cases and a simple single-node environment that wouldn’t require the extra features that a vetted in-memory database may have.
Below have some very rough paper estimations which will update as I get real data about content space usage. Overall, I expect my caches to be < 10 MB total – excluding go type overheads.
Profile Cache
This cache primarily holds the profile data shown in the hero /. The profile content is Markdown processed to HTML which consumes resources. Because it’s (1) not ideal to do this database call each front page load, and (2) less ideal to do this processing per page-load, I had created the profile cache.
To ensure there is no stale data whenever updated, I: 1. Invalidated the entry whenever there is a profile update (rare.) 2. Added a 24hr TTL as a fallback, as this will not be frequently updated.
While it’s eventually consistent, it reduces stress on the server which is more important for a small server. I expect this to take < 10KB of memory – excluding map overhead.
Experiences Cache
This holds a simple list of all experiences, which are shown on the homepage /. Similarly to the profile, I render Markdown content to HTML, so more processing here. Because there can be a lot of experiences loaded at once, this one was also as important to store and avoid re-processing the same data when it’s updated very sparingly (less than once a month).
To ensure there is no stale data whenever updated, I: 1. Invalidate the entry whenever there is an update to experiences. 2. Added a 24hr TTL as a fallback, similar to the profile cache.
Again, on a worse-case scenario, it’s eventually consistent, but it reduces even more stress than the profile as there are multiple Markdown contents whose post-processed outputs are now cached. I expect ~3 KB per experience, so for < 10 pieces of experience in my case, this would take < 30 KB of memory – excluding map overhead.
Posts Cache
Posts, as shown above, have a few fields. Namely the content field which is Markdown. In a similar pattern as the others, the service processes the Markdown into HTML which it then renders. Posts can be of any length long with any number of markdown effects, which can cause the processing to take some time. This introduces latency. To avoid this, caching the post-processed data is key in providing optimal loading speeds and lessened hits on the processing.
I decided on a simple Least-Recently Used (LRU) cache implementation for the top posts. Another choice was the Least Frequent (Recently) Used, which may make more sense in the long-run when I have more posts or become limited in space, but for now K.I.S.S.
Storing the “top” 50 post-processed posts should be fair on resources. I expect ~40 bytes per slug, ~4 KB per Post object, ~40 KB per post-processed HTML, and ~16 bytes per linked list node pointer. So for 50 posts, I expect < 5 MB of memory usage.
Slugs Cache
This one was admittedly an afterthought, but another important piece of caching. One that could lead to unbounded /blog/{generated-fake-slug} requests hitting the database.
Previously I had had a simple handler on /blog/{slug} that would check for the slug in the cache, and if not there, then the database before returning a 404. While load testing with random URLs, I noticed my database service increase in cpu + memory usage ~10% when calling those URLs.
To combat this, I have added a cache for slugs that simply store a map[string]int32 from slug to id. Now, the blog page hits check for existence first before checking the cache and falling back to the database. Afterwards, Postgres resource consumption under the same load testing constraint only saw ~3% increase in usage. A caveat being that this cache is the source-of-truth so it needs to be eventually consistent.
To ensure there is no stale data whenever updated, I: 1. CRUD slug entries whenever the admin CRUDs posts – always consistent. 2. Set up initial slugs list on app boot by querying the database on initialization. 1. This helps with backwards compatibility on migration. 3. Added a manual slug cache refresh on the admin panel.
For the slugs cache, I expect ~40 bytes per slug, each ID is an int32 therefore 4 bytes. Raw memory usage, excluding map overhead, would mean at 100 posts, this would be ~4.5 KB. A slow-growing structure that will heavily reduce Postgres load on attacks is worth the caching.
Analytics For Fun with Prometheus
I deploy an instance of Prometheus to handle a time-series database, so I can get a bit more analytics about resource consumption and app internals such as cache hit/miss ratio.
Using prometheus/node-exporter and a (*minimally-modified) version of cAdvisor, I’m able to scrape metrics from the containers I’m running over time.
On a separate unexposed port :9091, I expose the /metrics endpoints with metrics data that Prometheus can scrape. I use both the client - for exposing the metrics data, and scraper(?) - for fetching that data for analytics.
For admin users (me), the admin can view /analytics endpoint which scrapes from the Prometheus instance running. It shows a clean view of page visits, memory consumption per container over time (selectable time range and interval), and more! I’ll post an image of it here when I set up image uploads for posts.
Cardinality for Exposed Metrics
portfolio_page_views_total (Counter)
Labels: page, slug, status, bot
Label Variations:
page: ”home”, ”blog”, ”post”, ”404:post”, ”404:other” → 5 valuesslug: ”“ (for non-post pages) or actual post slug → 1 + N posts (unbounded)status: HTTP status codes (e.g., ”200”, ”404”, ”500”) → ~5-10 values, realisticallybot: ”true”, ”false” → 2 values
Base: 4 pages * 1 slug * 10 status codes * 2 bots -> 80 time series
Per Post: 1 post * 1 slug * 10 status code * 2 bots -> 20 time series per post
Total: 200 + (20 * N)
portfolio_cache_requests_total (Counter)
Labels: bucket, result
Label Variations:
bucket: ”profile”, ”experience”, ”post_list”, ”post_by_slug”, ”slug” → 5 valuesresult: ”hit”, ”miss” → 2 values
Base: 5 * 2 = 10 timeseries (fixed)
Total: 10
portfolio_referrers_total (Counter)
Labels: category, page, slug
Label Variations:
category: ”direct”, ”search”, ”social”, ”other” → 4 valuespage: ”home”, ”blog”, ”post”, ”404:post”, ”404:other” → 5 valuesslug: ”“ (for non-post pages) or actual post slug → 1 + N posts (unbounded)
Base: 4 * 4 * 1 -> 16 timeseries
Per Post: 4 category * 1 page * N slug -> 4 time series per post
Total: 16 + (4 * N)
portfolio_http_duration_seconds (Histogram)
Labels: page, slug
Label Variations:
page: ”home”, ”blog”, ”post”, ”404:post”, ”404:other” → 5 valuesslug: ”“ (for non-post pages) or actual post slug → 1 + N posts (unbounded)
Base: 5 pages * 1 slug = 5 timeseries -> 40 timeseries
Per Post: 1 page * N slugs -> 1 time series per post -> ~8 time series per post
Total: 40 + (8 * N)
Note: Histograms also create additional fields, so actual is ~8x more to account for these.
Total Cardinality Estimate
| Metric | Base | Per-Post | Total |
|---|---|---|---|
| page_views_total | 80 | 20 | 80 + 20N |
| cache_requests_total | 10 | 0 (fixed) | 10 (fixed) |
| referrers_total | 16 | 4 | 16 + 4N |
| http_duration_seconds | 40 | 8 | 40 + 40N |
| Total | 146 | 32 | 146 + 64N |
The post slugs are the biggest cardinality concern.
Based on this online calculation tool by robustperception, for 100 posts, I should expect memory consumption at ~29 MiB. With my VPS constraints, this is ~3% RAM.
Aside from memory, it also takes storage to store all the scraped time series. With my frequency of 15s (5,760 samples/day), a 1.7 byte-per-sample estimate, and 24,010 time series, I expect storage consumption of 65 MiB/day, making it <500 MiB/week.
I have a 5 GB, 7 day retention policy set for Postgres, so it should never go above 5 GB.
Post-Mortem Thoughts: I’ll look into the labels to get a more accurate sense of the calculations. Currently napkin-ed it, but querying prometheus should give me more accurate values to use for calculations. In the future, if this grows too much, I’ll look into alternatives, or even dropping some metrics (i.e. referrer, bounding status code to 2XX,3XX,etc.)
API
I created two sets of APIs: public and admin using go-chi/chi. It was fairly simple to set up routing and adding middleware (i.e. authentication).
The public APIs hold unprotected REST API calls for content shown on the website, which is essentially everything that is cache. There is also metrics functions for page visits.
The admin APIs are backed by a middleware that requires an authenticated session using gorilla/sessions + logging into the admin user account to store relevant cookies. These APIs include admin APIs such as analytics and content creation interfaces for the database.
On start, we ensure the portfolio database has been migrated when starting and establish a connection using jackc/pgx. Afterward we begin all in-memory caches and register the routes to expose.
Frontend
The frontend is served by go using a-h/templ to generate and render the pages, which are returned through chi router handlers. It was my first time using templ, but it was pretty neat to use a combination of Go + HTML, especially when it came time to handle annoyances due to “CSRF tokens” and setting up dynamic script hashes to abide by the CSP policy I set in place.
There are also some React components loaded as “islands”. The purpose of this being so that the initial page skeleton can be returned fast, where the island would soon-after load dynamic content.
There is currently the public-facing pages, which anyone can visit, and an admin page gated by login.
Additionally, there is a dynamically generated /sitemap.xml for SEO purposes. When new posts are CRUDed, the sitemap is regenerated with up-to-date routing information in order to stay in sync.
I Don’t Know What CSP and CSRF Are…
Full-transparency: I haven’t delved into frontend security, such as Content Security Policy (CSP) and Cross-Site Request Forgery (CSRF), before I began this project. I initially set a policy through Caddy based on some boilerplate and didn’t think much of it. Everything worked fine on my machine because of hot-reload dev setups. But once I thought it was all set for a 1.0 release, I noticed styles and scripts were not loading.
After going through logs, I saw a CSP warning regarding styles that were not able to be loaded due to my configuration which only allowed for 'self' loading. I could have set unsafe-inline and other settings, but I wanted to keep things strict in the environment. I don’t want to risk anything, especially if I’m a noob at this stuff.
After going through some docs and figuring out methods to resolve these security issue (nonce, hash-%s), I moved from setting the CSP in my Caddyfile to creating a generative one that sets the CSP based on route.
User-facing Pages:
- Use static JavaScript scripts that are stored in a
const. These scripts are hashed, stored in avarand can be loaded + passed throughsha256-%sas part of the policy.
Admin-facing Pages:
- Same as above.
- Additionally have a CSRF token generation middleware. This is only required for admin pages which have forms that require a token to avoid CSRF attacks.
Supporting mermaid translation
Related to the above, in order to render mermaid in-browser, a JS script is necessary – and some custom markdown translation to add mermaid class to these code blocks. The issue is that (1) I can’t use node modules, and (2) using the mermaid.min.js script included inline code which violated the CSP policies. Oh no!
Luckily there is a workaround: using iframes for isolation! Whenever we need a mermaid diagram shown, we (1) expose a /static/mermaid-renderer.html page with lax CSP security, (2) load an iframe that loads said page, then (2) pass the diagram data to this query. This way our main context is still secure while the iframe handles the non-CSP-conforming script.
On a major negative side of this, each mermaid diagram requires its own iframe that loads a 2.6 MiB script. Browser HTTP caching prevents re-downloading for subsequent iframes on the same page, but each iframe still parses and executes the script independently in its isolated context.
Admin Happy Path
I wanted an easy way to handle logging into an admin page. What’s simpler than username and password? Aside from this, we need an initial admin registration flow - one that can’t be hijacked by someone else snooping having them register their own account before the real admin.
The admin flow is of three main steps steps…
- On initial compose, they should have set a secure
ADMIN_SETUP_TOKEN.- On initial login, this setup token is required.
- After the setup token is validated, an admin can then set up a
usernameand securepassword. - Once registered, the admin will be given 6 recovery codes in case they forget their password.
- Afterward they are signed in to the admin dashboard.
Passwords and recovery codes are securely stored in the database using x/crypto/bcrypt and validated with CompareHashAndPassword().
Testing
The testing I did was limited to some simple go tests to ensure my in-memory caches worked as expected as well as local load testing using Grafana k6.
Below is the script I am currently using for minor load testing in my local environment.
import http from 'k6/http';
export let options = {
scenarios: {
constant_load: {
executor: 'constant-arrival-rate',
rate: 1000, // 1000 req/s, I don't EVER expect anywhere near this to be honest...
timeUnit: '1s',
duration: '60m', // 60 minutes of heavy load
preAllocatedVUs: 10,
maxVUs: 500,
},
},
};
const BASE_URL = 'http://localhost';
const paths = [
{ path: '/', weight: 0.2 },
{ path: '/blog', weight: 0.3 },
{ path: '/blog/ff15', weight: 0.05 },
{ path: '/blog/fortenforten', weight: 0.05 },
{ path: '/blog/13teen', weight: 0.05 },
{ path: '/blog/tw12', weight: 0.05 },
{ path: '/blog/11eleven', weight: 0.05 },
{ path: '/blog/ten', weight: 0.05 },
{ path: '/blog/nine', weight: 0.05 },
{ path: '/blog/ate8', weight: 0.05 },
{ path: '/blog/seven', weight: 0.05 },
{ path: '/blog/six-6', weight: 0.05 },
{ path: '/blog/test-5', weight: 0.05 },
{ path: '/blog/test-post-4', weight: 0.05 },
{ path: '/random', weight: 0.3 }, // special: generates random non-existent blog
]
function generateRandomPostSlug() {
const adj = ['test', 'sample', 'random', 'dummy', 'example', 'complete', 'advanced', 'beginner', 'expert', 'intermediate', 'cool', 'awesome', 'fantastic', 'incredible', 'magnificent', 'spectacular', 'wonderful', 'amazing', 'brilliant', 'exceptional', 'extraordinary', 'superb', 'terrific', 'wonderful', 'amazing', 'brilliant', 'exceptional', 'extraordinary', 'superb', 'terrific'];
const noun = ['post', 'article', 'blog', 'page', 'entry', 'item', 'tutorial', 'guide', 'overview', 'walkthrough', 'deep-dive', 'masterclass', 'blueprint'];
const tie = ['on', 'about', 'with', 'using', 'for'];
const topics = ['js', 'react', 'go', 'python', 'java', 'c++', 'c#', 'php', 'ruby', 'swift', 'kotlin', 'rust', 'scala', 'haskell', 'erlang', 'elixir', 'ocaml', 'clojure', 'scheme', 'prolog', 'lisp', 'javascript', 'react', 'go', 'python', 'java', 'c++', 'c#', 'php', 'ruby', 'swift', 'kotlin', 'rust', 'scala', 'haskell', 'erlang', 'elixir', 'ocaml', 'clojure', 'scheme', 'prolog', 'lisp'];
const adjIndex = Math.floor(Math.random() * adj.length);
const nounIndex = Math.floor(Math.random() * noun.length);
const tieIndex = Math.floor(Math.random() * tie.length);
const topicIndex = Math.floor(Math.random() * topics.length);
const num = Math.floor(Math.random() * 5) + 1;
const initPath = ['blog', '']
const initPathIndex = Math.floor(Math.random() * initPath.length);
return `/${initPath[initPathIndex]}/${adj[adjIndex]}-${noun[nounIndex]}-${tie[tieIndex]}-${topics[topicIndex]}-${num}`;
}
// Create a weighted path selector
function getRandomPath() {
const random = Math.random();
let cumulativeWeight = 0;
for (let i = 0; i < paths.length; i++) {
cumulativeWeight += paths[i].weight;
if (random <= cumulativeWeight) {
return paths[i].path;
}
}
return paths[0].path; // Fallback
}
export default function() {
let path = getRandomPath();
if (path === '/random') {
path = generateRandomPostSlug();
}
const url = `${BASE_URL}${path}`;
const params = {
headers: {
// 'User-Agent': "Googlebot/2.1 (+http://www.google.com/bot.html)"
}
}
http.get(url, params);
}
Current Performance (as of Dec. 29, 2025)
Response Time Distribution
<50ms: 98.7%<100ms: 0.6%<250ms: 0.5%<500ms: 0.2%500+ms: 0.0%
Peak CPU Usage Peak (Per-Container)
container_cpu_usage_seconds_total
- Caddy ->
0.798 CPU Cores - API ->
0.535 CPU Cores - PostgreSQL ->
0.182 CPU Cores - Cadvisor ->
0.067 CPU Cores - Node Exporter ->
0.035 CPU Cores - Prometheus ->
0.019 CPU Cores
1.636 CPU Cores / 2 -> 81% Usage. For the “heavy” traffic I put it through, I think this is reasonable.
Peak Memory Usage Peak (Per-Container)
container_memory_working_set_bytes
- API -> 208 MB
- Prometheus -> 182.1 MB
- Caddy -> 93.1 MB
- Cadvisor -> 52.7 MB
- Postgres -> 40 MB
- Node Exporter -> 18.3 MB
594.2 MB / 1 GB -> 60% Usage. This one is a bit concerning. I see the MB usage rise to about this state then do a drop of ~30%. I will need to watch out for this as days progress to ensure I (1) don’t have a memory leak, and (2) don’t get OOM’d.
Current Limitations
- There is no automatic way to post images on blog posts (i.e. dragging an image onto my post editor, and having the backend upload to some storage and get a ref).
- Single deployment, single source-of-truth.
- This was purposeful so I can cheap out on hosting (i.e. not paying for some managed Kubernetes control plane plus the cost of the server.)
- Unique tags are not cached, which could lead to performance issues as more throughput is handled when listing blog posts.
Notes
There are still some more TODOs before I feel more assured with the website:
- [ ] Cache unique
tagslist. - [ ] Support uploading images for blog posts.
- [ ] Update this blog with images and diagrams of the architecture.
- [ ] Update blog with GitOps setup for release cycles
- [ ] Update blog’s formatting
- [ ] p9x tracking