Notes

/Berlin Go Homelab Kubernetes Everything

Bookmarks (issue 4)

Go 1.19beta1. As usual, lots of good improvements in the language’s runtime and the compiler, with one particularly interesting addition being the new “knob” runtime/debug.SetMemoryLimit.

How to use gender-neutral language at work and in life (Grammarly). “Luckily, the English language is relatively gender-neutral in many respects” [at least, when compared to Russian and German languages].

Meet passkeys (Apple) and Everything you want to know about WebAuthn (OktaDev). As you can guess, I’m very excited with Apple stepping onto the path to a passwordless future, while betting on WebAuthn standard.

Replace CAPTCHAs with Private Access Tokens (Apple), Private Access Tokens: stepping into the privacy-respecting, CAPTCHA-less future we were promised (Fastly), Private Access Tokens: eliminating CAPTCHAs on iPhones and Macs with open standards (Cloudflare).

Bookmarks (issue 3)

Rust tracks on Exercism. More than 100 coding exercises to learn Rust through practice.

PostgreSQL Anonymizer 1.0. A PostgreSQL extension for declarative data masking (docs and examples).

Queries in PostgreSQL: 4. Index scan (Postgres Pro). An in-depth overview of how PostgreSQL decides if it will use an index. One particular thing I had no idea about before I read the article was that “The Index Scan cost is highly dependent on the correlation between the physical order of the tuples on disk and the order in which the access method returns the IDs”. That explains several cases from my own experience, where postgres kept using “unexpected” sequential scans, after we added “another index” to the database.

Monarch: Google’s planet-scale in-memory time series database (Micah Lerner). A review of the paper (PDF), which describes the latest iteration of Google’s in-house metrics system.

Google’s API Improvement Proposals (AIP). A collection of design documents that summarize Google’s API design decisions.

A real life use-case for generics in Go: API for client-side pagination

Let’s say we have a RESTful API for a general ledger, with the endpoints, that return a paginated collection of resources:

  1. GET /accounts, retrieves a list of accounts, filtered and sorted by some query parameters;
  2. GET /accounts/:uuid/transactions, retrieves a list of transactions for account;
  3. GET /postings, retrieves a list of postings stored in the ledger.

No, this one isn't about data-structures...

Go

Bookmarks (issue 2)

Digital object identifier (DOI) (Wikipedia). A DOI is a persistent identifier or handle used to identify various objects uniquely. It aims to be “resolvable”, usually to some form of access to the information object to which the DOI refers. This is achieved by binding the DOI to metadata about the object, such as a URL. Thus, by being actionable and interoperable, a DOI differs from identifiers such as ISBNs, which aim only to identify their referents uniquely.

Operations principles: securely deploying the graph to production at scale (Principled GraphQL). Lots of things listed there apply to any sort of APIs — not only to GraphQL.

Songs your English teacher will NEVER teach! (Learn English with Papa Teach Me). Vocabulary from “Savage”, “WAP”, and “34+35”. This video is definitely not for kids!

8 phrases to spring-clean from your emails (Grammarly).

Platforms and Power (Acquired). “7 Powers” author Hamilton Helmer and Chenyi Shi (Strategy Capital), joined Acquired Podcast to discuss platform businesses, and how the “Power” framework applies to them.

Halfthings (Mat Ryer). Building something for the users to play with, to touch, to feel, to break, makes all the difference and moves the conversations away from the meta. Doing “one thing” or “building an MVP” can easily pull you into a “too much” for a validation phase. Build a “halfthing” instead.

Bookmarks (issue 1)

Generics can make your Go code slower (PlanetScale):

  1. boxing vs monomorphization vs partial monomorphization (“GCShape stenciling with Dictionaries”)
  2. interface inlining doesn’t work well with the 1.18’s compiler
  3. generics work well for byte sequences (string | []byte)
  4. in simple cases, generics can be useful for function callbacks.

How Meta enables de-identified authentication at scale. The rational, the use-cases, and a high-level architecture of Meta’s Anonymous Credential Service (ACS).

Hidden dangers of duplicate key violations in PostgreSQL (AWS). INSERT … ON CONFLICT has additional benefits, if compared to relying on PostgreSQL’s “duplicate key violation” error:

  1. no additional space needed for dead tuples
  2. less autovacuum required
  3. transaction IDs aren’t used for nothing, preventing (postponing) the potential trx-id wraparound.

Diving into AWS IAM Roles for (Kubernetes) Service Accounts (IRSA).

Go talks I keep coming back to

I have a personal list of “top conference talks” that I keep referring back to, even after years of working with Go:

Keeping the list here, in public, should help my future self, in a situation where I’m stuck with a mind-blocker, and I need to quickly pull out a piece of community wisdom from the backyards of my memory. The list isn’t meant to be complete, and I expect to add more links here, moving forward.

Did I miss any? Share your suggestions with me on Twitter.

Love. Hate. Material Design

If Chrome had a notch

Chrome notch

Another witty picture under the cut…

I've got COVID. What do I do next?

Go to https://www.berlin.de/corona/massnahmen/abstands-und-hygieneregeln/ for the information (in German) about the latest regulations in Berlin. For the up-to-date information in your region, consult with your local authorities.

Everything I list below are the steps I found relevant, after I’ve self-tested positive for COVID in Berlin, in the end of January 2022.

Friday, 28th January 2022

I woke up with some mild symptoms of cold. On the day before I worked from home, and only had a usual an hour-long walk around Prenzlauer Berg — Mitte after the workday. The rapid antigen test (Schnelltest) was negative.

While still working from home, I took a “quiet Friday” at work, to simply focused on some mundane routines.

I went for a short walk in the afternoon, bought some grocery, and headed home.

Saturday, 29th January

Didn’t feel anywhere better or worse: mostly had a runny nose and a dry throat. I didn’t want to wait in the line for a quick-test, so I’ve just walked around the city for an hour and went home.

Sunday, 30th January

It felt the same as it’d been the day before, although the dry throat started to feel a bit more intense. Walked around for an hour, came home, made some coffee ;)

A couple hours later I started to cough. I made a self-test — we still had some at home — and,

Here we are! Welcome to the “two-stripes” club.

Keep reading…

Error messages in Go

When Go code propagates an error, the following pattern is very popular:

fmt.Errorf("failed to find a parking slot: %w", err)
// Or
fmt.Errorf("could not call mom: %w", err)

These “could not”, “failed to”, “unable to” make sense when my mind is in the local context of the function, method or package. But, in most of the cases I have to deal with, it makes the resulting log message overloaded with informational garbage:

unable to ask about the cat: failed to call mom: failed to do request: Get https://: context canceled

While discussing this issue with a colleague, we came up with the following “better” strategy:

  1. only the logger should express its attitude to the facts, using words “error”, “failed”, etc
  2. the business code must operate only with facts, e.g. “call mom”.
err := CallMom(number)
if err != nil {
    return fmt.Errorf("call mom (tel %s): %w", number, err)
}

This renders as following to the application logs:

error: ask about cat: call mom (tel 123): make request: Get https://: context canceled

For a long stack of errors, this makes the full error message more dense, showing more useful information per line.

Go

Good coffee places in Berlin

My definition of “good coffee places” includes, although, by no mean limited by, offering a “not too fruity” filter coffee or a decent americano.

Keep reading…

Making sense of requests for CPU resources in Kubernetes

Kubernetes allows a container to request several resource types:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
  - name: my-app
    image: images.example/my-app
    resources:
      requests:
        cpu: "100m"
        memory: "64Mi"
      limits:
        cpu: "500m"
        memory: "128Mi"

One particularly confusing type of the resource for me was cpu. For example, in the manifest above, the my-app container declares a request for “100m” of the CPU. What does that mean?

Keep reading…

One step closer to "Tabless" workflow

Many years ago I embraced “tabless” development workflow: I use buffers, when I’m in Vim; I also switch tabs off in both Goland and VS Code, as the first thing after I install the IDEs to a new laptop.

I’m trying the same with Firefox web-browser now:

Tabless Firefox 91
  1. Enable toolkit.legacyUserProfileCustomizations.stylesheets switch in Firefox’s config (via about:config page or inside user.js).

  2. Place the styles below into %PROFILE%/chrome/userChrome.css (I pick the actual path to the profile directory from Firefox’s about:support page).

// Hide the tabs.
// Beware that hidding the tabs with "display: none" will ruine you browser's recent history,
// I've learned that in a hard way ;)
#TabsToolbar > .toolbar-items {
    opacity: 0;
    pointer-events: none;
}

// Pull the navigation bar up, on top of the empty space, that left after we'd hidden the tabs.
#nav-bar {
    margin-top: calc((7px + var(--tab-min-height)) * -1);
}

// On macOS, when not in full screen, shift the urlbar's panel to the right,
// after close-minimise-expand buttons.
:root:not([inFullscreen]) #nav-bar-customization-target {
    margin-left: 65px;
}

I looked at mozilla-central/··/navigator-toolbox.inc.xhtml to get the structure of Firefox’s UI.

  1. Pick some nice and clean theme. My kudos to Safari - MacOS Monterey Light by a person nicknamed notcat.

  2. (Optional) In Firefox’s “General” preferences, switch off “Ctrl+Tab circles through tabs”. With that, pressing Ctrl+Tab exposes all currently open pages (similar to how on macOS or other OS one switches between the opened applications with ⌘+Tab).


Overall I’m fairly happy with how it ended up. Although, some things aren’t quite ideal yet (might add more as I use this setup):

I wish there was a shortkey to enter a “modal mode”, where I could filter the list of open pages, to search for a particular page, and to switch to this page. Something similar to :ls command in Vim, or ⌘+E in Goland. I can use Firefox’s “Search amongst Tabs” for that (press ⌘+L to focus into the URL bar, and querying with “%[space]”) but that requires some getting used to.

Firefox has “Show all tabs” button (Ctrl+Shift+Tab) but the way it works, at least in Firefox 91, is very confusing and random. It seems to me, its behaviour is tightly coupled to the browser’s Tab UI.

Update (2022-12-16) I’ve found Panorama Tab Groups extension, which exposes the opened pages, and it works pretty well for me. One very minor thing is that I have to use Cmd+Shift+E because Cmd+E is locked in Firefox.

Self-signed certificates with k3s and cert-manager

At least for now, my homelab cluster (4x Raspberry Pi, k3s, etc) is available only for the devices on my local network, inside a custom DNS zone k8s.pi.home. I don’t think there are practical reasons to run anything with HTTPS in that setup, but there are cases, like browser extensions, where it’s required.

Turned out, in 2021, it’s fairly straight forward to set up a Certificate Authority (CA), that will issue TLS certificates to “secure” the ingress resources. At least, it’s way simpler comparing to how I remember it was back in the days. All thanks to cert-manager and some YAML.

First thing is to install cert-manager to the cluster. k3s comes with helm-controller, that gives us a way to manage helm charts with Custom Resource Definitions (CRD). The following manifest defines a new namespace, and a resource of a kind HelmChart, to install cert-manager inside this namespace:

apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager

---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: cert-manager
  namespace: kube-system
spec:
  chart: cert-manager
  repo: https://charts.jetstack.io
  targetNamespace: cert-manager
  valuesContent: |-
    installCRDs: true
    prometheus:
      enabled: true
      servicemonitor:
        enabled: true

After applying the manifest above — kubectl apply -f cert-manager.yml — define a self-signed certificate, which is used to bootstrap a CA:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-cluster-issuer
spec:
  selfSigned: {}

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-ca
spec:
  isCA: true
  commonName: selfsigned-ca
  secretName: selfsigned-ca-root-secret
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: selfsigned-cluster-issuer
    kind: ClusterIssuer
    group: cert-manager.io

---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-issuer
spec:
  ca:
    secretName: selfsigned-ca-root-secret

And now, I can use selfsigned-issuer to issue TLS certificates for the ingress resources (Traefik ingress in the k3s’s case). E.g. to play around, I run an open-source version of LanguageTool server. The ingress manifests for the server looks like as following:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: languagetool-server
  annotations:
    kubernetes.io/ingress.class: traefik
    cert-manager.io/issuer: selfsigned-issuer
spec:
  rules:
  - host: languagetool.k8s.pi.home
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: languagetool-server
            port:
              number: 8010
  tls:
  - hosts: [languagetool.k8s.pi.home]
    secretName: languagetool-server-cert

Of course, any certificate signed by my CA won’t be automatically trusted by anyone, including my own system. If I try to access https://languagetool.k8s.pi.home, any HTTP client will raise a “failed to verify the legitimacy of the server” issue. I don’t know if there is a better way to solve that, but I can hack that around by installing the cluster’s root CA certificate into the system’s keychain, and telling the system, that it should “trust” the certificate:

$ kubectl get secret/selfsigned-ca-root-secret -o json \
  | jq -r '.data["ca.crt"]' \
  | base64 -D > ~/tmp/selfsigned-root-ca.crt
$ open ~/tmp/selfsigned-root-ca.crt

Choose “Always trust” the certificate in the keychain’s certificate settings. The server looks legitimate now!

Wireless-to-Ethernet island for homelab cluster: IPv6, NDP proxy and mDNS reflector

Initially, when I assembled a homelab cluster of Raspberry Pis, everything was directly connected to my Wi-Fi router with the Ethernet cables. This worked fine but this “stack of boards” behind the sofa in the centre of our small flat bugged me a bit.

Last year I decided to reorganise the cluster, turning it into a wireless-to-wired island, which I could relocate anywhere within the flat, without doing any special cable management, while staying cheap and avoid stacking the appartment with even more gadgets. After going through a number of trials and errors, the final setup looks as the following:

Homelab cluster as a wireless-to-ethernet island (2021)

Different colours contour the connections between two logical subnets — more on that later. Here what we have on the schema (top to bottom):

Keep reading…

Development environment in 2020

Development environment in 2020

Last year I made this one, after involving myself into a very random discussion on Twitter, about the unnecessary complexity and none-sense of modern days’ development environments.