Now that my workflow is fully integrated with artificial intelligence, I am never-ever working on only a single thing at a time. I’m usually not even working for a single client at a time. Because of this, I have restructured my working style around the New World Order. So until SkyNet comes and ruins the party, I need a way to keep track of all the things I care about.
One aspect I did not include in My Personal Ecosystem is the way I use AWS Lambda to perform certain tasks using serverless code. I will start off by saying that I do not think serverless code is a good idea as a general purpose solution. It has been my experience that such solutions tend to be more expensive than running a pod in a cluster I already have in place. However, it is sometimes useful for certain specific cases. I have the following Lambdas in my setup:
I want to document for myself and for anyone who might be interested (probably mostly me, but maybe a prospective employer will find this interesting) the current state of my personal ecosystem of software, devops, and what I consider state of the art within my personal projects. This article is primarily concerned with ops, but will touch on aspects of software development too.
I will have to divide this up into major bits. I build a lot of stuff for myself, just in dribs and drabs of spare time I steal from when I probably ought to be sleeping or playing video games. So, this is a list of what I have going and how I make it go. Let’s start by listing the products themselves. Then, I’ll describe the overall architecture of my system. Finally, I’ll break down how each project fits into that architecture.
Someday I am going to finally get around to describing how I’ve build the ticketing system I have built for our local Bethlehem Revisited project, which I call Bert or (for the most recent implementation) Gobert. However, in lieu of a full post on that, I’m going to just talk about one aspect of what I did to complete my work porting the system to Golang. One of the problems I ran into this time around was a maintenance issue with the website, which is quite a simple Vue.js-based single page application. When I performed this rewrite, I decided to use tooling that would allow me to dogfood the tools of my employer of the time. These tools, ultimately, ended up being far too unwieldy for my small project, but along the way I set up a rather nice gRPC to REST API gateway, and I’d really like to talk about that.
I am struggling to come up with a witty introduction to this topic, so let me just start by telling the origin story. Several years ago, I added some tooling to help me manage secrets related to my dotfiles. The first incarnation was a simple .netrc type solution where I stored the secrets in plain text in files that only my user could read. I added some scripts to help reading and managing these a bit. Over time this evolved into a complex system that allowed me to pull down secrets from my online password vault and store them in a local encrypted Keepass database. This created a new problem: every time a script used a secret, I needed to enter a password. Eventually, I added a small password service to help keep track of the key used to unlock the various password stores so I wouldn’t have to type the password every time.1
Coming from the world of Perl, one thing I find to be a weakness of most languages is their lack of CPAN. Having a comprehensive archive of all the code that is shared and mattered is amazing. You can cache locally. You can carry it with you on a laptop (at least, if you just want the most recent versions of everything). You can grep it. You can run a local documentation repository easily. And then there are tools like MetaCPAN (or search.cpan.org before it) that don’t just give you the docs, but actually help you find code that someone has written to solve the problem you want to solve. (Golang’s pkg.go.dev, by comparison, is awful if you want to do a plain text search for a library that solves some arbitrary problem.)
This is just to settle a dispute that mostly exists in my own head. It likely matters very, very little. However, it matters enough. I am currently interviewing for new jobs and during one interview, I was asked to design an algorithm for which I needed a set data type. Golang provides no built-in set data type, but it does provide a map, so I defined my set as follows:
Almost exactly two years ago, I began a little side project to create an email parser. I wanted to port an email sorting system I had previously written in Perl to Go. There was no particular reason to have my own email sorting system or to port it to Go except that I could and because I wanted to learn more practical skills in Go. However, in the process I was unable to find an email message parser which was able to meet all of my requirements. Those requirements included the following:
The headline is a little bit click-bait-y, but I want to highlight one of the easily overlooked risks of using slices in Golang. First of all, there’s nothing really wrong with the way slices work. A trade-off was made. It is not necessarily the same trade-off I would make, but such is engineering.
First, let’s map what a slice is because it might not be obvious unless you’ve
been writing Golang programs for a while. Let’s start by thinking about an
array. In Golang, an array is a fixed string of elements of a given type. For
example, we might define an array of int32 with ten elements as follows:
Updated 2023-01-11: This update is looooooong overdue. I had it dated as of May of last year and then never published it. My mistake. However, thanks to a reader, c0nscience, I am coming back around and geetting this correction made. He also suggested another implementation for reference by Jeff Linse on Github. After writing this post, I started working on some other projects and also started on another branch. Apparently, my testing of this change was very inadequate. I discovered, I’d actually been using the wrong method to perform the box sealing. I could make the Github change, but Github could not decrypt it. I have corrected the code below.
