Building a Physics Engine for the DOM: Part 1


I got into programming because I wanted to build video games. I love telling stories, I'm an illustrator, an accomplished musician & composer, but the part of games that always eluded me was the programming. I bought myself a couple books on C and C++, started learning Python, downloaded Unity but found that it worked with JavaScript and C#. By the time I felt I had all I needed to start building the game, I was so overwhelmed by the things I _didn't_ know, that the things I _did_ know paled in comparison. I also tend to build up volcanoes of energy for an idea only to have them sputter to little more than a cough by the time I feel like I'm actually ready to start. It's all very discouraging, and given that in order to build a game I felt like I would have to learn the totality of Computer Science, Linear Algebra, and Differential Calculus, I quickly gave up on the idea.

But I never _truly_ gave up on it. The more research I did, the more people I found saying things like, "Just learn Unity or Unreal! There are TONS of tutorials. If you really want to build games, then you should start working with whatever's available to you!" First off - that feels super gatekeeper-y. "If you _really_ want to build games..." What if I want to build games in a particular way? What if I just like knowing how things work? What if I want to understand all of the underlying game systems? But, secondly, _did_ I really want to build a game? What if these people were right? What if I had all the skills (except programming) and all the tools necessary, but the fact that I wasn't using them meant I actually didn't want to do anything with them?

Fast forward seven years, and now I have the programming skills. I'm a Software Engineer at one of the biggest tech companies in my area, I'm a tech lead on one of its core products, and I can lead workshops on languages like Rust and Go (and JS and Python and C# and...). And yet, I still haven't built my game. And I haven't built it for one. major. reason.

I'm still not sure that building a game is what I want to do. After seven grueling years of self-education, five years of on-the-job blood (yes blood!), sweat, and tears, overhauling how I write and compose music, and taking professional illustration classes, I find that building a game, in and of itself, isn't _that_ big of a draw for me. However, I have identified what is a draw.

The physics engine.

One of my favorite parts of the movie [IndieGame](https://en.wikipedia.org/wiki/Indie_Game:_The_Movie) is when Tommy Refenes talks about making Super Meat Boy's controls _feel_ spectacular. The way the character accelerates and jumps and slides along walls and surfaces, creating this graceful language of movement shared between developer and player. It captured my imagination because it brought to light something I had never consciously acknowledged, but had always known: some games feel better than others, and there's a reason for it.

So this brings me to today. Of course I still want to build a game, but I don't really have any fleshed-out ideas for game mechanics or storylines I want to build (yet). In the meantime, however, I _can_ continue my self-education around the various systems at work in a game, and seeing as how the physics engine is the one that's captured my attention the most, that's where I'll start.

My idea is simple: build a small physics engine for DOM elements. And not even elements that live inside Canvas. Just plain old DOM elements. `<div>` elements mainly, but really anything that satisfies the box model. I've got a solid understanding of rendering performance, so I'd like to combine that with my study of the engine itself to make a library that allows you to: throw a `<div>`, make it bounce around the screen, collide with other physics-enabled elements. Things like that. There may already be libraries that do this, but I'm not really concerned about that. This is purely an educational exercise: one that's meant to be fun, silly, and maintainable.

Speaking of maintainable, here are my goals for the project: keep it small, written in TypeScript, unit-tested, and open source. In fact, here's the [repo](https://github.com/shiplet/coma-physics). It's still a baby, so don't expect anything major yet. It'll get there.

I'll also keep track of what I'm learning in this series of blog posts in the form of an interactive, phase-based website. You can follow along here: [https://comaphysics.com](https://comaphysics.com). Why's it called Coma Physics you ask? Because comets are super cool, and one of the core elements of the comet is this layer of gas that surrounds the actual chunk of space rock, and that layer of gas is called The Coma. Combine that with the fact that physics are a core element of many games? Coma Physics.

Alright! Let's go make some stuff fly around the screen!


A quick overview of Rust's unique features

Core Rust Features for Non-Rustaceans


This is a quick overview of some unique Rust features. If you're familiar with Javascript, I think of this list as being analogous to a list of JS things like hoisting, prototypes, and the event loop. It's a living document, so if sections are missing, incomplete, or empty, that will change in the future!

## Types
- Strict, static typing system
- dynamically sized types, like strings or vecs, expose a pointer to the data and its length. This allows Rust to standardize the size of such variables - all variables of a single type must occupy the same amount of space in memory. When the variable is a length value and a pointer to the address in memory where a string begins, then technically that variable is the size of two `usize` values (length, pointer), rather than the size of the string it's pointing to.
    - this behavior is common to all dynamically sized types
    - remember, all unsized or dynamic types MUST be behind some kind of pointer
- Tuple struct (or newtype?) initializers look like function calls, for example: `Millimeter(5)`
    - these initializers are actually implemeted as functions that return an instance of the type constructed from its arguments
    - this means the initializer functions can be used as function pointers that implement closure traits, allowing them to be used in function signatures that accept closures

## Associated Types
This example is borrowed from the Rust Reference, with commentary.

The reference definition of associated types uses extremly precise language, which can often sacrifice clarity for brevity. Why use 100 words when 1 previously defined piece of jargon or domain lingo can do the trick? Well, this is my attempt at unzipping that 1 previously defined piece of domain lingo into the 100 words that make most sense to me... at least at the time of writing.

Associated types are placeholder types defined on traits, which allow for generalization of types passed to trait implementations. In other words - as you're designing a trait that performs some general task, and you want to ensure that something implementing your trait maintains a consistent type over the course of a series of functions required by your trait, then you define an associated type, like `type E;` below. Because this is an abstract, unknown type, you're enabling a future program to implement this trait such that `E` could be a primitive like `i32` or `bool`, something more complex, perhaps a custom type `Daemon` or `Process`, or a generic like `T`.

See the example below for a minimal implementation.

```rust
trait Container {
    type E;
    fn empty() -> Self;
    fn insert(&mut self, elem: Self::E);
}

impl<T> Container for Vec<T> {
    type E = T;
    fn empty() -> Vec<T> { Vec::new() }
    fn insert(&mut self, x: T) { self.push(x); }
}
```

The `associated type` is defined on the trait itself as `type E;`. This tells the rust compiler that any type that implements `Container` must also implement a definition for `E`.

The trait implementation below provides the `associated type definition`. This is the part where the associated type receives its definition, in this case the generic type `T`. In this case, the placeholder has been filled by a generic rather than a concrete type.

## Ownership
- at any given point, a piece of data can have either, *but not both*, of the following: one mutable reference, or any number of immutable references
- references must always be valid

## Return Values
- To return a value out of a Rust function, you omit the `;` semicolon on the final line of the function
- If the final line of the function contains a semicolon, it implicitly returns nothing, and the return type is `()`, canonically known as the `unit type`. It has only one value, `()`.

## Smart Pointers
- Smart Pointers are pointers with metadata and methods
- They abstract a lot of the more granular aspects of memory managment and mutability into their internal structure and public API
- The most common smart pointers:
    - `String` - closely related to `str`, but `String` owns the contents of the string and provides an API for mutating its internal state
    - `Vec<T>` - a vector of a single type that can grow in size
    - `Box<T>` - enables heap allocation
    - `Rc<T>` - enables multiple ownership via reference counting in single-threaded applications
    - `Ref<T>` - enforces borrowing rules at runtime rather than at compile time, which enables interior mutability of immutable entities
- Smart Pointers own they data the point to, but can also allow it to be used as a reference
- Implement `Deref` and `Drop` traits
- `Deref`
    - allows the smart pointer to be used like a reference
- `Drop`
    - allows you to customize what happens when a smart pointer goes out of scope
- deref coercion
    - a convenience feature that allows coercion between references to types with analogous trait implementations
    - applies in 3 instances
        - `From &T to &U when T: Deref<Target=U>`
            - coerces a reference of type `T` to a reference of type `U` when type `T` implements `Deref` such that it targets (or `return`s) a value of type `U`
        - `From &mut T to &mut U when T: DerefMut<Target=U>`
            - same as above, but with mutability
        - `From &mut T to &U when T: Deref<Target=U>`
            - can only coerce a mutable type into an immutable type, because in going from immutable to mutable, Rust is unable to guarantee that the mutable reference is the *only* such reference to that chunk of data.
    - Example: `&String` and `&str` can be used interchangeably

## Traits
- Traits are encapsulated behavioral abstractions that can be shared across an arbitrary number of discrete types
    - which is a fancy way of saying that several types can have similar behaviors if they implement the signature required by the Trait
    - `Copy` is a Trait shared by primitive data types, like `i32` and `bool`, and it refers to the behavioral ability to share data values rather than data references or pointers
- Trait Objects
    - ```rust
        pub struct Screen {
            pub components: Vec<Box<dyn Draw>>,
        }
      ```
    - in order to use traits as trait objects, they MUST go behind a pointer, like `Box` or `Rc`/`Arc`. This is because traits are also dynamically sized.
    - `Box<dyn Draw>` is the Trait Object. It refers to any type inside a Box, as long as that type implements the `Draw` trait.
    - the `dyn` is the piece, or the Trait feature, that indicates this is a Trait Object and that there is an expected abstraction of behavior in the `Draw` trait, which every entity that encounters it must implement.
    - Rust deliberately avoids refering to things as "objects" the way that other languages do. Structs and Enums are the primary object-like entities, but they differ because they separate data and behavior, whereas object-oriented or class-based languages will combine data and behavior in the same object.
    - Trait Objects combine data and behavior, but you can't _add_ data to a trait object: they allow an abstraction in behavior across several types.

## Objects
- As indicated above, Rust deliberately avoids referring to things as objects, because it doesn't explicitly combine data and behavior the way an object oriented or class-based language might. Structs and Enums can have data-based definitions with `impl` blocks appended after-the-fact, but the explicit separation of data and behavior means they're not strictly "objects" in the OOP sense.
- structs
    - related data of distinct types
    - similar to JS notion of an object, but data and behavior are separated by a discrete `impl` block (referring to "implementation") where methods or associated functions get defined
    - methods: take `&self` or `&mut self` as their first parameter - conceptually similar to a class method
    - associated functions: conceptually similar to `static` functions on a class, as in they don't depend on the struct being initialized or having access to internal state
- enums
    - a single abstract concept and its enumerated variants
        - `IPv4` and `IPv6` are both enum variants of the parent type of `IP`, so even though they're different at the granular level: one is v4 and the other v6; they can both be used when data of type `IP` is required.

## Lifetimes
- forthcoming

## Explicit Reference Counting Types
- For data with multiple owners, Rust provides types that explicitly indicate multiple ownership
    - `Rc` and `Arc`, or `reference counted` and `atomic reference counted` types, respectively
        - `Rc` is for use in single-threaded applications
        - `Arc` is for use in multi-threaded applications
- These types conventionally use `Rc::clone` or `Arc::clone` when "cloning" references to the data, because their associated functions merely increment the reference count, rather than deep cloning the data.
- This is a helpful convention when cloning other types via the deep clone method, or `pieceOfData.clone()`.

## Interior Mutability
- encapsulates unsafe code used to mutate owned data 

------


## Rust's unsafe features are meant to empower and enable...
Working with Rust means the compiler is inherently conservative. It'll forbid some valid programs if it means it never allows an invalid program. But, there are some things we, as programmers, can know that the compiler can't know. For example - taking mutable references from two distinct halves of the same vector. Rust doesn't _know_ they're distinct halves, it just knows you're trying to take two mutable references, which breaks one the borrow checker's rules. Fortunately, Rust has an alter-ego - *UNSAFE RUST*. Unsafe Rust lets you do the following:

- Dereference raw pointers
- Call unsafe functions or methods
- Access or modifying static mutable variables
- Implement unsafe traits
- Access fields on unions

## Rust's safety features are meant to prevent or impede...
 Rust's guarantees around safety exist in order to mitigate the effects of Undefined Behavior. Think of the MISSINGNO Pokemon - an excellent example from 2020's RustConf - which is the result of a set of deterministic undefined behaviors. Rust wants to avoid things like that. Here's how it does it. (This is drawn / copied from the [Rustnomicon](https://doc.rust-lang.org/nomicon/what-unsafe-does.html)).

 Rust's syntax and quirks are designed to prevent the following:
 - dereferencing dangling or unaligned pointers
    - a pointer/reference is dangling if it's null or if some of its bytes are part of a different allocation
 - breaking pointer aliasing rules
    - a pointer cannot outlive its reference
    - a mutable reference cannot be aliased
 - calling a function with the wrong call application binary interface (ABI) or unwinding from a function with the wrong unwind ABI
 - causing a data race
    - two or more threads concurrently accessing a location of memory
    - one or more of them is a write
    - one or more of them is unsynchronized
 - executing code compiled with target features that the current thread context does not support
    - an attribute applied to an unsafe function, which refers to a feature present in the system's chip architecture
 - producing invalid values
    - a bool that isn't 1 or 0
    - an enum with an invalid discriminant
    - a null fn pointer
    - a char outside ranges `[0x0, 0xD7FF]` and `[0x0, 0x10FFFF]`
    - a ! (never type?)
    - an integer, floating point value, or raw pointer read from uninitialized memory, or uninitialized memory in a str
        - uninitialized memory being basically a jangled mess of bits that may or may not refer to or form a valid piece of data
    - a reference/Box that is dangling, unaligned, or points to an invalid value
    - a wide reference, Box, or raw pointer with invalid metadata:
        - a `dyn Trait` that's not a pointer to a matching vtable (the table of functions for the trait) for the dynamic trait `Trait`
        - slice length has to be a valid usize, or else the metadata is invalid
    - a type with custom invalid values that is one of those values
        - for example, a `NonNull` that is `null`






CS Fundamentals

Exploring CS Fundamentals via Stacks and how they fit into the world we code.

Stacks


_Hi! So, every once in a while, even pros need to get back to the basics. 
And seeing as how I've been in the industry since 2014 and my only formal education was a 12-week part-time bootcamp at the beginning of 2015, getting back to the basics for me is, in many ways, an initial exploration of the basics.
I'm sure I'm not the only person in this position, so I'm gonna try out the tried and true method of learning by teaching._

_Today, I'm going over:_

## `Stacks`

Stacks are a data structure. 
What's a data structure? 
A data structure is a way of organizing data and logic which optimizes for certain use cases and/or real-world scenarios that need to read and write to memory efficiently & predictably.

So stacks are data structures, but what exactly does that mean, and how does it fit into the world as we know it?

### Undo/Redo

A perfect example of using a stack in daily life is when you hit `Ctrl-z` or `Cmd-z` to undo the last operation you performed.
Take for example the act of typing each letter of your name into an input box while signing up for an account on a website.
Each key you press generates an event that's stored by the browser, and the browser stores those events in - you guessed it - a stack!

![A stack represented by four boxes, with the letters "p", "i", "h", and "s"](/static/stack_entry.png)

Each letter-in-a-box above represents one of those events, and they're entered into the stack in the order they occur.
When you hit your "undo" shortcut, the browser goes to the stack of events, removes the top, most recent event, and reverses the operation it performed.
This act of taking the top, most recent item off of the stack is known as a `pop`. Conversely, adding an item to the end of a stack is known as a `push`.

### Why Stacks?

So why use a stack here instead of, say, an array? 
A lot of it comes down to the understanding that stacks _only work in this way_. 
You don't index into a stack the way you do an array.
You don't modify stacks at arbitrary locations like you might an array or an object/hashmap.
You've really only got 2 or 3 operations you can perform with a stack: `pop`, `push`, which we've already covered, and `peek`, which is like `pop`, but it doesn't alter the contents of the stack; it just returns the value of the topmost item.
They also frequently have helper methods that let you know the `length` of the stack, and whether or not it's empty.

In short, stacks are great at being a repository of high-performance, low-maintenance breadcrumbs: a way for your algorithm or program to retrace its steps to a prior state in memory. 
That's why they're used for `undo` operations, but they're commonly used in compilers while reading syntax tokens, as well as pathfinding algorithms in order to track decisions.
You'll also encounter one of the most common use cases while debugging an application: the `call stack`.
Each item of a call stack represents a function call and the program context under which it was called.
This allows each layer the program, procedure, or subroutine to quickly resume the proper memory context once the function below it returns or finishes.

### How am I Supposed to Know These Things Unless I'm Taught Them?

This is a question I ask myself every time I encounter a new concept in Computer Science.
"How does everybody just _know_ these things? Is it obvious to everyone but me?"
You'll be glad to know that no! it's not obvious to everybody but you!
Stacks didn't actually, formally exist as a concept until 1955, when their initial specification was published in a [German Computer Science journal](https://dl.gi.de/bitstream/handle/20.500.12116/4381/lni-t-7.pdf?sequence=1&isAllowed=y), and the first patent for a stack wasn't filed until 1957.
Of course, Alan Turing hinted at their existence as far back as 1946, but it still took 9 years after that for them to show up as a standard.

I take comfort in things like this because it means that someone at some point in history spent a lot of time trying to solve a very particular set of problems, and _this thing_, which in programming today seems so common, was what they came up with.
It just so happens that it was a good idea and stuck around!

### Okay, So How Do I Build One?

In my experience, programming languages don't typically include a default implementation of `stacks` in their prelude or standard library.
The justifications I've seen given for why is that stacks, while useful, aren't actually used in most day-to-day programming problems. 
And, if a stack is needed, it's usually a straightforward process to modify the language's support for `arrays` or `vectors` to create one.
So let's do that!
Let's build a stack in `JavaScript` and then for fun compare it to one built in `Rust`.
This comparison will also highlight a very important distiction with stacks - _their values should always be of the same type_.

Remember, the basic building blocks of a stack are 3, sometimes 4 methods, and 1 property:
- `push`: (method) adds an item to the end of the stack
- `pop`: (method) returns and removes the last item of the stack
- `peek`: (method) returns the last item of the stack without removing it
- `isEmpty`: (method) returns a boolean indicating if the stack is empty
- `length`: (property) returns the length of the stack

Given those parameters, here's the simplest definition of a JavaScript stack I can think of.
Notice how it's wrapping JS's standard implementation of an array.
The array forms the backbone of the stack, but as long as you're using the `Stack` class, the only values you're concerned about are the topmost value, the length of the stack, and whether it's empty.
There's no `splice`, or `slice`,  or accessing `stackItem[5]` - there's only the methods and property defined below.

```javascript
class Stack {
	constructor() {
		this.stack = []; // The standard JS array ✌
	}

	get length() {
		return this.stack.length;
	}

	// pushes the value onto the end of the stack
	// mutates the stack
	push(value) {
		this.stack.push(value);
	}

	// removes the topmost item from the stack
	// mutates the stack
	pop() {
		return this.stack.pop();
	}

	// returns the value of the topmost item
	// but does not mutate the stack
	peek() {
		return this.stack[this.length - 1];
	}

	isEmpty() {
		return this.length === 0;
	}
}
```

Not too shabby huh? If you Google "javascript stack", this is probably line-for line an exact copy of what you'll find out on the web.
Of course, there are some additional guardrails you can add and questions you can ask, such as:
- What happens if you call `pop` on an empty stack?
- How _big_ should you allow the stack to get?
- How would you go about writing a `flush` function that empties the entire stack?

---

Now let's take a look at one written in Rust.

It has a lot of similarities to JavaScript, but some key differences as well.
The first being that Rust doesn't really have a concept of a `Class` the same way more recent versions of JavaScript do.
Rust has `Structs` and `Enums`, which can have object-like characteristics, but are their own entities.
For a more in-depth discussion of these differences, see my post [Core Rust Features for Non-Rustaceans](https://shiplet.dev/post/core-rust-features-for-non-rustaceans).

The other thing to notice here is that, because Rust is statically typed, and in order to make our `Stack` usable for a wider range of data types, it contains some `generic` parameters.
Generic parameters signal to the Rust compiler that you anticipate this `struct` being used for a wide variety of data types.
I'll call these out in code comments below.

However, generics also enforce, at the compiler level, the rule I stated before we dove into the JS solution: _a stack's values should all be of the same type_.
You can get similar functionality with TypeScript, but because it ultimately transpiles down to JavaScript, I wanted to include an example where the language/compiler will refuse to build if there's any chance of violating the type safety (unless you explicitly account for it).

```rust
// src/main.rs

// the `<T>` here is the generic.	
// it creates a marker that we use in later method definitions
// to indicate that every operation should expect the same
// type of data
struct Stack<T> {
	stack: Vec<T>
}                

impl<T> Stack<T> {
	// this creates a new Stack instance.
	// you might notice the vec![] doesn't have a specific type annotation.
	// this is because Rust is smart enough to infer the type after the first
	// time you use it.
	pub fn new() -> Stack<T> {
		Stack {
			stack: vec![]
		}
	}

	// because `push` mutates the stack, we have to explicitly borrow `self`
	// as mutable
	pub fn push(&mut self, value: T) {
		&self.stack.push(value);
	}

	// returning a type of Option<T> signals we should
	// be prepared to handle cases where there's no data left	
	pub fn pop(&mut self) -> Option<T> { 
		self.stack.pop()
	}

	pub fn peek(&self) -> Option<&T> {
		self.stack.get(self.stack.len() - 1)
	}

	pub fn len(&self) -> usize {
		self.stack.len()
	}

	pub fn is_empty(&self) -> bool {
		self.stack.len() == 0
	}
}
```

Some clarifications: the `impl` block is Rust's way of saying, "implement these methods for this data type."
So in the case of our `Stack` struct, we have a `stack` property, which is of a type `Vec<T>` or a "vector of data of type `T`, whatever `T` might be."
That `T` could refer to integers, strings, or other more complex data types.

Notice also the difference between the return types of `pop` and `peek`.
`pop` returns an `Option<T>`, but `peek` returns `Option<&T>`.
The `&` in `Option<&T>` indicates we're returning a _reference_ to the data at that location, rather than the data itself.
Whereas in `pop`, because it's `Option<T>`, we know that we're receiving the data itself, and that the data will no longer belong to the stack.
This is how Rust handles non-destructive vector indexing in this case, and I dig how explicit it is!

### Stacks!

I hope this answers some questions about stacks!
I definitely learned some new things and am feeling a bit more confident after writing it, so I can only hope that some of that transfers to you as well.

Up next are Queues! Stay tuned :) 

Serverless in Go

An example of how to get an object's ref with fauna & go

How to Get a Fauna Value's Ref in Go


The [FaunaDB docs](https://docs.fauna.com/fauna/current/) are pretty thorough, but I've noticed that every example of querying an object by `ref` assumes you already have the ref ID handy.

Take, for instance, this [Update](https://docs.fauna.com/fauna/current/api/fql/functions/update?lang=go) example in Go:

```go
result, _ := client.Query(
  f.Update(
      f.RefCollection(f.Collection("spells"), "181388642581742080"),
      f.Obj{
          "data": f.Obj{"name": "Mountain's Thunder", "cost": nil}}))

if (err != nil) {
  fmt.Println(err)
} else {
  fmt.Println(result)
}
```

The ref ID `181388642581742080` is hard-coded! But what if you don't just have a handy map of all the refs in your collection??

Don't worry, I gotchu. Go's typing system makes the process a little additionally verbose, but it's reliable.

```go
import (
    f "github.com/fauna/faunadb-go/v3/faunadb"
)

// setup code

client := f.NewFaunaClient("YOUR_CLIENT_KEY")

ref, err := client.Query(f.Get(f.MatchTerm(f.Index("ANY_INDEX_YOURE_USING"), theSearchValue)))
if err != nil {
    // handle error
}

var refAssert *f.RefV
if err = ref.At(f.ObjKey("ref")).Get(&refAssert); err != nil {
    // handle error
}

// the rest
```

The key piece is running `ref.At().Get(&refAssert)`, where `refAssert` is of type `*f.RefV`. This accurately maps to the response value you get from your query and should allow you to use `refAssert.ID` for any future queries.

Enjoy!

A retrospective on sustained unsustainable effort

On Pushing Too Far


As much as I may like to think so, I don't have an infinite supply of mental energy or focus. Whether it's ADHD, some other undiagnosed neurological condition, or just the basic limitations of my extremely human abilities, for any given task requiring a moderate amount of mental focus and acuity, I will inevitably reach a point where:

1. I'm near enough to completion of the task that I feel compelled to finish the task
2. My ability to finish the task rapidly degrades with each additional attempt to do so
3. My ability to complete, or even approach, any non-related task also rapidly degrades, often to the point that I'll reach a momentary period (5-50+ minutes) of something like a total system shutdown

For example: today I spent nearly 8 hours reading technical specifications for a new programming language and in that time managed to get through 75% of the total volume of documentation available. For the duration of that time I was sharp, absorbent, and able to rapidly incorporate a significant amount of the available information into my existing mental taxonomy of programming languages. Given that I'd already gotten through 18 of 24 chapters in the book, getting through the remaining 6 chapters felt imminently possible.

However, irrespective of any increase or decrease in the complexity of the information, I reached a point where my brain-sponge had saturated. Earlier in the day I could read a sentence like, "This is extremely important for these reasons: ..." and not only would I recognize both the importance and application of the very important information, I'd actually anticipate what was coming next: "Oh, given this, these issues or use-cases will become important." Having those kinds of brainwave moments is addictive - it's like the rapid progression model most video games employ: it sets up mechanics and challenges and *quickly* rewards you for anticipating and correctly applying your prior learnings.

But once I'd reached saturation, I could legitimately read the sentence "It's important to breathe because your body runs on oxygen" and all I'd remember is that there was something like... existentially important in Chapter 19. Like. So important that if I didn't know it and were talking to somebody very knowledgeable on the subject of being alive, and I didn't know this information, they'd instantly mark me as a doofus, a charlatan, a pretender, a poser, a fakely alive person.

The next problem arises *because* I've recognized this. I don't want to seem like a fakely alive person, so I panic and re-read the oxygen sentence 35 times, each time getting closer and closer to maintaining the encompassing context that gives heft to the importance of the information, until I finally *get it*. I've fit that last nugget into my already saturated brain-sponge.

And then I get up to take a break and realize it's past dinner time, my partner is starving and angry, my dog is also starving and needs to poop, and I've forgotten to do any kind of meaningful prep for the evening or, you know, anything in general.

At which point I completely shut down, my low blood-sugar catches up to me and I start shaking violently. Then the cold-sweats come and all I can think about is getting glucose, carbohydrates, protein, and anything else I can find, stuffed down my throat as fast as humanly and safely possible. Yes, I know the indigestion is gonna suck. I know my digestion and elimination tomorrow is also gonna suck. But if I don't eat as much as I possibly can *right now* I'm not entirely convinced I won't die where I stand.

Which is why I'm writing this. I've recognized this meta-pattern, and this brain-dump is an opportunity to put on some music, think critically about my day and experiences, and see what things I can do to prevent this total system failure from happening in the future. So far writing like this has been *very* effective, because it's like coasting the bike down the canyon pass after having spent the entire day climbing to the top. I'm still technically working, but I'm working *by letting go*. I'm letting the pent-up and neglected non-directional parts of my brain spring out and have their say.

It's super liberating actually. You've probably already noticed this, but you can even see the releasing of tension as represented by the change in my tone. The beginning of this post is very academic & technical - compressing *lots* of information into as few words as possible. And then starting somewhere around the end of the 4th and beginning of the 5th chapters (it was "brain-sponge," that was the moment), I started letting go. The tone shifted, as well as the rigidity & pressure of the piece.

And now here I am. I'm still a bit wordy, but I'm grooving. I'm back in the flow and excited again, but for reasons totally unrelated to the inner workings of a particular programming language. I can't guarantee this is what'll happen every time I try to short-circuit a total system shutdown, but it happened this time which means there's opportunities to explore here.

My usual come-down methods used to involve exercise, playing music or video games, or jumping right into a long list of house chores.

Of those 4, here's how things would typically go:

1. exercise => I push *way* too hard, like body-harmingly hard
2. music => I play *way* too hard, like shred my fingers and melt my ears hard
3. video games => I play *way* too long, like lose myself in escapist fantasy lands until I'm shaken back to reality by bodily functions (mine or otherwise)
4. chores => these actually tend to go okay, but what suffers is the prioritization; i.e. it could be well past time to start preparing dinner, but for some reason I'm cleaning the back porch, or I've gone to the grocery store and come back with a box of cereal, a couple protein bars, and some frozen waffles...

So here I am. It's 4:55 PM and I've spent nearly 8 consecutive hours reading documentation, and instead of spiraling into one of those 4 whirlpools, I wrote this. I feel good. I feel ready to handle the rest of the night, and do so in a way that supports not only myself, but the people around me as well.

Now all I have to do is post this to my 2-years-out-of-date Next.js blog and resist every urge I have to fix all the warnings that just showed up in my terminal...

The result of researching a maddening error

Resolving FaunaDB Privacy Conversions in Go


So, apparently if you save your Fauna data in standard JSON notation, like this:

```json
{
    "name": "some name",
    "age": 25,
}
```

And then you define your struct in Go like this, to match it (for serialization purposes):

```go
type Person struct {
    name string
    age  int
}
```

When you run:

```go

var person Person
res, err := data.At(f.ObjKey("data")).Get(&person); err != nil {
    // handle err
}
```

You're NOT going to find any data in `person`! This is most likely due to how Go handles public vs private exports (i.e. - anything that starts with a lowercase value is private, anything with uppercase is public).
When the fauna library tries to convert the data from the found object to your struct, it doesn't have access to any of those properties, so you'll just get an object with a bunch of defaults or nil values.

However, if you update your struct to have publicly exported properties:

```go
type Person struct {
    Name string
    Age  string
}
```

The parser STILL won't know what to do with it, because the field names in your JSON are all lowercase! In this instance, it's the opposite problem - Go's happily serving up the fields it can support
but Fauna's all like, "none of those match pal".

The only time it works, is if BOTH data sets/types are capitalized.

```json
{
    "Name": "some name",
    "Age": 25
}
```

```go
type Person struct {
    Name string
    Age  string
}
```

At that point, both peculiarities are satisfied, and you can get the data you're looking for.

Granted, Fauna probably has some annotations you can append to the struct in order to avoid this issue, similar to how you'd annotate \`json:"value"\` for anything you're trying to json.Marshal into a particular format.
But I haven't found the documentation on that yet, and it's currently 2:33AM, so I'm not likely going to put that much more effort into it tonight.

Hope this helps someone else though!

A quick & dirty local server for Netlify functions written in golang

How to Serve Golang Netlify Functions Locally


I've been writing a lot of Netlify (AWS Lambda) functions lately, and I got tired of having to create an
entirely separate control flow in order to test my logic. I wanted to be able to run these functions locally
just like the JavaScript peeps.

So I created this little localServer package! It's not published in git anywhere yet, so I'll update this if/when that happens. But, the 
gist of what's going on is: I made a stdlib golang http server that captures incoming http events, mocks them to the
`events.APIGatewayProxyRequest` and `events.APIGatewayProxyResponse` objects which the `aws-lambda-go/events` package 
looks for, and then passes those objects to the main AWS Lambda handler.

So far it only supports GET, POST, and OPTIONS requests (it throws `Access-Control-Allow-Origin: *` on everything), and it 
only mocks the proxy items that I'm actually using, like `Method`, `QueryStringParameters`, `Body`,
and so forth. But it's working for me so far, and that's as good a place to start as any.

I'm sure the function signatures could stand some cleaning-up, but I'll get there when I get there.

Here's the code for it:

```go
package localServer

import (
	"fmt"
	"github.com/aws/aws-lambda-go/events"
	"io/ioutil"
	"net/http"
)

func LocalServer(endpointMock string, port int, proxyHandler func(event *events.APIGatewayProxyRequest)(*events.APIGatewayProxyResponse, error)) {
	http.HandleFunc(endpointMock, handleLocal(proxyHandler))
	log.Printf("listening on port %d", port)
	log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", port), nil))
}

func handleLocal(proxyHandler func(event *events.APIGatewayProxyRequest)(*events.APIGatewayProxyResponse, error)) func(w http.ResponseWriter, r *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		var body []byte
		keys := make(map[string]string)
		w.Header().Set("Access-Control-Allow-Origin", "*")
		w.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
		w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
		w.Header().Set("Content-Type", "text/html; charset=utf-8")

		if r.Method == http.MethodOptions {
			return
		}

		if r.Method == http.MethodPost {
			parsed, err := ioutil.ReadAll(r.Body)
			body = parsed
			if err != nil {
				http.Error(w, "failed to read body", http.StatusInternalServerError)
				return
			}
		}

		if r.Method == http.MethodGet {
			queryKeys := r.URL.Query()
			for k, v := range queryKeys {
				keys[k] = v[0]
			}
		}

		request := &events.APIGatewayProxyRequest{
			HTTPMethod:            r.Method,
			RequestContext:        events.APIGatewayProxyRequestContext{},
			Body:                  string(body),
		}

		if len(keys) > 0 {
			request.QueryStringParameters = keys
		}

		res, err := proxyHandler(request)
		if err != nil {
			http.Error(w, fmt.Sprintf("failed to convert request to lambda event: %s", err), res.StatusCode)
			return
		}

		w.WriteHeader(res.StatusCode)
		fmt.Fprintf(w, res.Body)
	}
}
```

And then to use it in your Netlify function's `main.go`, add it to a sibling directory called `localServer`, and invoke it
like so:

```go
package main

import "main/localServer"

func main() {
	//lambda.Start(HandleEvent)
	localServer.LocalServer("/YOUR_ENDPOINT", 3000, HandleEvent)
}
```

The final file-tree should look like:

```plaintext
functionRoot
└── main.go
      localServer
      └──  localServer.go
```

And voila! You've got a local server you can test & debug against. Just make sure to move everything back over to the
`lambda.Start()` once you're finished. Also, it's not COMPLETELY like the JavaScript package, because you'll have to run
this on a function-by-function basis (just make sure you update your port), rather than at the top of your functions directory, 
or even better, the project root.

Future work!



A Simple CSV Writer in Go


Hey future-Shiplet, this is the pattern you usually follow. I gotchu.

\- past-Shiplet


```go
package main

import (
	"encoding/csv"
	"fmt"
	"io"
	"log"
	"os"
)

type CustomCSVWriter struct {
	Writer  *csv.Writer
	Records [][]string
}

type CustomCSVRecord struct {
	ColumnA string
	ColumnB string
	ColumnC string
}

func (c *CustomCSVRecord) ToCsvString() []string {
	return []string{
		c.ColumnA, c.ColumnB, c.ColumnC,
	}
}

var HeaderRow = [][]string {
	{"ColumnA", "ColumnB", "ColumnC"},
}

func initCSV(writer io.Writer) *CustomCSVWriter {
	return &CustomCSVWriter{
		Writer: csv.NewWriter(writer),
		Records: HeaderRow,
	}
}

func buildCSV(inMemCSV *CustomCSVWriter) {
	for i := 0; i < 25; i++ {
		record := CustomCSVRecord{
			ColumnA: "test",
			ColumnB: "test2",
			ColumnC: "test3",
		}
		inMemCSV.Records = append(inMemCSV.Records, record.ToCsvString())
	}
}

func main() {
	title := "CSV_TITLE.csv"
	file, err := os.Create(title)
	if err != nil {
		log.Fatalln("failed to create csv: ", err)
	}

	inMemCSV := initCSV(file)
	buildCSV(inMemCSV)
	inMemCSV.Writer.WriteAll(inMemCSV.Records)
	fmt.Printf("wrote %d records to %s", len(inMemCSV.Records), title)
}
```


Thoughts on patterns I've observed in infosec

Why Is Security Both So Important and So Clunky?


_These are personal observations about the role of security in everyday interactions with technology. It's not intended_
_to be exhaustive or dogmatic._

---

Information Security is important because it's the technological response to plotholes in our collective understanding of computers and the people who use them.

Aside from the _lowest_ levels of physics and electrical engineering at work in the design of chips - or
the things that turn electricity into logic - and many times even then, every functional layer of a computer is the result of a decision
made by a person or group of people. The people making these decisions are (ideally) intelligent, scientifically informed,
rigorous, and, inevitably, isolated from the totality of the ways in which their decisions can be used or abused.

The cumulative effect of these rigorous and informed decisions is that each successive or "downstream" decision is based on 1.) assumptions
made _about_ and 2.) assumptions made _by_ the structural layers immediately before it. We can know what those assumptions are at either
a surface level or at the level of their actual implementation, but unless we specialize in every facet of a particular
vertical of logic, it becomes unsustainable to maintain a complete and unproblematic understanding of the structure and
nuance of an entire machine, let alone hundreds, all of which contain their own architecture and design. Factor in the use of 
general- and special-purpose libraries to perform certain critical or non-critical tasks, and the complexity multiplies
significantly.

Both hackers and the people who slow them down know this, so they often focus on and initially redefine the _assumptions_ themselves, 
rather than their particular _implementations_. What this tends to reveal is that the most vulnerable
parts of a structure are the parts that humans interact with most closely and most frequently, both in their engineering and in their consumption, 
and so those assumptions shift to allow for the _unpredictability of human behavior_ as well as the more general
_predictability of human motivations_. The interplay between these two forces raises questions and inspires answers
that affect nearly every element of technology and how we interact with it. It also, to the furthest extent possible,
shifts the burden of responsibility to the consumer through individual liability, rather than the engineer.

This exchange is predicated on and enforceable through a social contract between the engineer and the consumer. 
The engineer's portion of the contract is that they've done their due diligence in simplifying & streamlining 
usage while also hardening against attack vectors, and that if the consumer uses the product _in the way in which it's intended_, 
they'll be kept safe. The consumer's portion of the contract is that they'll follow these guidelines, indeed use the product as intended, 
and absolve the engineer of the steepest responsibilities of failure. Either half of the contract is rendered unenforceable in cases 
of negligence.

Given this, the role of Information Security is to translate human behavior and motivation into reproducible steps either to
uphold OR to break that contract and thereby redefine and re-provision it. The weakest clauses of this contract are: 

  1. That the logic of any particular decision is sufficiently complete.
  2. That the definition of _intended use_ is sufficiently broad.

This results in an iterative exchange between engineers and consumers, informed by attackers as a third party, and which
fluctuates among varying ergonomic degrees. Fortunately, these degrees of clunkiness have, overall, trended toward less 
friction in adoption, rather than more.

shiplet.dev

CS Fundamentals

Serverless in Go

Generalist Fancies