Rendering a Triangle with Apple's Metal API, using Go

Did you learn that the OpenGL API is being deprecated in macOS 10.14?
Are you interested in giving Apple's Metal API a try?
Are you a fan of Go and don't feel like switching to Swift or Objective-C for this?
Then this post is for you.

By the end of this post, we'll render a single frame with a colorful triangle using Metal. We'll render to an off-screen texture, and then copy its contents into an image.Image for further inspection.

I'll focus more on the Go code to make this happen. If you're not already familiar with the general principles of modern low-level GPU APIs such as Metal, Vulkan, Direct3D 12, it's a good idea to learn more about them first. There are lots of resources on this topic, and they're easily findable. For Metal specifically, I can recommend watching the Metal for OpenGL Developers session from WWDC18 that gives a pretty good overview, especially for those who are familiar with OpenGL.

General approach

Metal API is officially available in Objective-C and Swift variants. We will use cgo, which allows calling C code from Go, to use the Objective-C Metal API. A wrapper Go package can be created, exposing Metal as a convenient Go API.

I've already started a Go package for this at dmitri.shuralyov.com/gpu/mtl, and I'll be using it below. It's in very early stages of development though, so its API is going to evolve over time. You're welcome to use it as is, as a starting point for your own version (i.e., fork it), or just for reference.

Hello Triangle high-level overview

At a high level, there are 10 steps we'll follow to render the triangle:

  1. Create a Metal device. (It needs to be available on the system.)
  2. Create a render pipeline state. (This includes vertex and fragment shaders.)
  3. Create a vertex buffer. (It will contain vertex data: the position and color of each triangle vertex.)
  4. Create an output texture. (To render into. We'll specify a storage mode, dimensions and pixel format.)
  5. Create a command buffer. (We'll encode all the commands for rendering a single frame into it.)
  6. Encode all render commands.
  7. Encode all blit commands. (This is to synchronize the texture from GPU memory into CPU-accessible memory.)
  8. Commit and wait. (Until all encoded commands have completed executing.)
  9. Read pixels from output texture. (Into an image.)
  10. Save the image. (As a PNG.)

Let's look at each of the steps in more detail.

1. Create a Metal device

This is the starting point.

It will work as long as your system has a Metal device on it, otherwise the error will report there isn't one. See the system requirements for Metal if you're not sure about your Mac.

device, err := mtl.CreateSystemDefaultDevice()
if err != nil {
	log.Fatalln(err)
}

2. Create a render pipeline state

Metal uses the Metal Shading Language. Here's a starting program containing very basic vertex and fragment shaders:

#include <metal_stdlib>

using namespace metal;

struct Vertex {
	float4 position [[position]];
	float4 color;
};

vertex Vertex VertexShader(
	uint vertexID [[vertex_id]],
	device Vertex * vertices [[buffer(0)]]
) {
	return vertices[vertexID];
}

fragment float4 FragmentShader(Vertex in [[stage_in]]) {
	return in.color;
}

The vertex shader emits vertices from the vertex buffer. Note that each vertex contains a float4 position and color. float4 are four 32-bit float values.

We'll come back to this in the next step, since the vertex data we supply from the Go side needs to align with how the vertex shader will interpret it.

For simplicity, we'll put the Metal Shading Language program source code in a const in our Go code, and have Metal compile the shaders from that source. Then, we can use them to create a render pipeline state. It looks like this:

// Create a render pipeline state.
const source = `#include <metal_stdlib> ...`
lib, err := device.MakeLibrary(source, mtl.CompileOptions{})
if err != nil {
	log.Fatalln(err)
}
vs, err := lib.MakeFunction("VertexShader")
if err != nil {
	log.Fatalln(err)
}
fs, err := lib.MakeFunction("FragmentShader")
if err != nil {
	log.Fatalln(err)
}
var rpld mtl.RenderPipelineDescriptor
rpld.VertexFunction = vs
rpld.FragmentFunction = fs
rpld.ColorAttachments[0].PixelFormat = mtl.PixelFormatRGBA8UNorm
rps, err := device.MakeRenderPipelineState(rpld)
if err != nil {
	log.Fatalln(err)
}

3. Create a vertex buffer

This is where our geometry data is described. Each vertex contains a 4D position and an RGBA color. We'll use the f32.Vec4 type, which map 1:1 to the aforementioned float4 type in the vertex shader.

// Create a vertex buffer.
type Vertex struct {
	Position f32.Vec4
	Color    f32.Vec4
}
vertexData := [...]Vertex{
	{f32.Vec4{+0.00, +0.75, 0, 1}, f32.Vec4{1, 0, 0, 1}},
	{f32.Vec4{-0.75, -0.75, 0, 1}, f32.Vec4{0, 1, 0, 1}},
	{f32.Vec4{+0.75, -0.75, 0, 1}, f32.Vec4{0, 0, 1, 1}},
}
vertexBuffer := device.MakeBuffer(unsafe.Pointer(&vertexData[0]), unsafe.Sizeof(vertexData), mtl.ResourceStorageModeManaged)

To create a buffer, we need to give it some raw bytes. At this time, we'll use C-style unsafe code to pass the pointer to the beginning of the memory block and its size. It might be worth considering avoiding unsafe and instead passing a safely converted []byte, but I'll leave that for future work.

4. Create an output texture

We'll specify an RGBA pixel format, with each component being a normalized uint8 ranging from 0 to 255. 512x512 will be the output size.

// Create an output texture to render into.
td := mtl.TextureDescriptor{
	PixelFormat: mtl.PixelFormatRGBA8UNorm,
	Width:       512,
	Height:      512,
	StorageMode: mtl.StorageModeManaged,
}
texture := device.MakeTexture(td)

Importantly, we've used managed storage mode. That means there are two copies of the texture data: one in GPU memory, and another in CPU-accessible memory. As a result, when one makes changes to it, we'll need to synchronize the resource before the other can safely access the latest contents.

We'll deal with that in step 7 using a blit command encoder.

5. Create a command buffer

To render a frame, we need a command buffer to encode commands into. We can get a command queue from the device, and create a command buffer from it.

cq := device.MakeCommandQueue()
cb := cq.MakeCommandBuffer()

6. Encode all render commands

This step encodes all render commands into the command buffer.

// Encode all render commands.
var rpd mtl.RenderPassDescriptor
rpd.ColorAttachments[0].LoadAction = mtl.LoadActionClear
rpd.ColorAttachments[0].StoreAction = mtl.StoreActionStore
rpd.ColorAttachments[0].ClearColor = mtl.ClearColor{Red: 0.35, Green: 0.65, Blue: 0.85, Alpha: 1}
rpd.ColorAttachments[0].Texture = texture
rce := cb.MakeRenderCommandEncoder(rpd)
rce.SetRenderPipelineState(rps)
rce.SetVertexBuffer(vertexBuffer, 0, 0)
rce.DrawPrimitives(mtl.PrimitiveTypeTriangle, 0, 3)
rce.EndEncoding()

We've created a render command encoder that clears the color attachment on load, stores the results at the end, clears using black clear color, and uses texture as the render target.

We set the render pipeline state from step 2, the vertex buffer from step 3, and then issue a draw call with triangle primitive type, starting at index 0 and with 3 vertices in total.

EndEncoding indicates the end of encoding commands into the render command encoder.

7. Encode all blit commands

This step is necessary because the output texture we created in step 4 uses the managed storage mode:

Importantly, we've used managed storage mode. That means there are two copies of the texture data: one in GPU memory, and another in CPU-accessible memory. As a result, when one makes changes to it, we'll need to synchronize the resource before the other can safely access the latest contents.

We'll deal with that in step 7 using a blit command encoder.

The commands we've encoded in the previous step ensure a triangle is rendered to the texture by the GPU, so now's the time to tell it to synchronize that texture for CPU access.

// Encode all blit commands.
bce := cb.MakeBlitCommandEncoder()
bce.Synchronize(texture)
bce.EndEncoding()

The Synchronize blit command does exactly that.

8. Commit and wait

By now, we've encoded all the render and blit commands we wanted. We commit the command buffer, which gets the GPU to start executing them all, and wait for the all encoded commands to finish executing.

cb.Commit()
cb.WaitUntilCompleted()

If we didn't do the wait and moved on to the next step right away, we might get a partially-rendered triangle when reading the texture pixels.

9. Read pixels from output texture

We did wait, so we can now safely call texture.GetBytes to read the texture pixels. We'll copy them into an *image.NRGBA that we create with the same dimensions as the texture we created. Its pixel byte layout matches exactly, so we can just copy directly into img.Pix byte slice:

// Read pixels from output texture into an image.
img := image.NewNRGBA(image.Rect(0, 0, texture.Width, texture.Height))
bytesPerRow := 4 * texture.Width
region := mtl.RegionMake2D(0, 0, texture.Width, texture.Height)
texture.GetBytes(&img.Pix[0], uintptr(bytesPerRow), region, 0)

10. Save the image

We have the image.Image, let's write it to disk as a PNG with image/png package:

// Write output image to a PNG file.
err = writePNG("triangle.png", img)
if err != nil {
	log.Fatalln(err)
}

Where writePNG is a simple utility function:

// writePNG encodes the image m to a named file, in PNG format.
func writePNG(name string, m image.Image) error {
	f, err := os.Create(name)
	if err != nil {
		return err
	}
	defer f.Close()
	err = png.Encode(f, m)
	return err
}

Result

Putting all the steps together gives us the hellotriangle command. You can see it at dmitri.shuralyov.com/gpu/mtl/example/hellotriangle.

When we run it on a Metal-capable system, we get a beautiful triangle!

The triangle you render yourself will look even prettier to you, I promise. Good luck and have fun!

1 comments

Custom JSON unmarshaler for a GraphQL client

(This post was originally posted as part of the GopherAcademy Advent 2017 series.)

In this post, I will tell a story of how I had to build a custom JSON unmarshaler for the needs of a GraphQL client library in Go. I'll start with the history of how everything started, build motivation for why a custom JSON marshaler was truly needed, and then describe how it was implemented. This is going to be a long journey, so strap yourself in, and here we go!

History of GraphQL Client Library in Go

GraphQL is a data query language for APIs, developed internally by Facebook in 2012, and made publicly available in 2015. It can be used as a replacement for, or in addition to REST APIs. It some ways, it offers significant advantages compared to REST APIs, making it an attractive option. Of course, as any newer technology, it's less mature and has some weaknesses in certain areas.

In May of 2017, I set out to build the first GraphQL client library for Go. Back then, only GraphQL server libraries existed in Go. My main motivation was wanting to be able to access GitHub GraphQL API v4 from my Go code, which had just come out of early access back then. I also knew that a general-purpose GraphQL client library would be useful, enabling Go projects to access any GraphQL API. There were GraphQL clients available in other languages, and I didn't want Go users to be missing out.

I spent a week on the initial investigation and research into what a Go client for GraphQL could look like. GraphQL is strongly typed, which is a good fit for Go. However, it also has some more advanced query syntax and features that play better with more dynamic languages, so I had my share of concerns whether a good client in Go would even be viable. Fortunately, at the end of that week, I found that a reasonable client in Go was indeed possible, and pushed a working initial prototype that had most basic functionality implemented, with a plan for how to implement and address the remaining features.

I documented the history of my findings and design decisions made in this issue. I've also given a talk (slides here) about the rationale and thinking behind many of the API and design decisions that went into the library. In this post, I want to talk about something I haven't covered before: implementing a custom JSON unmarshaler specifically for the needs of the GraphQL client library, in order to improve support for GraphQL unions.

JSON Unmarshaling Task at Hand

Unmarshaling JSON into a structure is a very common and well understood problem. It's already implemented inside the encoding/json package in Go standard library. Given that JSON is such a well specified standard, why would anyone need to implement their own JSON unmarshaler?

To answer that, I need to provide a little context about how GraphQL works. The GraphQL client begins by sending a request containing a GraphQL query, for example:

query {
	me {
		name
		bio
	}
}

The GraphQL server receives it, processes it, and sends a JSON-encoded response for that query. The response contains a data object and potentially other miscellaneous fields. We're primarily interested in the data object, which looks like this:

{
	"me": {
		"name": "gopher",
		"bio": "The Go gopher."
	}
}

Notice it has the same shape as the query. That's why the graphql package was designed so that to make a query, you start by defining a Go struct variable. That variable then both defines the GraphQL query that will be made, and gets populated with the response data from the GraphQL server:

var query struct {
	Me struct {
		Name string
		Bio  string
	}
}
err := client.Query(ctx, &query, nil)
if err != nil {
	// Handle error.
}
fmt.Println(query.Me.Name)
fmt.Println(query.Me.Bio)

// Output:
// gopher
// The Go gopher.

Initially, encoding/json was used for unmarshaling the GraphQL response into the query structure. It worked well for most things, but there were some problems.

Motivation for Custom JSON Unmarshaler

There were at least 3 clear problems with encoding/json for unmarshaling GraphQL responses into the query structure. These served as motivation to write a custom JSON unmarshaler for graphql needs.

  1. The largest problem with using encoding/json became apparent when looking to support the GraphQL unions feature. In GraphQL, a union is a type of object representing many objects.

    query {
        mascot(language: "Go") {
            ... on Human {
                name
                height
            }
            ... on Animal {
                name
                hasTail
            }
        }
    }
    

    In this query, we're asking for information about Go's mascot. We don't know in advance what exact type it is, but we know what types it can be. Depending on whether it's an Animal or Human, we ask for additional fields of that type.

    To express that GraphQL query, you can create the following query struct:

    var query struct {
        Mascot struct {
            Human struct {
                Name   string
                Height float64
            } `graphql:"... on Human"`
            Animal struct {
                Name    string
                HasTail bool
            } `graphql:"... on Animal"`
        } `graphql:"mascot(language: \"Go\")"`
    }
    

    The JSON-encoded response from GraphQL server will contain:

    {
        "mascot": {
            "name": "Gopher",
            "hasTail": true
        }
    }
    

    You can see that in this case the shape of the response doesn't quite align with the query struct. GraphQL inlines or embeds the fields from Animal into the "mascot" object. The encoding/json unmarshaler will not be able to handle that in the way we'd want, and the fields in the query struct will be left unset. See proof on the playground.

    You could try to work around it by using Go's embedded structs. If you define query as:

    type Human struct {
        Name   string
        Height float64
    }
    type Animal struct {
        Name    string
        HasTail bool
    } `graphql:"... on Animal"`
    var query struct {
        Mascot struct {
            Human  `graphql:"... on Human"`  // Embedded struct.
            Animal `graphql:"... on Animal"` // Embedded struct.
        } `graphql:"mascot(language: \"Go\")"`
    }
    

    That gets you almost the right results, but there's a significant limitation at play. Both Human and Animal structs have a field with the same name, Name.

    According to the encoding/json unmarshaling rules:

    If there are multiple fields at the same level, and that level is the least nested (and would therefore be the nesting level selected by the usual Go rules), the following extra rules apply:

    1. Of those fields, if any are JSON-tagged, only tagged fields are considered, even if there are multiple untagged fields that would otherwise conflict.

    2. If there is exactly one field (tagged or not according to the first rule), that is selected.

    3. Otherwise there are multiple fields, and all are ignored; no error occurs.

    Multiple fields are ignored. So, Name would be left unset. See proof on the playground.

    An initial reaction might be that it's a bug or flaw in encoding/json package and should be fixed. However, upon careful consideration, this is a very ambiguous situation, and there's no single clear "correct" behavior. The encoding/json unmarshaler makes a sensible compromise for generic needs, not GraphQL-specific needs.

  2. To have additional control over the GraphQL query that is generated from the query struct, the graphql struct field tag can be used. It allows overriding how a given struct field gets encoded in the GraphQL query. Suppose the user happens to use a field with a name that doesn't match that of the GraphQL field:

    var query struct {
        Me struct {
            Photo string `graphql:"avatarUrl(width: 194, height: 180)"`
        }
    }
    

    The JSON-encoded response from GraphQL server will contain:

    {
        "me": {
            "avatarUrl": "https://golang.org/doc/gopher/run.png"
        }
    }
    

    As a result, query.Me.Photo will not be populated, since the field is "avatarUrl" in the response, and the Go struct has a field named "Photo", which doesn't match.

    This happens because encoding/json unmarshaler is unaware of the graphql struct field tags.

  3. Consider if the user supplied a query struct that happened to contain json struct field tags, for example:

    type query struct {
        Me struct {
            Name string `json:"full_name"`
        }
    }
    

    (Suppose the user wants to serialize the response later, or uses some struct that happens to have json tags defined for other reasons.)

    The JSON-encoded response from GraphQL server will contain:

    {
        "me": {
            "name": "gopher"
        }
    }
    

    As a result, query.Me.Name will not be populated, since the Go struct has a JSON tag calling it "full_name", but the field is "name" in the response, which doesn't match.

    This happens because encoding/json unmarshaler is affected by json struct field tags.

This motivation lead to the conclusion that for GraphQL-specific needs, a custom JSON unmarshaler is unavoidably needed.

Implementing a Custom JSON Unmarshaler

Discarding a well written, thoroughly tested, battle proven JSON unmarshaler in the Go standard library and writing one from scratch is not a decision to be taken lightly. I spent considerable time looking at my options and comparing their trade-offs.

Writing it from scratch would've been the last option to consider. I could've made a fork of encoding/json and modified it. But that would mean having to maintain a fork of encoding/json and keep it up to date with any upstream changes.

The key insight was that the process of JSON unmarshaling consists of two independent parts: parsing JSON, and populating the target struct fields with the parsed values. The JSON that GraphQL servers respond with is completely standard, specification-compliant JSON. I didn't need to make any changes there. It was only the behavior of populating target struct fields that I needed to customize.

In Go 1.5, the encoding/json package exposed a JSON tokenizer API to the outside world. A JSON tokenizer parses JSON and emits a sequence of JSON tokens, which are higher-level and easier to work with compared to the original byte stream. I could make use of this to avoid having to parse the JSON myself.

The encoding/json JSON tokenizer is available via the Token method of json.Decoder struct:

// Token returns the next JSON token in the input stream.
// At the end of the input stream, Token returns nil, io.EOF.
//
// ...
func (dec *Decoder) Token() (Token, error)

Calling Token repeatedly on an input like this:

{
	"Message": "Hello",
	"Array": [1, 2, 3],
	"Number": 1.234
}

Produces this sequence of JSON tokens, followed by io.EOF error:

json.Delim: {
string: "Message"
string: "Hello"
string: "Array"
json.Delim: [
float64: 1
float64: 2
float64: 3
json.Delim: ]
string: "Number"
float64: 1.234
json.Delim: }
io.EOF error

Great! We don't have to deal with all the low-level nuances of parsing JSON strings, escaped characters, quotes, floating point numbers, and so on. We'll be able to reuse the JSON tokenizer from encoding/json for all that. Now, we just need to build our unmarshaler on top of it.

Let's start by defining and iterating on our decoder struct that contains the necessary state. We know we're going to base it on a JSON tokenizer. To make it very clear we're only ever using the Token method and nothing else from json.Decoder, we can make it a small interface. This is our starting point:

// decoder is a JSON decoder that performs custom unmarshaling
// behavior for GraphQL query data structures. It's implemented
// on top of a JSON tokenizer.
type decoder struct {
	tokenizer interface {
		Token() (json.Token, error)
	}
}

And the exported unmarshal function will look like this:

// UnmarshalGraphQL parses the JSON-encoded GraphQL response
// data and stores the result in the GraphQL query data
// structure pointed to by v.
//
// The implementation is created on top of the JSON tokenizer
// available in "encoding/json".Decoder.
func UnmarshalGraphQL(data []byte, v interface{}) error {
	dec := json.NewDecoder(bytes.NewReader(data))
	dec.UseNumber()
	err := (&decoder{tokenizer: dec}).Decode(v)
	return err
}

We create a new JSON decoder around data, which will act as our JSON tokenizer. Then we decode a single JSON value into v, and return error, if any.

Pop Quiz: Is there a difference in behavior between unmarshaling and decoding a single JSON value?

Here's a pop quiz. Suppose you have some JSON data and and you're looking to unmarshal it into a Go variable. You could do one of two things:

err := json.Unmarshal(data, &v)
err := json.NewDecoder(r).Decode(&v)

They have slightly different signatures; json.Unmarshal takes a []byte while json.NewDecoder accepts an io.Reader. We know that the decoder is meant to be used on streams of JSON values from a reader, but if we only care about reading one JSON value, is there any difference in behavior between them?

In other words, is there an input for which the two would behave differently? If so, what would such an input be?

This was something I didn't quite know the answer to, not before I set out on this journey. But now it's very clear. Yes, the behavior indeed differs: it differs in how the two handle the remaining data after the first JSON value. Decode will read just enough to decode the JSON value and stop there. Unmarshal will do the same, but it doesn't stop there; it continues reading to check there's no extraneous data following the first JSON value (other than whitespace). If there are any additional JSON tokens, it returns an "invalid token after top-level value" error.


To stay true to unmarshaling behavior, we perform a check to ensure there are no additional JSON tokens following our top-level JSON value; if there is, that's an error:

func UnmarshalGraphQL(data []byte, v interface{}) error {
	dec := json.NewDecoder(bytes.NewReader(data))
	dec.UseNumber()
	err := (&decoder{tokenizer: dec}).Decode(v)
	if err != nil {
		return err
	}
	tok, err := dec.Token()
	switch err {
	case io.EOF:
		// Expect to get io.EOF. There shouldn't be any more
		// tokens left after we've decoded v successfully.
		return nil
	case nil:
		return fmt.Errorf("invalid token '%v' after top-level value", tok)
	default:
		return err
	}
}

Ok, now let's figure out all the remaining state we need to keep track of in the decoder struct. We will implement unmarshaling with an iterative algorithm rather than recursive, and keep all relevant state in decoder struct.

We know that the JSON tokenizer provides us with one token at a time. So, it's up to us to track whether we're in the middle of a JSON object or array. Imagine you get a string token. If the preceding token was [, then this string is an element of an array. But if the preceding token was {, then this string is the key of an object, and the following token will be its value. We'll use parseState json.Delim to track that.

We'll also keep a reference to the value where we want to unmarshal JSON into, say, a v reflect.Value field (short for "value").

What we have so far is:

type decoder struct {
	tokenizer interface {
		Token() (json.Token, error)
	}

	// What part of input JSON we're in the middle of:
	// '{' is object, '[' is array. Zero value means neither.
	parseState json.Delim

	// Value where to unmarshal.
	v reflect.Value
}

That's a good start, but what happens when we encounter a ] or } token? That means we leave the current array or object, and... end up in the parent, whatever that was, if any.

JSON values can be nested. Objects inside arrays inside other objects. We will change parseState to be a stack of states parseState []json.Delim. Whenever we get to the beginning of a JSON object or array, we push to the stack, and when we get to end, we pop off the stack. Top of the stack is always the current state.

We need to apply the same change to v, so we know where to unmarshal into after end of array or object. We'll also make it a stack and rename to vs []reflect.Value (short for "values").

Now we have something that should be capable of unmarshaling deeply nested JSON values:

type decoder struct {
	tokenizer interface {
		Token() (json.Token, error)
	}

	// Stack of what part of input JSON we're in the middle of:
	// '{' is object, '[' is array. Empty stack means neither.
	parseState []json.Delim

	// Stack of values where to unmarshal. The top of stack
	// is the reflect.Value where to unmarshal next JSON value.
	vs []reflect.Value
}

That should be enough for now. Let's look at the code for unmarshaling next.

Remember that the UnmarshalGraphQL function calls decoder.Decode method. Decode will accept v, set up the decoder state, and call decode, where the actual decoding logic will take place.

// Decode decodes a single JSON value from d.tokenizer into v.
func (d *decoder) Decode(v interface{}) error {
	rv := reflect.ValueOf(v)
	if rv.Kind() != reflect.Ptr {
		return fmt.Errorf("cannot decode into non-pointer %T", v)
	}
	d.vs = []reflect.Value{rv.Elem()}
	return d.decode()
}

decode is implemented as an iterative algorithm that uses the state in decoder struct. This is the entire algorithm at a high level:

// decode decodes a single JSON value from d.tokenizer into d.vs.
func (d *decoder) decode() error {
	// The loop invariant is that the top of the d.vs stack
	// is where we try to unmarshal the next JSON value we see.
	for len(d.vs) > 0 {
		tok, err := d.tokenizer.Token()
		if err == io.EOF {
			return io.ErrUnexpectedEOF
		} else if err != nil {
			return err
		}

		// Process the token. Potentially decode a JSON value,
		// or handle one of {, }, [, ] tokens.
		switch tok := tok.(type) {
			...
		}
	}
	return nil
}

There's a big outer loop. At the top of the loop, we call d.tokenizer.Token to get the next JSON token. The loop invariant is that the top of the vs stack is where we unmarshal the next JSON value we get from Token. The loop condition is len(d.vs) > 0, meaning we have some value to unmarshal into. When the vs stack becomes empty, that means we've reached the end of the JSON value we're decoding, so we break out and return nil error.

Each loop iteration makes a call to Token and processes the token:

That's basically it. The rest of the code are the details, managing the parseState and vs stacks, checking for graphql struct field tags, handling all the error conditions, etc. But the algorithm is conceptually simple and easy to understand at this high level.

Except... We're still missing one critical aspect of making it handle the GraphQL-specific needs that we set out to resolve originally.

Let's recall the GraphQL unions example, where the JSON-encoded GraphQL server response contained:

{
	"mascot": {
		"name": "Gopher",
		"hasTail": true
	}
}

And we're trying to unmarshal it into:

var query struct {
	Mascot struct {
		Human struct {
			Name   string
			Height float64
		} `graphql:"... on Human"`
		Animal struct {
			Name    string
			HasTail bool
		} `graphql:"... on Animal"`
	} `graphql:"mascot(language: \"Go\")"`
}

The behavior we want is to unmarshal "Gopher" string into all matching fields, which are these two:

But the top of our vs stack only contains one value... What do we do?

We must go deeper. Cue the music from Inception, and get ready to replace vs []reflect.Value with vs [][]reflect.Value!

Multiple Stacks of Values

That's right, to be able to deal with having potentially multiple places to unmarshal a single JSON value into, we have a slice of slices of reflect.Values. Essentially, we have multiple (1 or more) []reflect.Value stacks. decoder now looks like this:

type decoder struct {
	tokenizer interface {
		Token() (json.Token, error)
	}

	// Stack of what part of input JSON we're in the middle of:
	// '{' is object, '[' is array. Empty stack means neither.
	parseState []json.Delim

	// Stacks of values where to unmarshal. The top of each stack
	// is the reflect.Value where to unmarshal next JSON value.
	//
	// The reason there's more than one stack is because we
	// might be unmarshaling a single JSON value into multiple
	// GraphQL fragments or embedded structs, so we keep track
	// of them all.
	vs [][]reflect.Value
}

We need to modify decode to create additional stacks whenever we encounter an embedded struct or a GraphQL fragment (field with graphql:"... on Type" tag), do some additional bookkeeping to manage multiple stacks of values, check for additional error conditions if our stacks run empty. Aside from that, the same algorithm continues to work.

I think getting the data structure to contain just the right amount of information to resolve the task was the most challenging part of getting this to work. Once it's there, the rest of the algorithm details fall into place.

If you'd like to learn even more of the low-level details of the implementation, I invite you to look at the source code of package github.com/shurcooL/graphql/internal/jsonutil. It should be easy to read now.

Payoff

Let's quickly revisit our original GraphQL unions example that wasn't working with standard encoding/json unmarshaler. When we replace json.UnmarshalJSON with jsonutil.UnmarshalGraphQL, the Name fields get populated! That's good news, it means we didn't do all that work for nothing.

See proof on the playground.

jsonutil.UnmarshalGraphQL also takes graphql struct field tags into account when unmarshaling, and doesn't get misled by json field tags. Best part is we're reusing the rigorous JSON tokenizer of encoding/json and its public API, so no need to deal with maintaining a fork. If a need to apply further GraphQL-specific changes to unmarshaling behavior arises in the future, it will be easy to do so.

Conclusion

It has been a lot of fun implementing the GraphQL client library for Go, and trying to make the best API design decisions. I enjoyed using the tools that Go gives me to tackle this task. Even after using Go for 4 years, it's still the absolutely most fun programming language for me to use, and I'm feeling same joy I did back when I was just starting out!

I'm finding GraphQL to be a pretty neat new technology. Its strongly typed nature is a great fit for Go. APIs that are created with it can be a pleasure to use. Keep in mind that GraphQL shines most when you're able to replace multiple REST API calls with a single carefully crafted GraphQL query. This requires high quality and completeness of the GraphQL schema, so not all GraphQL APIs are made equal.

Note that there are two GraphQL client packages to choose from:

I've had a chance to actually use githubv4 for real tasks in some of my Go projects, and it was a pleasant experience. That said, their GraphQL API v4 is still missing many things present in GitHub REST API v3, so I couldn't do as much with it as I would've liked. They're working on expanding it, and it'll be even better when fully complete.

If you want to play around with GraphQL or take a stab at creating your own API with it, you'll need a GraphQL server library. I would suggest considering the github.com/neelance/graphql-go project as a starting point (if you want a complete list of options, see here). Then, you can use any GraphQL client to execute queries, including the graphql package from this post.

If you run into any issues, please report in the issue tracker of the corresponding repository. For anything else, I'm @shurcooL on Twitter.

Happy holidays, and enjoy using Go (and GraphQL) in the upcoming new year!

0 comments

Running gofmt in Browser with 10 Lines of Code (Using GopherJS)

I saw a message in Gophers Slack #general chat that stood out to me:

but running gofmt in the browser is, um, hard

The context was that they had a 100%-client-side JavaScript application that produced some Go code, but the Go code wasn't gofmted.

"Wait a minute, that's not hard," I thought. "It's trivial! Getting gofmt to run in the browser? I bet I could do it in 10 lines of code!"

Right?

Then I realized. It's not trivial. It's not obvious. It just seems that way to me because I've done a lot of things like this. But many people probably haven't.

So that inspired me to write this blog post showing how you, too, can have gofmt functionality running in the browser alongside your JavaScript code very easily. The only thing I'll assume is you're a Go user and have a working Go installation (otherwise, you're unlikely to be interested in gofmt).

gofmt has relatively complex behavior, so how can it all be implemented in just 10 lines? The trick is that we don't have to rewrite it all from scratch in JavaScript. Instead, we can write it in Go, as that'll be much easier.

To get Go to run in the browser, we'll use GopherJS. GopherJS is a compiler that compiles Go into JavaScript, which can then run in browsers. We'll use two Go packages:

Building JavaScript that implements gofmt

So, let's get started. I am assuming you already have the current version of Go installed. First, you'll need to install the GopherJS compiler if you don't already have it. Its README has a detailed Installation and Usage section, but it boils down to running one command:

$ go get -u github.com/gopherjs/gopherjs

If you have your $GOPATH/bin added to your PATH, you'll be able to easily invoke the installed gopherjs binary.

Now, let's create a small Go package that'll implement our functionality. Make a new directory somewhere in your GOPATH, cd into it and write a main.go:

$ mkdir -p $(go env GOPATH)/src/github.com/you/gofmt
$ cd $(go env GOPATH)/src/github.com/you/gofmt
$ touch main.go
package main

import ("go/format"; "github.com/gopherjs/gopherjs/js")

func main() {
	js.Global.Set("gofmt", func(code string) string {
		gofmted, _ := format.Source([]byte(code))
		return string(gofmted)
	})
}

Then run gopherjs build in same directory to build it:

$ gopherjs build --minify

The --minify flag causes it to write minified output. (You'd also better use HTTP compression ala Content-Encoding: gzip when serving it, because the generated JavaScript compresses very well.)

The output will be in the form of a gofmt.js file (and an optional source map in gofmt.js.map, useful during development). We can then include it in any HTML page:

<script src="gofmt.js" type="text/javascript"></script>

Once executed, a gofmt function will be available for the rest of your JavaScript code to use. The function will format any Go code you provide as a string, returning the output as a string:

Image

That's 10 lines which implements gofmt—we're done! Okay, not quite. I'm ignoring errors for now, and the code itself is not gofmted. We'll come back and fix that next, but first, let's see what's happening in the code.

Code Explanation

First, we're creating a function literal that takes a string as input and returns a string. format.Source is used to format the incoming Go source code, and then it's returned. We're reusing a lot of existing Go code, the same that the real gofmt command uses, which is well tested and maintained.

Then, we're using js.Global.Set with the parameters "gofmt" and our function literal.

js.Global is documented as:

Global gives JavaScript's global object ("window" for browsers and "GLOBAL" for Node.js).

And Set is:

Set assigns the value to the object's property with the given key.

What we're doing is assigning the JavaScript variable named "gofmt" with the value of a func we've created.

GopherJS performs the neccessary conversions between Go types and JavaScript types, as described in the table at the top of https://godoc.org/github.com/gopherjs/gopherjs/js. Specifically:

Go type JavaScript type Conversions back to interface{}
string String string
functions Function func(...interface{}) *js.Object

So Go's string becomes a JavaScript String type, and a Go func becomes JavaScript Function.

By making that js.Global.Set("gofmt", func (code string) string { ... }) call, what we've done can be effectively expressed with the following JavaScript code:

window.gofmt = function(code) {
    // Implementation of gofmt, in JavaScript, done for you by the GopherJS compiler.
    // Lots and lots of generated JavaScript code here...
    return gofmted;
}

Just like that, you've got an implementation of gofmt that can run in the browser.

Finishing Touches

Let's come back to the code and make it nicer. We'll gofmt it, to avoid the irony, and add some error checking:

package main

import (
	"go/format"

	"github.com/gopherjs/gopherjs/js"
)

func main() {
	js.Global.Set("gofmt", func(code string) string {
		gofmted, err := format.Source([]byte(code))
		if err != nil {
			return err.Error()
		}
		return string(gofmted)
	})
}

Now we get nicer output when there's an error formatting the code:

Image

But maybe you'd rather just fall back to showing the original input if formatting fails:

gofmted, err := format.Source([]byte(code))
if err != nil {
	return code
}

Or maybe you want to include the error message on top:

gofmted, err := format.Source([]byte(code))
if err != nil {
	return err.Error() + "\n\n" + code
}

Or maybe return a boolean indicating success:

func(code string) (string, bool) {
	gofmted, err := format.Source([]byte(code))
	if err != nil {
		return code, false
	}
	return string(gofmted), true
}

This depends on the exact needs of your JavaScript application. I'll leave it to you.

Additional Notes

One of potential concerns might be that the generated JavaScript is relatively large, so it may negatively impact page loading time. There are tradeoffs in compiling Go to JavaScript, and large size is one of them.

I have two thoughts on that:

  1. You can always do progressive enhancement. Load your page as usual, but use async attribute in the script tag for the gofmt code. Page loads as quickly as before, but during the first second, user can't gofmt (it can print a nice placeholder message like "hang on, that functionality is still loading...", or maybe fall back to a remote server). Once the gofmt functionality loads, it starts working locally. Most people are unlikely to notice that first second the app isn't completely functional yet. And on second load, it'll be cached, etc.

  2. In the future, WebAssembly will let you do/write things that will beat pure JavaScript in terms of parse/load speed. The Go code you write today will work largely unmodified, but become more optimized in the future. The large generated JavaScript output is a short term implementation detail.

In the end, there are tradeoffs between various approaches, so you should use the tools and approaches that make most sense for your project.

Conclusion

Hopefully, you saw just how easy it is to take some non-trivial functionality that happens to be already written in Go, and expose it to your JavaScript application.

Next week, I'll post another example of using net/http, encoding/csv packages in the browser to stream CSV data from a server and parse it on the fly, so stay tuned.

Leave a reaction at the bottom of this post if you found it helpful, and comment if you have feedback.

3 comments

Using method values to get rid of global variables

Have you ever had a piece of Go code like this:

// TODO: Get rid of this global variable.
var foo service

func fooHandler(w http.ResponseWriter, req *http.Request) {
	// code that uses foo
}

func main() {
	foo = initFoo()

	http.HandleFunc("/foo", fooHandler)
}

One way to get rid of that global variable is to use method values:

type fooHandler struct {
	foo service
}

func (h fooHandler) Handle(w http.ResponseWriter, req *http.Request) {
	// code that uses h.foo
}

func main() {
	foo := initFoo()

	http.HandleFunc("/foo", fooHandler{foo: foo}.Handle)
}

Of course, net/http also has http.Handle so in this case you can simply change fooHandler to implement http.Handler interface:

type fooHandler struct {
	foo service
}

func (h fooHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	// code that uses h.foo
}

func main() {
	foo := initFoo()

	http.Handle("/foo", fooHandler{foo: foo})
}

But method values are great when you need to provide a func, for example in https://godoc.org/path/filepath#Walk or https://godoc.org/net/http#Server.ConnState.

2 comments

Trying out the Oculus Rift CV1 for the first time

Last night I got a chance to setup and spend a few hours with the long-awaited first consumer version of the Oculus Rift (aka Oculus Rift CV1). I'll describe my experience in some detail.

Background

This is not my first time trying on a VR headset. I've tried the first developer kit (DK1) a long time ago, but only for a few minutes. It was compelling enough to make me understand that the final release, scheduled a few years off at the time, would be incredible, but of course there were a few "must fix this and must make that better first" items remaining. I had to use my imagination to predict what it'd feel like in the end. I tried two experiences: a pretty basic unexciting demo of pong in 3D, and a virtual walkthrough through Japan in the form of a 360 video. The latter was great for making me feel the presence factor and ability to experience an exotic real-world location.

Some time later, I tried the second dev kit (DK2) with a pretty sweet experience. There was a demo booth at a movie theater that was playing Interstellar, and they had a 5-8 minute Interstellar-themed experience. You sat in a nice, comfy chair, they put on the DK2 and a pair of high-quality headphones, and you were transported into a situation on board of a futuristic spaceship. It was a high-budget demo, so the 3D environment was professionally made with lots of detail. You could look around, and primarily, you could feel the sense of presence... Actually being in that spaceship, and exploring what it had to offer. My favorite parts were the section where they made you feel weightlessness. With a countdown from 3, the experience creators used a combination of motion, a "whoosh" sound, and the chair you were sitting in slightly dropping down, to very convincingly make you feel that suddenly gravity was turned off. The other part I loved was the end, where you got to a very detailed cockpit of the spaceship, complete with many controls, buttons, joysticks. When you looked out the windows, you could see some planet in the nearby distance...

Initial setup

Ok, fast forward to now and the experience of unboxing and setting up the consumer version of the Rift.

From the moment you see the box itself, you can tell it's a very high-quality and well made product. As you release the side magnetically held in place and open up the briefcase, your immediate reaction is confirmed. This is a really cool product. The unboxing and setup experience is quite close to what you'll get with Apple products. If you appreciate that, you'll like this.

I'm not going to talk much about the PC part of the equation (because I dislike it), so I'll say this to get it out of the way. We got an Oculus Ready PC from their recommended list. I think it's great they have that list so you can pick something you know will work, but it's not a great experience to setup, have and maintain a Windows PC in my opinion. I wish it'd "just work" with something like OS X (which I use) or Linux (because it's open source and makes a better potential foundation for everything), or possibly the PS4 or the future generation consoles. Until then, I had to setup Windows, uninstall McAfee anti-virus which was preinstalled and causing problems, and put up with Windows. If your standards are not as high as mine, you might be fine with it, but I didn't think it was great. If you want more reasons why Windows isn't great, just look at some tweets from Jonathan Blow, he points it out pretty often.

When you open up the box, you see a large note saying "To setup your Oculus Rift, go to oculus.com/setup", which is pretty much what I'd hope for in 2016. You run their installer, and it says it'll take 30-60 minutes to set everything up. I found that to be quite accurate. It walks you through everything step by step with very friendly directions, and in general worked well (aside from McAfee thing causing it to not install at first). Having a computer that meets the system requirements is absolutely mandatory for you to have a good time.

The only minor issue I ran into was getting a plastic tab out of the remote so that its battery would form a closed loop. I literally had to use pliers to pull the plastic tab out. After that, everything was fine (and everything was 100% smooth). The setup then proceeded to download and update the firmware on the tiny little remote, which was pretty telling of the age we live in, and then onto next steps.

Putting it on

Halfway through the setup, once everything is connected, you've placed the sensor, made some room, and learned how to adjust the straps so it can sit comfortably on your head, the setup asks you to put on your remote, and then the rift, in order to proceed with the rest of the setup... in virtual reality.

The first thing you see is a futuristic, mostly all white scene, with cool smoke effects, and large text guiding you in front of you. You can also see the tracking sensor, which I've placed on the table in front, in VR.

The coolest part is how it maps 1:1 to the sensor's actual location in the real world. The tracking and responsiveness felt absolutely perfect to me. There is full degrees of motion tracking, so you could move your head anywhere, tilt it, rotate it, and it all just worked. I did not think or feel that there is a computer rendering frames at a certain frame-rate (not until later, anyway, when I looked up some developer docs). Instead, it felt like magic. You felt as if you're inside a virtual world that you could comfortably look around.

The setup concluded with a brief sample demo VR experience, made of a few short standing scenes. A cartoony field with a moose and rabbit in front of a campfire. Then, another planet with an alien standing right in front of you, looking at you while mumbling, and making some faces. It felt like he was right there, and I just stood still in disbelief of how realistic it felt. Then, a scene with a huge t-rex dinosaur walking up to you in a hallway. As I stood there, I couldn't help but grin in anticipation of learning what will happen next. "There's no way he'll try to eat me, this has got to be a friendly demo... right?" I was definitely a little freaked out by that possibility. The dinosaur gets pretty up close, but doesn't attack you, which was great.

Top three experiences

In the next few hours, I tried some of what the Oculus Home store had to offer and these are my top three experiences.

Oculus Dreamdeck

The Oculus Dreamdeck, which is a few additional scenes from the end of setup, is a great introduction and a sample of the interesting locations and situations that VR can instantly teleport you into, and make you feel like a part of. For me, it simply shows that content creators now have a really open new paintbrush to use in order to create interesting and compelling experiences. Previously, they were limited to books, photos, movies. But what can you achieve with VR? I think it's very interesting to find out.

Lucky's Tale

Lucky's Tale. A cute, family-friendly, traditional platformer game in 3D. But made for VR.

It was absolutely cool how the world felt pretty small, because you were pretty up close to it. And you could look around just by moving your head... and looking around. This was a great sitting experience, played with a controller. VR allowed you to be a part of the world, rather than simply looking at it through a window in the form of your display.

The most telling aspect here was the next day, when I remembered the experience of playing Lucky's Tale differently. I remembered being actually inside the 3D world and seeing all its details in front of me. I did not remember sitting in a chair, or looking at a TV, or what my environment was when I was playing the game. I remembered actually being inside the game's world.

Live for Speed

Live for Speed. It's a very realistic and well made racing simulator. Its developers have prioritized getting top-notch VR support with the latest patch, and it was incredible.

The Oculus Rift currently suffers from not having a made-for-VR dedicated controller, so we have to use traditional input devices like keyboard, mouse, gamepad. There is no tracking of your hands, so when in VR, you typically feel as if you have no body. However, in LFS, the driver has a body, with his hands on the steering wheel and legs near pedals. It's pretty much where your body would be. When you put your headset on, you instantly end up being inside a car. It's probably the most realistic and believable experience. I could look around, look at the mirrors, look at the seats in the back, out the windows, etc., just by moving my head.

Social

One of the defining aspects of VR is its ability to instantly teleport you into a typically unattainable situation or environment. Want an entire stadium all to yourself? Or have an entire theater for your personal enjoyment? Or a sweet home theater setup? It's easy in VR.

If you want to get away from people temporarily, VR can probably simulate that really well. But it can also make you feel very lonely quickly, if the environment you're in is really cool but you have no one to share it with. That's why social aspects are going to be big and very important. It's no surprise Facebook bought it.

After trying it for the first few minutes, the first thing I wanted to do was to share this really cool experience with other people and tell them about it.

Content and possibilities of the medium

With the Oculus Rift, you get Oculus Home. It's impressive what they've been able to put together in such short time, but it clearly lacks behind the likes of Steam by a large margin.

Over time, there will be more content and more experiences, games, and tools created for VR. But it's going to take hard work and money (to pay for the time).

Still, it's very interesting to see what'll happen. I feel this is a whole new medium that is as exciting as the year TV became a reality. Imagine how you'd feel after seeing the very first movie in your life, knowing that directors can now share stories and come up with cool new experiences for that medium. Well, I can't wait to see what talented directors can do with VR, and what they'll be able to come up with when they control not just the vertical and horizontal pixels on a flat screen, but so many more of your senses.

As any medium, it can be used in good ways but also abused. It's no different than books in this regard.

It's also possible to ruin VR for yourself if you approach it adversarially. It's still quite basic and it's easy to push it to its limits and find ways in which it breaks, so if you want to prove you're not afraid of falling down a virtual ledge, it's not hard. The point is that if you can suspend your disbelief just a little, then the feeling of presence offered can be very compelling. That lets you be able to enjoy amazing experiences.

Closing thoughts

After trying the final consumer version, it's very clear to me how big this is and going to become. It's not a gimmick like the 3D vision goggles were 5-10 years ago. This is the real thing, finally happening now. It's a whole new medium that will unlock all kinds of possibilities that weren't possible before. It's a great time to be creative and artistic, because you can express yourself in whole new ways.

It will take some time, after all, the preorders are still only beginning to ship, but I expect over the next few years, more and more people will have tried and understood VR for themselves, resulting it in being adopted by consumers en masse, not too dissimilar from how most people have mobile devices now.

Eventually (some years later), as the technology improves and becomes more accessible and commonplace, and VR headset resolutions increase, the need for monitors will go away, and our work desks will start to look very different from now. They had paper on them first, then large CRT displays, replaced by LCD panels of today, and VR headsets in the future.

Special thanks to Sourcegraph for getting a Rift for our office, so that we can start thinking about and looking for more ways to bring the future sooner.

1 comments

Fireside chat with Dmitri Shuralyov, dev tools hacker

Dmitri Shuralyov (shurcooL) was one of the first users of Sourcegraph whom we didn't personally know. We've enjoyed meeting up with him over meals and seeing him at all of the Go meetups in San Francisco. He is the quintessential early adopter—and creator—of new kinds of development tools, such as Conception-go (a platform for dev tools experimentation) and gostatus (which lets you track changes to a whole GOPATH as if it's a single repository). He's bringing the Go philosophy to other languages, starting with markdownfmt (gofmt for Markdown).

We talked with Dmitri to learn about his sources of programming inspiration, and to hear about his current Go projects.

How and why did you start getting involved in open source programming?

I'm no stranger to using closed source software over the years. Some of it is really nice, and I enjoy using it. But as someone who writes code, I inevitably want to make it even better, fix bugs or contribute in other ways. But I can't do that unless the project is open source.

For all my personal projects, I always default to making them open source. Just because if someone out there likes it and wants to help out and make it better, I want to welcome that instead of denying them that opportunity.

In addition to that, I strongly believe in the benefits of code reuse, and open source libraries allow for that. When you have a specific task and there's one great library for it, you can use it. If you or someone else makes an improvement, everyone using that library benefits, and the improvement only needs to be done once.

What are the libraries and authors you have learned the most from (or who have most influenced your work)?

Without a doubt, the biggest influence and inspiration for my personal work has been Bret Victor. His talks and articles, especially Inventing on Principle and Learnable Programming, have helped me understand and convey what really drives me. I have a set of guiding principles that I believe would help make software development better, and my personal work is about finding out if that's the case.

On the programming side, one of the first non-trivial Go projects I wanted to make was to extend the Go-syntax printing of vars ala fmt.Printf("%#v", foo) but have it work recursively on nested structs, producing gofmt-ed output. Dave Collins (@davecgh) had created go-spew, which did the deep printing part, but it wasn't quite gofmt-ed Go-syntax. With his permission, I forked go-spew to create go-goon, and proceeded to make the necessary output formatting changes. At that time, I was just learning about Go's reflection, so having a high quality working solution was a tremendous help.

I've made a few PRs to gddo, the source of godoc.org. Every time I thought I had a decent solution, Gary Burd (garyburd) would provide some really helpful feedback and suggestions. The end result would be much better code and seem obvious in retrospect. I've definitely learned a lot from that experience.

Richard Musiol (@neelance) is incredibly interesting to follow on GitHub. I've "watched" the GopherJS project since its early days, and it's incredible the rate at which he's able to write high quality code that implements features, fixes bugs, and solves challenging tasks that are a part of compiling Go into JavaScript.

Gustavo Niemeyer (@gniemeyer) comes to mind as having some of the highest quality, nicely documented Go packages I've seen. He really sets a high bar in that regard. If you want to see how nice a Go package can be, definitely take a look at some of his work.

And of course the core members of the Go team that are largely responsible for most of the standard library. It's very often I end up peeking into the source of common funcs from std lib in order to confirm some detail or learn more about how they work. The Go standard library is an excellent learning material source in every sense.

What are you currently working on?

I've recently come to realize that the project I'm currently working is essentially my GOPATH workspace. It includes my own open source repos, but also various open source libraries that I rely on.


Conception-go

My main personal project continues to be Conception, now written completely in Go. Being a very ambitious undertaking, it still requires a lot of work both on the frontend and backend. Luckily, the backend code is shared between many of my other repos. For example, I use my github_flavored_markdown package to render GFM inside Conception. My gostatus tool, which recently gained support for displaying git stash info, uses the same internal code to figure out the status of Go package repos, and Go-Package-Store reuses that yet again to present a list of Go packages in your GOPATH that have updates, complete with the commit messages of what has changed.


markdownfmt: gofmt for Markdown

Lately, I've been working on improving the diff highlighting within github_flavored_markdown as well as inside Conception. Ideally, I want the same code to be doing the highlighting in both places, but the highlighting interfaces inside Conception need to be rewritten before that's possible. I've also just recently improved my GoPackageWorkingDiff() func to include new/untracked files.

I'm also working on adding support for displaying notifications to trayhost, which is needed for the InstantShare client. InstantShare sits in your menu bar and allows you to instantly upload images and get a URL. The reason there's no delay, even for large images, is because both the upload and download are streamed. The backend server part was written by my friend Pavel Bennett (@pavelbennett/pavben) in Go.

Now that we're out of go1.3 feature freeze, I really want to fix issue 5551 in go/format of the Go standard library. It has existed since go1.1, and it has caused me a lot of grief and blocked many things I wanted to build that would require it. I still have to continue to shell out to the gofmt binary, which performs poorly for large inputs and isn't available in all contexts.

What do you use Sourcegraph for?

I use Sourcegraph on various occasions to get answers to ad-hoc programming questions. There was a time where I was looking for Go code that implements a certain public interface. Another time I wanted to find example usages of the (url.Values).Encode() method.

Just recently, I was looking at Close bool in net/http.Request. I really wanted to find examples of how it was being used in the real world, and I was easily able to do that with the help of Sourcegraph. That's just not something I would've been able to do in any other way.

But I feel that the things Sourcegraph offers now is just the tip of the iceberg, a preview of things to come. If you look at what it's doing, it's essentially going through all the open source code out there and performing code analysis on it. Then it's making this connected graph of all the identifiers and types and so on, currently presented as a nice web interface with links that let you navigate code easily. As its doing that, it's gaining the ability to answer more and more questions about code that you may run into during your day. The kind of questions and answers that may help you learn how to do something new, or figure out what code will be affected if you change some public API, and help you get your task done just a little faster. That's pretty exciting if you ask me!

What do you hope will improve about the field of programming in the next 5 years?

I like that you've grounded the question by specifying 5 years. But that also makes it harder to answer, because it has to be more realistic, and I can't just say "flying cars and code that writes itself" yet. 😃

But you've also mentioned "hope", so I will be a little over-optimistic.

I hope new formats that are released follow suit of Go and come with standard formatter tools (like gofmt) from day one. Having a tool that formats your and everyone else's code on save to look the same and pretty is incredibly helpful, and retrofitting such tools later on is much harder (if not impossible) than doing it right away.

I hope we will have more widespread code refactoring and high level manipulation tools at our disposal. I do love my text editor for writing code and it's great, but sometimes I want to be able to easily rename identifiers, not perform find and replace with a manually crafted query. Or move packages seamlessly without having to manually move folders and update import paths. Tools like @bradfitz's goimports that automatically add/remove my Go imports are so transformative, it's hard to imagine going back to having to do that manually. I'd love to see more of similar innovation and improvement in this field.

But most importantly, I really hope we have some kind of breakthrough that would allow public APIs to change more often and more easily, without breaking packages that depend on the old version of the API. It's really a shame to see API improvements being purposefully held back due to concerns about backwards compatibility. There are some attempts to help with this, for example applying semver to APIs. But that still requires people to manually update to new library APIs. gofix is an inspiration here, and perhaps it can be used as a starting point in turning this into reality. Can we do better than the status quo? I'm looking forward to finding out!


Thanks to Dmitri for his contributions to the dev tools and Go communities and for sharing his inspirations and projects with us!

This interview was conducted by Quinn Slack (@sqs).

0 comments

How I use GOPATH with multiple workspaces

First off, I want to make it clear I have a fixed GOPATH that I do not change per project. The GOPATH env var is set inside my ~/.bash_profile file and doesn't change. Every Go package I have exists in no more than one place. I tend to have all my personal dependencies on latest version, simply because it's easier to have everything up to date than any alternative.

I do make use of the fact that the GOPATH environment variable is defined as a list of places rather than a single folder.

From http://golang.org/cmd/go/#hdr-GOPATH_environment_variable,

The GOPATH environment variable lists places to look for Go code. On Unix, the value is a colon-separated string. On Windows, the value is a semicolon-separated string. On Plan 9, the value is a list.

My GOPATH consists of 3 folders or GOPATH workspaces.

The first one is my landing workspace. Since it's listed first, whenever I go get any new package, it always ends up in this workspace.

Go searches each directory listed in GOPATH to find source code, but new packages are always downloaded into the first directory in the list.

I make it a rule to never do any development in there, so it's always completely safe to clean this folder whenever it gets too large (with Go packages I don't use). After all, it only has Go packages that I can get again with go get.

My second workspace is for all my personal Go packages and any other packages I may want to "favorite" or do some development on. I move things I use regularly from first workspace into second.

My third workspace is dedicated to the private Go packages from my work, and their dependencies. It's convenient to have my work packages separate from all my personal stuff, so they don't get in each other's way.

With that setup, multiple GOPATH workspaces feel a lot like namespaces. The reason I have more than one, to me, is quite similar why one might want to break a medium-sized Go package into several .go files. The result is effectively the same since multiple .go files share the same scope but allow one to have "namespaces".

Similarly, multiple GOPATH workspaces share the same scope (i.e., it's your "effective GOPATH") but allow you to have namespaces for categories of packages. Having 3 GOPATH workspaces with 100 packages each is no different than 1 GOPATH workspace with 300 packages. They don't overlap, similarly to how code in multiple .go files of a Go package doesn't overlap.

Conclusion

As long as multiple GOPATH workspaces are a supported feature, I don't see the motivation to actively force yourself to use only one GOPATH workspace. You should do whatever is optimal for your particular use case.

That said, I feel that having 2 GOPATH workspaces is nice just because it lets you easily "undo" go getting a package you no longer want to keep by having the first GOPATH workspace act as a temporary place. If I had to have just one GOPATH, I would feel very hesitant before doing go get on any new, unfamiliar Go package. What if it brings in 100 dependencies and I decide I don't want to keep it anymore? Undoing a go get currently is not straightforward... Unless you use 2 GOPATH workspaces and don't mind simply blasting the first one away.

5 comments

Open Questions in Software Development

As a kid, growing up, it seemed like math was a solved problem. It'd be very hard to invent something new, because so much has already been done by others. You'd have to get to a very high level, before you could go somewhere where no one has gone before.

In software development, it feels like the opposite situation. There are so many basic things that are currently unsolved and not possible. But they can be made possible with just some effort spent on it.

Here are some open questions that I can think off the top of my head (using the Go language for context):

I'll add more as I run into them.

0 comments

On understanding code

When understanding code, we build gigantic mental dependency graphs relevant to the task at hand. This requires much focus, time, memory.

— Dmitri Shuralyov (@shurcooL) February 14, 2013

Building these dependency graphs in your head from code is very intensive mentally, that's why programmers cannot take interruptions.

— Dmitri Shuralyov (@shurcooL) February 15, 2013

So when you're "in the zone" being very productive, it's because this graph is built. You know what will be affected when you change stuff.

— Dmitri Shuralyov (@shurcooL) February 15, 2013

The problem with that approach is you're storing valuable, hard to build information in your short term memory. Next morning it'll be gone.

— Dmitri Shuralyov (@shurcooL) February 15, 2013

If another person reads your code, they will not benefit from anything you've come up with in your mind. They will have to redo this work.

— Dmitri Shuralyov (@shurcooL) February 15, 2013

Optimize for how humans think and change things as a visual graph, not how computers used to store bytes 40 years ago as plain text files.

— Dmitri Shuralyov (@shurcooL) February 15, 2013

Code/functionality should be stored in a way that embeds this dependency information. This is what I'm trying to create with Conception.

— Dmitri Shuralyov (@shurcooL) February 15, 2013

So when you look at a completely unfamiliar line of code, nearby will be an exhaustive list of _all_ that will change/break as you edit it.

— Dmitri Shuralyov (@shurcooL) February 15, 2013

Sprinkle some live programming magic, and you have a system where it's very easy to experiment, test, see all relevant changes and undo. :D

— Dmitri Shuralyov (@shurcooL) February 15, 2013
0 comments

Two opposite paths to making life better

A friend shared the following article on Lifehacker about Otixo, a "Convenient File Manager for Dropbox, Google Drive, SkyDrive, and All Your Other Cloud Services," and it made me realize something.

http://lifehacker.com/5905784/otixo-is-a-convenient-file-manager-for-dropbox-google-drive-skydrive-and-all-your-other-cloud-services

It seems most products/services these days, and the accompanying articles describing them, follow a similar model:

"Create something that doesn't exist to solve some problem, or make some task easier."

In this specific case, they figured if a person has >1 cloud storage, managing them separately is hard so it'd make sense to create a unified interface that allows you to do that. And that's great!

However, I've found that recently (only within the last year or two, I think some time after I got my iPhone) I've started to look for another way to solve problems: by simplifying. I figure out what's essential to me, and try to throw everything else out.

So if I need to have cloud storage, there are 2 paths:

or

The advantage of path 1 is you get more space, but it involves more things. However, if u can get by with path 2, it involves less things, which I appreciate (more so than having that extra space, cuz right now 24 GB of free Dropbox is way beyond what I need).

My point is this: it seems my line of thinking (simplifying) is rather under-represented in terms of human efforts. How many people are working on products/services/writing articles about how to simplify, throw things out? Seems like very few.

If you want to improve your life and solve problems by simplifying, you gotta do it on your own. If you wanna solve problems by adding things, just go to lifehacker.com and such.

2 comments

Pursuit of Perfection

Apple is known as a company that strives for perfection in everything they do. They place a lot of attention on getting the smallest of details right. The little things that you rarely notice, not until you see someone else doing it wrong.

This attention to detail shows up everywhere. Take the power indicator on a MacBook for instance. It is meticulously programmed to light up in a solid white color only when the following two conditions are met:

  1. The MacBook is turned on
  2. Its LCD screen is not lit up

This ensures you can tell whether the computer is turned on at all times, yet the indicator light is never on when it would be redundant. After all, if the LCD is on, you will already know that the computer is on.

iOS is a fine example of high-quality Apple software. It's not always perfect, but the level of polish is above and beyond what you typically find in this very young but fast growing mobile industry. However, one of the side-effects of highly polished software is that when there are flaws in it, they become quite glaring in contrast to everything else that is done well.

There is one such flaw that I've found. Being able to tell the time on your phone is quite important, that's why it always takes its place front and center in the middle of the status bar.

When an iPhone is locked, time and date are displayed in a large font. To avoid displaying the time twice, needlessly, time within the status bar is replaced by a small lock icon.

The problem occurs when you get a call while your device is locked.

At this point, you lose the ability to find out the time until you respond to your phone call.

It's not a big deal, but I've found myself quite jarred by this a few times recently, when I wanted to find out the time before picking up the call.

Update: This has been fixed in iOS 6.0.

0 comments

The effect of motion blur on our perception of frame rate

I got my trusty CRT monitor capable of high refresh rates from the basement, to try my recent motion blur demo on it. It looked great on my 60 Hz LCD, so I had high expectations for 120 Hz.

I was shocked with the outcome. As expected, going from 60 to 120 Hz made a huge, noticeable difference in my demo with motion blur off. But turning it on, there was no significant difference. For the first time, I couldn't believe my eyes.

As a competitive gamer and someone very passionate about display technologies, I am very familiar with the concept of refresh rate. The more, the better, I always thought. 60 Hz is okay, but 120 is much better! I could always easily tell the difference. Until now.

After thinking about it, it made sense. This is probably the root of all "but the human eye can only see at most X frames per second" arguments on the internet. It matters what the frames are, or more precisely, how distinct they are. I can easily tell the difference in frame rate when they are very distinct, but I wasn't when motion blur made them less so.

You can experience the motion blur demo for yourself if you have a WebGL-capable browser.

The examples are abundant. Take movies. At 24 frames per second, the motion looks acceptably smooth. But take any video game running at 24 frames and it's unplayable. The difference is in motion blur. Each frame in film captures the light for roughly 1/24th of a second. In a video game, each frame is typically rendered as if with instant exposure time. For these type of frames, yes, the higher the frame rate, the smoother the motion appears.

I am planning to confirm my findings on a 120 Hz LCD monitor soon, but I am certain that the results will be similar.

3 comments

How to make your networked game smooth as butter

"By avoiding a future arrow from hitting your present self, you ensure your past self will dodge the present arrow."

Suppose you are designing a networking system for a multiplayer game. One of the biggest factors in the quality of networking design is how smooth the game feels for its players. People are not happy when they see snapping, jittering or other unexpected non-smooth behaviours. Such things occur because of the latency over the internet, and games have to try to predict things that are unpredictable by nature (e.g. player movement). Each time the prediction is wrong, the user sees an unpleasant correction.

One way to reduce such negative effects is to try to mask the corrections, by using smoothing.

I propose another approach. We can try to rely less on information that is uncertain, and rely on what is known. The latency is still there, but we can work around it.

Imagine an internet game server with multiple players, where everyone has less than 100 ms round-trip latency (and under 50 ms single-way latency). Those are good internet conditions, but not too unrealistic for today.

That means each player can know exactly all other players have been 100 ms ago, without having to use prediction. Let's render them there. The local player still sees himself at present time, but he sees all other players 100 ms in the past, 100% smoothly (ignoring dropped packets).

We want to let players aim at enemies, and not where they think enemies are due to their current latency (ala Quake). So the server performs hit collisions with regard to what the player shooting saw. Bullets travel in present time, but they collide with players' positions 100 ms in the past.

All the above has been done before (see Half-Life 1). But here's the kicker of this post.

What if you try to dodge a bullet by doing something within 100 ms of it hitting you. With the above system, you physically can't. No action, however drastic, within 100 ms of the bullet hitting you can save you, since the bullet will still hit where you were 100 ms ago (before you took said action).

It's not that big a deal for bullets, since they travel quite fast so there's little you can do in 100 ms to change anything. We can accept that as is. But can we do better?

What if, instead of a bullet, we have a slower moving projectile like an arrow. The player might want to jump out of the harm's way just before it hits them, just to spite their opponent. With the above system, they will see themselves clearing the arrow, yet it still hits their 100 ms in the past alter-ego and the player quits in frustration.

Yes, we can do better. We take advantage of the principle that player movement is not easily predictable, but arrow projectiles are (all you need are the initial conditions). That's why we render other players 100 ms in the past, but we can try to render other player's projectiles 100 ms in the future.

Now, when you see an arrow just about to hit your face, and you duck at the last millisecond, you're actually ducking 100 ms before the real arrow will have hit you. By avoiding a future arrow from hitting your present self, you ensure your past self will dodge the present arrow. :D

Additional Notes

7 comments

A Subway Ride

You are riding the subway. It's a regular train car. It feels familiar. There aren't many people. Some are reading newspapers. A woman takes a sip from her coffee cup. An older woman in a wheelchair waits patiently for her stop. The train moves silently, as if through a thick layer of fog.

It makes a stop and a male gets on.

The train continues its monotone passage through a moonless night. It's smooth sailing under the stars, except they're not visible from within the tunnel.

A man in the seat ahead talks out loud, either to himself or into a headset. More likely the former.

People continue to mind their own business, each one almost completely oblivious to the presence of others.

An empty coffee cup rocks back and forth under the seat in front, caressed gently by the soft ride.

Another stop. The next station is Eglinton. They become more frequent, each one seemingly marking a period at the end of a sentence in the lifetime story of the train.

Three girls chatter happily at the end of the train car. Some laughter is heard from the other end. As I near my destination, my heart rate goes up ever so slightly. The smell of excitement fills the car.

The night is young. Whatever adventures await ahead are still unwritten.

1 comments

Versionless Software

Suppose you have an ideal piece of software at version X with one bug in it. When the developer fixes the bug in version Y, you want to have that bug fixed too.

Without automatic updates, this means you have to manually update to Y. This has some cognitive effort associated with it. When multiplied by hundreds of different pieces of software, it is quite significant.

Wouldn't it be better to have it automatically update without the end user having to worry about or even know anything about versions? Yes and no:

I think the reasons for the 'no' answer can be, in theory, avoided, leaving yes and therefore making it possible to reduce cognitive load. To make our lives simpler by not having to regularly update software or deal with a major update from a very old version.

0 comments

Online non-simulations

I think there is a lot of untapped potential for online games. I mean something else altogether.

Most online games right now are perfect simulations, or rather they attempt to be. Most online games imply multiplayer. Each player connected to the same server will be a part of the same virtual world. What player A sees going on is very close to what player B sees (albeit from a different perspective).

Let's break that constraint. Suppose there is an online multiplayer game where two players are connected to the same server, yet they live and interact in two completely different realities. Think about the doors this opens. The opportunities are endless. However, not many of them would make fun games. But I believe there may be something worthwhile out there. We just have to explore this direction a bit.

There already are some examples of this situation happening. Take a RTS game that is peer-to-peer. Imagine for some reason the two players go out of sync. At this point, they may both think they are winning the battle. There will be two winners and no losers, and each may have a lot of fun... until they realize their games are not in sync (via talking), where that fun will quickly end and frustration will set in.

But that's only an example of a broken simulation. The multiplayer RTS game was designed to be in sync, not out.

Let's look at online games that were designed to be not in sync.

I remember in a multiplayer FPS America's Army, there were 2 teams of players fighting each other. The game was made so that whichever team you picked, it would appear as the "good guys" and the other team is the "terrorists." This is a very minor visual aspect that has no gameplay value whatsoever. Maybe it wasn't even worth mentioning. ;)

As all good writers do, I'm leaving the best example for the last. There is a short simple indie game called 4 Minutes and 33 Seconds of Uniqueness. Assuming you don't mind the following spoiler, the game objective is simply to be the only player in the world playing it for 4 minutes and 33 seconds. It's a simple as that. But it is a great example of a game with online capabilities that does not attempt to simulate the same reality for multiple players at the same time.

What else is possible out there? I hope we are yet to find out.

0 comments

The most boring post

There's a conflict of interest. I've just realized this, but I think I see what it is that makes some stories or people interesting to hear and some not.

It's a conflict of interest. When you are the one telling the story, a story you're excited about and want to convey it in the most interesting way possible. You don't want to give away any spoilers that will ruin it. So you take your time and explain everything in detail. Because that's how you would've liked to hear it, or so you imagine. The only reason you wanna hear it that way is because you already know everything, and now you're just going over it the second time.

Other people, on the other hand, have absolutely no idea what your story will be about. They want to know right away. You could tell them in one sentence, but you wouldn't feel satisfied with that. You want to paint the story in its full colours and detail. But other people want to know it as soon as possible. They would love to hear all the details if they already knew the big picture.

I guess the problem is that communication using words takes time; it's not instantaneous. If it were, we'd have no problems.

Personally, I don't even like the idea of movie or book titles, because they give away some of the surprise for me. Yet I want to know in advance if the movie or book will be interesting to experience. It's a conflict of interest.

I'm sure I could've worded this exact idea in just a few sentences, but I haven't. Why? I don't know, it's hard, and it's something I'll have to think about more and hopefully work on.

0 comments

In my defence, or why I don't care

This post is way overdue, and it is a perfect example of what I'm about to talk about.

I'm not very good at English writing or clearly and consicely expressing my thoughs in words. Yes, here I am talking about my weaknesses, and you're thinking why should you even continue reading what this guy has to say.

But it's ok, because I don't care. In a good way.

See, I have this theory that allows me to think that it's ok for people to be wrong or bad at something. At least, it shouldn't stop them from working on what they love (as long as it doesn't negatively impact others).

The reason for that is since there are many people, there's nothing wrong with them fanning out and trying to work in different directions (think of a non-deterministic Turing machine). Some will be right, some wrong. The alternative is catastrophic: groupthink. If everyone were to try ony one approach, then everyone is simply screwed if the group's consensus is wrong.

Consider the following example. At some point, people thought the Earth was flat. If not for the person/people who dared to go against the flow, perhaps we would never have discovered otherwise.

Hence, I believe that no matter how wrong or stupid something I'm doing may seem to others, I have perfect grounds to stick with it if I believe it to be the right thing.

Of course, this doesn't mean I will ignore all constructive critisim. On the contrary, I will happily consider all such feedback, and try to adjust if I see the error of my ways.

As far as my posts go, basically, I have two choices. Either care a lot about my spelling/grammar/presentation, spend too much time editing my posts until I believe they are ready for mass (lol) viewing, and post rarely. Or, not pay much attention to that stuff, but rather post things as they come to my mind with little editing or rewriting. This will allow me to actually use this blog the way it's meant to be. I hereby choose the latter.

0 comments

My interests

There are a bunch of topics that are of high interest to me, and I'll be using this blog to talk about some of them.

My main passion and a dream job is no doubt game development. This will likely be the main subject of this blog, as well as other computer related topics. However, I'm also into things like psychology, philosophy, HCI/User Interfaces, physics, racing, driving, snowboarding, longboarding, RC cars, racing simulators, displays, pixels, portable devices, in arbitrary order.

I like cars in general, and anything that has to do with driving, racing or making them. One of my long term dreams has always been to create a playground realistic driving sim where I'm able to experiment with various car designs. Until then, I have LFS. I also tend to play around with some remote controlled cars that I have.

I also like to delve into contemplations about the meaning of life, and other philosophy/psychology questions. I might post some of my ideas on that here.

Anyway, I doubt I will talk about every single of my interests here, so perhaps throwing all those tags out wasn't the best idea. Oh well.

0 comments

First post

So I've finally decided to create a blog. I will be posting things related to my various interests here.

I know I'm kinda throwing this out into a void here, but that's ok for now. It might a good outlet for me to get my ideas out and to think about them as I do. After all, it can't hurt, can it?

0 comments