Phil's Musings notes from a peripatetic programmer

Recent Posts

Introducing a Serverless Issue board for GitHub repositories and organizations!

At different projects use different ticket trackers because different people have different likes and preferences. We use Trello, Pivotal, GitHub Issues, and Wunderlist. But one thing that is common across all of our open source projects is GitHub.

As we’ve grown and as our projects have started to overlap we’ve realized there is value in having a common place to look and see what’s going on in all of the projects.


gh-board does all the things we need and more. If it doesn’t do something, submit a Pull Request and it will!

Other ticket trackers

Any ticket tracker presents additional friction to use with GitHub:

  • it’s hidden behind a login
    • (when a ticket is linked in our IM client we don’t get a nice preview)
  • the logins are different so @mentioning people is annoying
  • you have to remember a different type of markup language
  • you have to remember to link everything twice
    • so people looking at the ticket can get to the code and vice-versa
  • you have to update the tracker when the Pull Request status changes (created, review, tested, merged, etc)
  • they don’t show the state of Pull Requests so you then have to click to see what the Pull Request status is
  • URL’s are difficult to share because frequently the state of the page is not in the URL
    • ie which milestones, columns, or other filter criteria are being used

As a developer/tester/UX: you’d still have to check multiple places to stay on top of everything (or hope that your email client doesn’t explode!)

GitHub isn’t perfect either

But GitHub Issues is not without its limitations:

  • Issues are per-repository (we have 100 repositories)
  • Milestones are per-repository
  • Labels are per-repository
  • It is difficult to add additional metadata to a ticket
  • There is no easy way to have kanban-style columns

How is this different from other GitHub-based trackers?

It has a few features that other ticket trackers lack (like huboard or waffleio):

  • open source & free!
  • you can run it anywhere!
    • you can still use vanilla GitHub (nothing to import/export and no vendor lock-in)
  • real-time collaborative editing of Issues
  • shows the state of related Issues/Pull Requests
  • shows CI status and merge conflict status
  • has charts (burndown, gantt, etc)
  • keeps track of multiple repositories from different organizations
  • and productivity-enhancing Easter Eggs!


CI Status and Merge Conflict

  • CI Status shows up as a green check mark or a red x on the top-right corner of a card
  • Merge conflicts are shown with a yellow warning and have a diagonal striped background

image image image

Real-time Collaborative Editing


Issue Images

If an Issue or Pull Request contains an image then it will be shown in the Issue


Easter Eggs

Plus, it comes with productivity-enhancing easter eggs you can unlock!



Since it stores all the open and closed tickets locally, we can generate all the fancy charts that other ticket trackers generate.

  • Burnup chart: it clearly shows when new work is added to a Milestone
  • Gantt chart: shows when milestones are due and colors the bar based on the status of all the Issues


How does it work?


  • uses octokat.js and polls the GitHub API for changes
  • uses the 1st repository in the list to find the column labels and milestones
    • columns are defined as labels of the form ## - Column Title or you can specify a regular expression
  • stores all the Issues and Pull Requests in the browser (thanks to IndexedDB)
    • think of it like git clone but for Issues & Pull Requests
  • searches the IndexedDB to find related issues
Keep Reading...

Introducing the atom pull-requests package!

As a programmer that uses GitHub, Pull Requests are a great way to discuss code but whenever I get feedback on code in a large codebase it is annoying to have to find where that change was. Since I use as my text editor, I decided to write a plugin that adds a great feature of GitHub (Pull Requests) into my text editor (which also happens to be written by GitHub). And viola! pull-requests.

Whenever you check out a branch that has a Pull Request and open it in GitHub, you’ll see the GitHub comment directly on the line of code inside atom. And to help you find it, the Tree view on the left will show how many comments are in the directory so you can find the file.


Here’s a slightly out-of-date screencast showing the whole process (including installing the plugin):


How’s it made?

It uses the octokat.js npm package and the linter atom package. pull-requests checks if your code is in a git repository (actually, a GitHub one) and then checks if the branch corresponds to a Pull Request in the repository (or the parent repository if this is a forked repository) and pulls out the comments on the changes of a file.

Then, it uses linter to add lint messages on the lines of the files. Since linter supports HTML, pull-requests also converts the comment into HTML (complete with emojis) and adds a link to get back to the Pull Request on GitHub so you can continue discussing.

Hope that helps you!

Keep Reading...

The Death of Openstax?

OpenStax isn’t going anywhere; we have a ton of high-quality content and are revolutionizing textbook publishing for the benefit of students. But here are my thoughts on how it could be made a bit more open (and cheaper to boot!).


As background, openstax books contain a few pieces:

  1. an editor for creating a part of a book (Aloha) and organizing parts of a book into a Table of Contents
  2. rules for attribution (who authored the book)
  3. a way to convert the book into various formats (ePUB and PDF)
  4. a way to read the book online for free
  5. a way to mix-and-match and create a new book
  6. Ideally, openstax will have a way to allow others to suggest edits to a book



  1. Use GitHub to store the book content (book viewer, source for openstax books, and blog post)
  2. Use a browser editor to edit the book
  3. Use a little server to automatically create PDFs and ePubs (travis-ci or philschatz/pdf-ci)
  4. Support attribution automatically (example)
  5. Support “Suggested Edits” to content automatically via GitHub’s “Pull Request” (example)
  6. Support “Derived Copies” of content automatically via GitHub’s “Fork”
  7. Development of code and content using GitHub Issues via philschatz/gh-board


  1. All book content is stored directly in GitHub (see sources for examples)
  2. Replace the editor with oerpub/github-book-editor which saves directly to GitHub
  3. Replace the web view with Autogenerated Sites using GitHub (see and the various book repositories )
  4. Replace PDF (and ePub) generation with travis-ci or philschatz/pdf-ci (Every time GitHub updates, generate a new PDF)
  5. Support Derived copies using GitHub’s “Fork”
  6. Support “Suggested Edits” (which openstax used to have) via GitHub’s Pull Requests
  7. Support a diff of “Suggested Edits” via GitHub’s Markdown diff view (see blog post on books in GitHub)
  8. Support software (and content) development using something on top of GitHub Issues like huboard or ideally philschatz/gh-board (no server/subscription required)

Free Stuff!

  1. No server costs! (except PDF generation which can be minimized)
  2. autogenerated PDFs for every edition (see GitHub Tags and Releases)
  3. easy contributions from the entire world
    • travis-ci can make sure the Markdown is well-formed
  4. human-readable changes
  5. Issue tracking for content in each book
  6. Revisions for each book
  7. More reusability/remixability than you could imagine

The Kink

There is a “kink” in this process which I’d be happy to elaborate on:

  1. Each book needs to be a separate repository (cannot “easily” combine multiple books into 1)
  2. Need to learn MarkDown (specifically kramdown) unless the editor is “smart enough”

That’s how I would “kill” OpenStax with minimal effort and stay true to the openness that has helped OpenStax thrive.

For more projects check out my repositories

Keep Reading...

Openstax Textbooks Reader on GitHub using Kramdown

Over the weekend I converted all of our popular textbooks from our new HTML format to kramdown (a Markdown variant that allows classes) and tossed it on GitHub.

Since gh-pages automatically converts kramdown files to HTML using jekyll, I also created a tiny book reader and linked to it in the book. Click the screenshot to read all the books!

image Bonus Features

By using a Markdown variant instead of HTML, there are several things GitHub provides for free.

  • repo pages will render instead of just showing the markup
  • links between pages just work
  • diffs and Pull Requests render and add changebars/colors to make the changes clearer

Clickable Table of Contents


Note: {: ...} is kramdown’s way of adding classes

The markup:

layout: page

1.  {: .preface} [Preface](contents/
2.  {: .part} Unit 1: Levels of Organization
    1.  {: .chapter} [An Introduction to the...](contents/
        1. [Overview of Anatomy and Physiology](contents/
        2. [Structural Organization of the...](contents/
        3. [Functions of Human Life](contents/

    2.  {: .chapter} [The Chemical Level of...](contents/
        1. [Elements and Atoms: The Building...](contents/
        2. [Chemical Bonds](contents/
        3. [Chemical Reactions](contents/

Rendering in normal

The markdown files render on repo pages (see example page on GitHub)


and the raw kramdown:

title: "Anatomy of Selected Synovial Joints"
layout: page

<div data-type="abstract" markdown="1">
By the end of this section, you will be able to:

* Describe the bones that articulate together to form selected...
* Discuss the movements available at each joint
* Describe the structures that support and prevent excess movements


# Articulations of the Vertebral Column

In addition to being held together by the intervertebral discs,
vertebrae also articulate with each other at synovial joints formed
between the superior and inferior articular processes called
**zygapophysial joints**{: data-type="term" :} (facet joints) (see
[link text]( These are
plane joints that provide for only limited motions between the
vertebrae. The orientation of the articular processes at these joint
varies in different regions of the vertebral column and serves to
determine the types of motions available in each vertebral region.
The cervical and lumbar regions have the greatest ranges of motions.

![Figure alt text](../resource/fig1.jpg "Figure caption"){: data-title="Figure Title" .half-page :}

Rendered Changes

GitHub will also render the changes of kramdown files instead of showing a diff (see example of deleting a chapter)



  • unobtrusive design
  • big next/prev buttons
  • reading progress bar
  • keyboard left/right buttons work
  • Table of Contents shows pages you have read
  • spinner shows when page is loading
  • just 300 lines of code
  • works on any book hosted on too! Examples of all the Openstax College books
  • search button reuses the Table of Contents
  • does not require building the files for gh-pages (unlike GitBook)

Book Reader Links


  • the reader works even if files are stored in places other than /contents (thanks to URI.js)
  • kramdown does not support <figure> (unless you HTML escape it) so I added attributes to the <img> tag (for the title) and JS converts them to a <figure>
  • internal links do not work on the OSC books because I ran out of time and used <base href="">
  • Jekyll requires all Markdown that is converted to HTML must have a YAML header
  • I removed autogenerated ids on paragraphs and lists so the kramdown is cleaner (id="fs-id*")
Keep Reading...

Creating a "DSL" for GitHub's API (rewriting philschatz/octokit.js)

I originally wrote philschatz/octokit.js to interact with GitHub’s API using Promises. I started by forking an existing API michael/github and rewrote it using CoffeeScript and jQuery Promises. Then, thanks to contributions, they removed the dependence on Underscore and I removed the dependence on jQuery (and added support for several promise implementations).

But the code was getting too large as people incrementally added features so I thought it was time to rewrite with a focus on implementing as much of GitHub’s API concisely and consistently (with the hope of getting it adopted by GitHub officially).

Given my interest in Programming Languages, I decided to make a “Domain Specific Language” for GitHub (it’s in quotes because technically it’s still just Javascript but read on.

The results are at philschatz/octokat.js.

Necessary Features

This new library needed to:

  1. support ~100% of the GitHub API from the start
  2. have as little code as possible to maintain
  3. work in NodeJS and the browser
  4. have multiple source files
  5. have tons of unit tests
  6. support NodeJS callbacks and Promises

Let me quickly go through each of these and the challenges each presented.

Support ~100% of the GitHub API

This library abstracts all the authentication, status codes, caching, headers, HyperMedia (URL templates), and pagination returned from GitHub.

Abstracting Requests and Responses

Understanding how to form a request and how to parse the response is the bulk of the API; the logic is in src/ and src/

The other part is a chaining function that constructs a valid request.

Constructing Valid Requests

In the original philschatz/octokit.js there was a ton of copy/pasta dedicated to just constructing valid URLs.

Instead, this library has a regular expression that validates all URLs before calling GitHub and constructs objects dynamically through chaining (see src/

By reading the documentation at a developer implicitly constructs a URL and then issues a request by calling one of the verb methods.

For example, to list all comments on an issue convert the documentation URL to the following:

GET /repos/:owner/:repo/issues/:number/comments

    .repos(owner, repo).issues(number).comments.fetch()

Here are 3 ways to list all comments on an issue:

octo = new Octokat()

REPO = octo.repos('octokit/octokit.rb') # for brevity

# Option 1: using callbacks
REPO.issues(1).comments.fetch (err, comments) ->
  console.error(err) if err
  console.log(comments) unless err

# Option 2: using Promises
.then (comments) ->

# Option 3: using methods on the fetched Repository object
.then (repo) ->
  # `repo` contains returned JSON and additional methods
  .then (issue) ->
    # `repo` contains returned JSON and additional methods
    .then (comments) ->

There are several verb methods to choose from:

  • .fetch() sends a GET and yields an Object
  • .read() sends a GET and yields raw data
  • .create(...) sends a POST and yields an Object
  • .update(...) sends a PATCH and yields an Object
  • .remove() sends a DELETE and yields a boolean
  • .add(...) sends a PUT and yields a boolean
  • .contains(...) sends a GET and yields a boolean

Pagination and HTML templates

GitHub returns headers for lists of results. These are automatically converted to nextPage, previousPage, firstPage, and lastPage methods on the resulting JSON.

Paging through all the issues on a repository looks like this:

# Create an issues object (but does not fetch anything yet)
ISSUES = octo.repos('octokit/octokit.rb').issues

# Option 1: with callbacks
ISSUES.fetch (err, issues) ->
  issues.nextPage (err, moreIssues) ->

# Option 2: with Promises
.then (issues) ->
  .then (moreIssues) ->

Maintain a Minimal Amount of Code

With this API you are prevented from constructing an invalid URL because every request is validated against the URL_VALIDATOR regular expression before sending the request to GitHub.

Instead of writing largely copy/pasta code that constructs classes I opted for a regular expression that represents the entire GitHub API.

This can be found in src/

Works in NodeJS and the Browser

Getting this to work was a bit challenging.

NodeJS and AMD have slightly different syntaxes; enough to require adding in some boilerplate code to convert between the two.

Each of the source files contains 2 lines of boilerplate on the top of the file and the bottom.

Attempt 1

Originally, I had the entire library in a single file.

I was able to have very little boilerplate. Just something like the following:

@define ?= (cb) -> cb((dep) -> require(dep.replace('cs!', '')))

define (require) ->

  foo = require 'foo'


  module?.exports = Octokat # For NodeJS
  window?.Octokat = Octokat # For browsers
  return Octokat            # For browsers using AMD

This worked well but resulted in a single large file.

Attempt 2

I then split up the library into multiple files and got it working but ran into a problem when trying to build everything into one file.

I spent about a week trying to get jrburke/r.js and jrburke/almond to build a single file but could not get NodeJS, AMD, and the single file to all work at the same time.

Attempt 3

Finally, I opted for compiling the coffee files and concatenating them together with a custom define method similar to what other libraries do (see the require() method in less/less.js.

With this option, the tests run:

  • on the source coffee files in NodeJS
  • on the source coffee files in the browser
  • on the built dist/octokat.js file in the browser

Finally, multiple source files and tests running in all 3 environments!

Fixtures and Recording HTTP Requests

The library currently runs about 400 tests. There are about 120 unique tests, 80 are alternates using callbacks instead of promises, and 200 are the same tests but running in the browser.

In order to not pummel GitHub with duplicate requests I use linkedin/sepia to generate “cassettes” to replay the HTTP requests.

Unfortunately, sepia only runs on NodeJS so I wrote philschatz/sepia.js which plays back (and can record) linkedin/sepia tests.

Support NodeJS Callbacks and Promises

In order to support both callbacks and promises, the asynchronous methods (called verb methods) all support a function as the final argument and return a Promise.

This way, you can always end each line with a callback or with a .then() and the code just works.

The client only returns a Promise if one of the supported Promise libraries are detected (jQuery, angular, Q, or native Promises).

If you use one of those libraries but still prefer to use callbacks just make sure the callback is the last argument and the code will just workTM.

Keep Reading...