Staticman with GitHub and Zeit Now - part 2

A quick recap

In part 1 of this series we rationalized our choice to use Staticman as a comment engine for Blogger Bust, and Zeit Now to host Staticman. We also discussed how to create a GitHub bot account and began the process of authorizing the Zeit Now GitHub app, but we ran into some trouble when our account was flagged by the Zeit abuse system. After answering a few of their questions they whitelisted our account, a process that takes 24 hours to propagate through their system. We may now continue our quest to setup Staticman via Zeit Now.

Re-register with Zeit Now

Hopefully enough time has passed for registration to complete successfully. The first step is to sign in to GitHub using your GitHub bot account that we created in part 1 of this series. Next, revisit the Zeit Now GitHub setup page and click CONTINUE WITH GITHUB. For me, the page redirected to itself. I decided to try their sign-in page instead. After clicking the CONTINUE WITH GITHUB link I was presented with a view containing the text “Are you BloggerBust-bot?:”. Since I am, I clicked the CONTINUE AS BloggerBust-bot link and I was finally signed in to their dashboard.

If you ever want to revoke the Zeit Now app from your GitHub bot account you can visit the Authorized GitHub Apps view and click the Revoke link.

Install Now for GitHub

Please refer to the official Zeit Now documentation to install Now for GitHub if you have any trouble following these instructions. From your Zeit Account Settings view find the card with the heading GitHub Integration and click the INSTALL NOW FOR GITHUB link. You will then be prompted to pick between authorizing the Zeit Now GitHub app for all repositories or limit authorization to a whitelist. The choice is yours to make. I decided to grant the Now app access to a whitelist of repositories despite the fact that, at the time of this writing, this account has but a single repository. Once you have made your choice, read over the permissions that it is requesting and if you are okay granting it those permissions then go ahead and click the Install button at the bottom of the view.

Later, if you decide that you want to uninstall the Zeit Now GitHub account, simply visit your GitHub account’s Installed GitHub Apps view, then click the Configure button followed by the Uninstall button that will be visible on the app’s configuration view. From the configuration view you can also review the permissions that the app has been granted and update the repository access settings.

Install the Now CLI

The Now CLI is open source on GitHub. Please follow their instructions to install the CLI. If you choose to install the CLI with npm then you must have Node.js installed. Alternatively, you may choose to install the Now Desktop which includes the Now CLI. I choose not to install “desktop” applications of this nature because I do not like how tightly they integrate my development environment with the provider’s service infrastructure.

Install npm

If you installed the Now CLI with the Now Desktop app then you can technically skip this section. Otherwise, you will need to install npm to install the Now CLI. Even if you did install the Now Desktop app, you may want to be able to contribute to Staticman.net, or just run the unit tests as a sanity check, in which case you should install npm.

Npm is the default JavaScript package manager for Node.js. Staticman.net was designed to be hosted by Node.js. Zeit Now was designed to host serverless Node applications. My preference is to install npm with a custom prefix so that I can avoid installing global packages in a location that would require root privileges.

mkdir ~/.npm-global
npm config set prefix "~/.npm-global"
npm install now -g

/home/dustfinger/.npm-global/bin/now -> /home/dustfinger/.npm-global/lib64/node_modules/now/download/dist/now

> now@14.2.4 postinstall /home/dustfinger/.npm-global/lib64/node_modules/now
> node download/install.js

> For the source code, check out: https://github.com/zeit/now-cli



+ now@14.2.4
added 2 packages from 2 contributors in 10.423s

If you choose to change your npm global prefix then you will also need to update your path environment variable to include ~/.npm-global/bin or you won’t be able to run now without providing the path to the binary. I simply add the following to my ~/.bash_profile

PATH=$PATH:~/.npm-global/bin

and then source it.

source ~/.bash_profile

Generate RSA PEM Private & Public Keys

Staticman uses a PEM encoded RSA private key to decrypt a select set of sensitive configuration parameters using the node-rsa npm package. The private key must be generated using one of the format schemes supported by node-rsa. The corresponding public key is kept in a file named staticman_key.pub and is checked into the Git repository. The public key is meant to be used by the end user to encrypt sensitive parameters that need to be stored in the staticman.yml of a pregenerated GitHub or GitLab repository. As you will soon see, I am not going to be using this approach. None the less, I will show you how to generate both the public and private RSA keys. The git ignore rules exclude a file named staticman_key, so that seems like a good place to store our private key for later use. Let’s create an RSA private key for this purpose now. Remember – Keep it secret, keep it safe.

openssl genpkey -outform PEM -algorithm RSA > staticman_key
openssl pkey -inform PEM -in staticman_key -pubout > staticman_key.pub

Keep our secrets safe for Now

The Staticman Readme - Setting up the server documentation instructs us to copy the config.sample.json, to config.{environment}.json, as a starting place when configuring the Staticman server. However, Now GitHub deployments are immutable, therefore we would be required to push a commit containing our config.production.json. Under these circumstances, if we thoughtlessly followed instructions, then our GitHub bot repository would contain our GitHub token and private RSA key within the config.production.json resulting in a fallible deployment strategy. I have decided to improve security by taking advantage of Zeit Now’s support for securing environment variables using secrets. It so happens that the githubToken and rsaPrivateKey config properties have corresponding environment variables. Moreover, all of the config properties correspond to an environment variable, so we could get rid of config.production.json entirely. In fact, that is what we are going to do.

Sign into Now using the CLI

The Now CLI requires an authentication token in order to operate on your account. To that end, we simply call the login verb from a terminal.

now login

The protocol will instruct the Zeit Now service to send you a confirmation email. The email will contain a link that when opened in your browser will present a CAPTCHA. I hate CAPTCHAs. As soon as you answer the CAPTCHA, the Now CLI will store the authentication token.

For your amusement I will tell you that I had opened this CAPTCHA in a separate EXWM workspace and did not notice when the Now CLI had validated the email confirmation. As a result, I answered several CAPTCHAs before I got annoyed and flipped back to the workspace containing my terminal with the CLI output, only to find that it had already validated.

> We sent an email to trevor.wilson+bot@bloggerbust.ca. Please follow the steps provided
  inside it and make sure the security code matches Handsome Snowshoe.
✔ Email confirmed
> Ready! Authentication token and personal details saved in "~/.now"

Be warned, the CAPTCHA may continue to present you with new CAPTCHAs without end. Although, on subsequent trials I was instead presented with the following confirmation message directly in the browser immediately after passing the CAPTCHA. I guess the behaviour may vary:

Create the RSA private key secret

After looking over the node-rsa Import/Export keys example I was under the impression that end-of-line delimiters need to be removed from the key data. In retrospective, removing end-of-line delimiters is probably not necessary, but since I have not tested it otherwise I will leave the bash pipeline transformation as is. For convenience we will use the tr (translate) command to delete carriage returns and newline characters, then pipe the result to xargs which will invoke sh with our private key as a positional argument. The sh command will read our now secret command from a string, perform positional parameter substitution using the parameters passed to it by xargs and evaluate the resulting expression. The final product will be a Now Secret, named staticman-rsa-private-key, saved in your Now account and readable only by the code that is running it.

tr -d ['\r','\n'] < staticman_key | xargs -0 sh -c 'now secret add -- staticman-rsa-private-key "$0"'

> UPDATE AVAILABLE The latest version of Now CLI is 15.0.1
> Read more about how to update here: https://zeit.co/update-cli
> Changelog: https://github.com/zeit/now-cli/releases/tag/15.0.1
> Success! Secret staticman-rsa-private-key added (trevorwilsonbot) [289ms]

Create the GitHub token secret

Copy the GitHub access token that you created for Staticman and add it as a Now secret just as we did with the RSA private key.

now secret add staticman-github-token "6GhMuCvs9AXS3XCxxozUSIYGwuTKTHdUFcHYIm2d"

> Success! Secret staticman-github-token added (trevorwilsonbot) [290ms]

Prepare Staticman for Now

In part 1 of this series we cloned the staticman repo. However, I found some issues with the Staticman master branch and have since made a few pull requests – PR #285, PR #288, PR #289. For now, you might want to just clone my working branch until upstream stabilizes. Or fork it and then switch to your forked copy of my working branch if you wish to take advantage of automatic serverless deployments.

Configuration for Now

The Now GitHub app requires a now.json configuration file to exist in the root directory of any app that we wish to configure for automatic serverless deployment. It is in the Now configuration that we can set Staticman environment variables to either hard coded values or our Now secrets as discussed in Keep our secrets safe for Now. Let’s create that configuration file now.

{
  "version": 2,
  "env": {
    "NODE_ENV": "development",
    "GITHUB_TOKEN": "@staticman-github-token",
    "RSA_PRIVATE_KEY": "@staticman-rsa-private-key"
  },
  "builds": [
    { "src": "index.js", "use": "@now/node-server" }
  ],
  "routes": [
    { "src": "/(.*)", "dest": "/index.js" }
  ]
}

The point of serverless hosting is to host one lambda per route so that server resource utilization is minimized. It also frees up developers from having to concern themselves with server-side resource management. If you have been sitting there this whole time with your hand up, squirming in your seat, desperately wanting to point out that Staticman has server code and isn’t designed for serverless, then you may put your hand down now. You are correct – sort of.

Zeit Now provides the @now/node-server official builder for Node.js applications that manage their own server-side resources. If you thought that Staticman is not designed for serverless, then you are right, but if you thought that our deployment of Staticman is not serverless, then you are wrong. You see, Staticman uses Express web framework which the @now/node-server builder then wraps in a single lambda named index.js. This is why we route everything to index.js in the /routes property of the now.json pasted above. As soon as the route has been resolved to the one and only lambda, Staticman Express routing takes over and handles application level routing internally.

If you are curious what else you can add to the now.json configuration, then on your quest for more details, I refer you to the Now Deployment Configuration documentation. You might also be interested in this article on customizable lambda sizes.

Ignore things for Now

Just like Git, Zeit Now has an ignore file called .nowignore. The syntax is identical to .gitignore. It is best to ignore everything that is being ignored in .gitignore. You wouldn’t want to accidentally deploy your staticman_key to be served up publicly now would you. Also, the node_modules directory can be quite large, it would be nice to not have that as part of the deployment. npm can install the node modules on the Zeit Now instance as part of the build procedure.

cp .gitignore .nowignore

Let’s edit our .nowignore to ignore a few more things explicitly.

.idea/
*.iml

config.json
config.*.json
!config.test.json
!config.example.json
coverage/
node_modules/
staticman_key
test/
docs/
**/*.md
!LICENSE.md
Dockerfile

Deploy our Staticman instance

IMPORTANT: As I mentioned in Prepare Staticman for Now I found some issues in the upstream master branch. I have a working branch named now_master in my fork of the Staticman repository that has already been prepared for Now deployments. I recommend that you use my working branch until the upstream master branch has stabilized.

Once I felt that my fork of Staticman was ready for deployment I pushed my changes to my remote expecting the GitHub Now App to automatically deploy my modified Staticman app and provide me with the details. That did not seem to happen though. I looked around both my GitHub and Now dashboards for a while. After trying quite a few different things I finally resolved myself to just use the now CLI and deploy it manually.

now

> UPDATE AVAILABLE The latest version of Now CLI is 15.0.1
> Read more about how to update here: https://zeit.co/update-cli
> Changelog: https://github.com/zeit/now-cli/releases/tag/15.0.1
> Deploying ~/dev/staticman under bloggerbust-bot
> Using project staticman
> https://staticman-ol2ejawgs.now.sh [v2] [2s]
┌ index.js        Ready               [1m]
└── λ index.js (12.36MB) [sfo1]
> Ready! Aliased to https://staticman.bloggerbust-bot.now.sh [1m]

That worked! I will mess around with automatic serverless deployment using the Now GitHub app another time. For now, I am happy to manually deploy.

Conclusion

That covers the basis of what you need to do to setup Staticman with Zeit Now. In the third and final part of this series I will talk a about how I configured Blogger Bust to interopt with my instance of Staticman the end result being functional static comments on all of my posts.