Adding a Dependency Based on Autotools to a Bazel Project
In my spare time I have been experimenting with the Bazel build runner. I wanted to add Libsodium as a dependency to my Bazel C/C++ project. The Libsodium project is not a Bazel package and therefore Bazel will not know how to treat it as a dependency. If I wish to add Libsodium as a repository based dependency to my project, then It makes sense to write a BUILD file defining how Bazel should build the Libsodium library.
What you need to know
Rules_foreign_cc is an experimental Bazel extension with build rules for interfacing with non-Bazel build systems. By the time you read this post, the API may be many versions ahead and might not maintain backwards compatibility. With that in mind, the example contained within this post will target a fixed version of the API right before they removed backwards compatibility with Bazel v0.22.
This is not a tutorial on using Bazel to build C/C++ projects. If you are not familiar with the basics of building C/C++ projects with Bazel, then I recommend that you start with an Introduction to Bazel: Building a C++ Project before reading on. In fact, I recommend that you read through everything in the C++ and Bazel documentation, but that is not strictly necessary.
I would also like to point out that I have only tested this with Bazel v0.20.0 on a Gentoo system. If you are planning to try this from a system running Microsoft Windows, then you should install Bazel v0.23+. See Bazel versions compatibility for up to date compatibility information.
Initially I thought that I would have to write a bazel extension with custom build rules, but then I discovered the experimental rules_foreign_cc project after coming across this old Google Groups bazel-discuss thread. The rules_foreign_cc project contains a Bazel extension named rules_foreign_cc//:workspace_definitions.bzl that provides support for adding dependencies that require foreign toolchains. In fact, the workspace definitions extension has the CMake toolchain as a registered native default toolchain and is sufficient for building GNU configure/make projects like Libsodium. With that in mind, I will restrict my focus to what I needed to do to add Libsodium to my project so that the project builds successfully and Libsodium is able to be initialized at run-time.
The full source code is available at my BloggerBust/bazelLibsodiumTest GitHub repository. Before we get started, let’s take a brief look at the project’s layout:
tree ~/dev/bazelLibsodiumTest/ -I "bazel*"
/home/dustfinger/dev/bazelLibsodiumTest/ ├── BUILD ├── README.org ├── src │ ├── BUILD │ ├── lib │ │ ├── BUILD │ │ ├── TestLibsodium.cpp │ │ └── TestLibsodium.h │ └── main.cpp └── WORKSPACE 2 directories, 8 files
- a README in org-mode format explaining what the project is about
- a Bazel workspace where we will load the rules_foreign_cc workspace definitions extension
- a build file where we will load the rules_foreign_cc configure extension and provide parameters needed to build the Libsodium library.
- defines the main package and calls the built in cc_binary rule for building C/C++ binaries.
- This is the bazelLibsodiumTest entry point.
- defines the test_libsodium package. This depends on
- Initializes Libsodium and prints to the standard output a success or failure message
Load the rules_foreign_cc workspace definitions extension
On March 11, 2019 the rules_foreign_cc project removed support for Bazel version less than 0.22. The workspace file in my GitHub repository has a comment explaining how to set the rules_foreign_cc master branch as the dependency, but in this post I will only discuss setting the final commit that provided support for Bazel pre version 0.22, since many distributions do not include the latest Bazel version in their stable package tree.
Determine the sha256 hash
The purpose of the sha256 hash validation is to ensure that subsequent downloads from either the same or different environments are building exactly the same dependencies. It is more about build integrity than security. Please download the zip archive for commit
a3593905f73ce19c09d21f9968f1d3f5bc115157 and then calculate its sha256 hash. You can determine the URI of a commit using the pattern:
wget https://github.com/bazelbuild/rules_foreign_cc/archive/a3593905f73ce19c09d21f9968f1d3f5bc115157.zip sha256sum a3593905f73ce19c09d21f9968f1d3f5bc115157.zip rm a3593905f73ce19c09d21f9968f1d3f5bc115157.zip
Now we know that the sha256 hash is
6f3484eacc172c90d605e79130f9f01ec827a98b99c499c396eddc597a9c219d and with that we can configure the build to validate the hash before building the dependency.
It is important to understand that what I have just explained is a flawed protocol if my intention was to protect myself against a sophisticated man in the middle attack. Consider that if I download the archive and calculate the hash, then ask the build to download the same archive and compare its calculated hash to what I had previously calculated, of course I should expect the hashes to be the same. Following this protocol I should expect the hash to validate even if a hostile proxy was delivering a modified archive. To mitigate the threat of a man in the middle attack, the Libsodium project supports cryptographic integrity checking which I will not be covering in this post.
Add rules_foreign_cc as a dependency
When the archive is extracted the root directory will be the name of the archive, followed by a hyphen, followed by the name of the branch or commit. It is best to strip away this top level archive directory since it causes issues with target paths. We do this by setting strip_prefix to rules_foreign_cc-a3593905f73ce19c09d21f9968f1d3f5bc115157. Additionally, we want to set the sha256 hash of the archive. If the sha256 hash of the downloaded archive does not match the value of the sha256 attribute, then the build will be halted.
workspace(name = "com_github_bloggerbust_bazelLibsodiumTest") load('@bazel_tools//tools/build_defs/repo:http.bzl', 'http_archive') # for Bazel 0.22 and below use last supported commit http_archive( name = "rules_foreign_cc", strip_prefix = "rules_foreign_cc-a3593905f73ce19c09d21f9968f1d3f5bc115157", url = "https://github.com/bazelbuild/rules_foreign_cc/archive/a3593905f73ce19c09d21f9968f1d3f5bc115157.zip", sha256 = "6f3484eacc172c90d605e79130f9f01ec827a98b99c499c396eddc597a9c219d" )
To initialize rules_foreign_cc we must load the rules_foreign_cc_dependencies function from the workspace_definitions extension. Once loaded, we can immediately call the function without passing any arguments so that the registered default toolchain will be selected. Calling this function will also initiate necessary code generation needed for C++ and Starlark API support.
load("@rules_foreign_cc//:workspace_definitions.bzl", "rules_foreign_cc_dependencies") rules_foreign_cc_dependencies()
Configure the Libsodium dependency
The simplest way to ensure that we capture all of the source files that must be included in the Libsodium library is to create a file group using wildcard globs and assign it to the build_file_content attribute. I named the file group libsodium_all, but you can name it whatever you like. Use the same technique that we used under the subheading Determine the sha256 hash to determine the sha256 hash for the Libsodium archive.
all_content = """filegroup(name = "libsodium_all", srcs = glob(["**"]), visibility = ["//visibility:public"])""" http_archive( name = "org_libsodium", url = "https://github.com/jedisct1/libsodium/releases/download/1.0.17/libsodium-1.0.17.tar.gz", sha256 = "0cc3dae33e642cc187b5ceb467e0ad0e1b51dcba577de1190e9ffa17766ac2b1", strip_prefix = "libsodium-1.0.17", build_file_content = all_content )
Create the a BUILD file for Libsodium
Now it is time to configure make. We need to let make know the following:
- Where the library source code is for Libsodium: This is satisfied by setting the lib_source attribute to a label targeting the file group that we created under Configure the Libsodium dependency.
- The path to where the compiled library binaries will be written to following a successful build: This is satisfied by setting the out_lib_dir attribute to a directory relative to the libsodium_configure package. I simply named this directory
- The name of the resulting shared library that we would like bazelLibsodiumTest binary to link to: This is satisfied by setting the shared_libraries attribute. Alternatively you can specify a static library with the static_libraries attribute.
load("@rules_foreign_cc//tools/build_defs:configure.bzl", "configure_make") configure_make( name = "libsodium_configure", lib_source = "@org_libsodium//:libsodium_all", out_lib_dir = "lib", shared_libraries = ["libsodium.so.23"], visibility = ["//visibility:public"] )
Since the BUILD file is in the root of our project the label for the configure_make target will be
Create a BUILD file for our TestLibsodium library
To add the Libsodium library as a dependency of the TestLibsodium Library package, simply assign the
//:libsodium_configure label to the deps attribute.
cc_library( name = "test_libsodium", srcs = glob(["*.cpp"]), hdrs = glob(["*.h"]), deps = ["//:libsodium_configure"], visibility = ["//visibility:public"] )
Time to build and run
There are a few strange things in this build command that I think need some explanation:
- For security reasons I mount /tmp as non executable, therefore I must override the default path since Bazel executes the configure script from this location. You probably won’t need this part of the command.
- Required for Bazel 0.20 to 0.21
- I discovered that all the compiler output was sent to
stderr. This redirects
stdout. Perhaps this has been fixed in a more recent version of Bazel.
env TMPDIR=~/dev/tmp bazel build --experimental_cc_skylark_api_enabled_packages=@rules_foreign_cc//tools/build_defs,tools/build_defs,@foreign_cc_impl //src:main 2>&1
INFO: Invocation ID: b03a9e49-ba0f-46bb-a232-43cf82087212 Loading: Loading: 0 packages loaded Analyzing: target //src:main (0 packages loaded, 0 targets configured) INFO: Analysed target //src:main (0 packages loaded, 0 targets configured). INFO: Found 1 target... [0 / 1] [-----] BazelWorkspaceStatusAction stable-status.txt Target //src:main up-to-date: bazel-bin/src/main INFO: Elapsed time: 0.062s, Critical Path: 0.00s INFO: 0 processes. INFO: Build completed successfully, 1 total action INFO: Build completed successfully, 1 total action
Since the build was successful we should be able to run the main binary and see a message stating that Libsodium has been initialized.
bazel run --experimental_cc_skylark_api_enabled_packages=@rules_foreign_cc//tools/build_defs,tools/build_defs,@foreign_cc_impl //src:main
going to initialize Libsodium, wish me luck ////////////////////////////////////////////////////// // W00t! the Libsodium library has been initialized // //////////////////////////////////////////////////////
If anything goes wrong, then add
--verbose_failures --sandbox_debug optional flags to the build command.
Thanks to the rules_foreign_cc project adding Libsodium as a dependency was easy. For projects based on GNU configure/make Autotools that have foreign dependencies of their own, additional work would be needed. Also, keep in mind that the project is experimental and is unlikely to maintain backwards compatibility until it is deemed stable. I hope that you found this post useful.
Be the first to comment on this article.