Installing ClickHouse on an M1 Mac
Posted on Tue 18 January 2022 in Development • 5 min read
Update: Nov 2023 -- none of this is necessary any longer, as ClickHouse now provides binaries for ARM devices.
Soon after the release of the M1 Pro and M1 Max processors and the upgrade to the physical hardware of the MacBooks, I finally made the decision to upgrade.
I've suffered through several years of using a 2017 15" Macbook Pro, with the godawful thermals and even more egregious keyboard and after a year on the market, it appeared to me that the majority of the software that I use for work had been upgraded to support the M1s.
After only about 3 months of ownership, I would say that I was completely mistaken, and would've been far better served by simply moving back to an X86-64 laptop. I'm quite disappointed with the amount of incorrect information out there, stating that for a developer, it's already quite safe to make the migration to the M1s, as they simply are in no way shape or form, ready for getting real work done.
The major things--yes, they're reasonable well supported. Docker runs, my IDEs run, but it's always the details.
My old i7 2017 was a productivity killer, this machine is worse yet.
All that said, I'm unwilling to make the move back to my i7 at this point, so suffering forward, at work currently I've been working on a project to move some of our analytics data storage over from PostgreSQL to ClickHouse. This post won't go into the reasons behind that, it's just going to focus on getting setup with ClickHouse on an M1 Mac.
Docker
At first glance, this would appear to be the obvious choice, however the official CLickHouse docker images simply do not support ARM64 CPUs.
djcurtis@djc-mpb ~ % docker pull yandex/clickhouse-server
Using default tag: latest
latest: Pulling from yandex/clickhouse-server
7b1a6ab2e44d: Already exists
7ef9fa67d54f: Pull complete
8f815171d59f: Pull complete
592cc7ab07c7: Pull complete
4aecf5861a1e: Pull complete
75282093810f: Pull complete
37f33ec685ad: Pull complete
Digest: sha256:7d58af905fef789288892f1eac0d570ac23ef4c9b326baf684f202ac2894af09
Status: Downloaded newer image for yandex/clickhouse-server:latest
docker.io/yandex/clickhouse-server:latest
djcurtis@djc-mpb ~ % docker run --name nope yandex/clickhouse-server
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
runtime: failed to create new OS thread (have 2 already; errno=22)
fatal error: newosproc
runtime stack:
runtime.throw(0x4cb21f, 0x9)
/usr/local/go/src/runtime/panic.go:566 +0x95
runtime.newosproc(0xc420028000, 0xc420037fc0)
/usr/local/go/src/runtime/os_linux.go:160 +0x194
runtime.newm(0x4d6db8, 0x0)
/usr/local/go/src/runtime/proc.go:1572 +0x132
runtime.main.func1()
/usr/local/go/src/runtime/proc.go:126 +0x36
runtime.systemstack(0x53ae00)
/usr/local/go/src/runtime/asm_amd64.s:298 +0x79
runtime.mstart()
/usr/local/go/src/runtime/proc.go:1079
goroutine 1 [running]:
runtime.systemstack_switch()
/usr/local/go/src/runtime/asm_amd64.s:252 fp=0xc420022768 sp=0xc420022760
runtime.main()
/usr/local/go/src/runtime/proc.go:127 +0x6c fp=0xc4200227c0 sp=0xc420022768
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2086 +0x1 fp=0xc4200227c8 sp=0xc4200227c0
Configuration file '/etc/clickhouse-server/config.xml' isn't readable by user with id '101'
Looking at "WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8)", perhaps naively, I had some hope that setting the platform flag in docker, would provide an easy solution here.
djcurtis@djc-mpb ~ % docker run --name still-nope --platform linux/amd64 yandex/clickhouse-server
runtime: failed to create new OS thread (have 2 already; errno=22)
fatal error: newosproc
runtime stack:
runtime.throw(0x4cb21f, 0x9)
/usr/local/go/src/runtime/panic.go:566 +0x95
runtime.newosproc(0xc420028000, 0xc420037fc0)
/usr/local/go/src/runtime/os_linux.go:160 +0x194
runtime.newm(0x4d6db8, 0x0)
/usr/local/go/src/runtime/proc.go:1572 +0x132
runtime.main.func1()
/usr/local/go/src/runtime/proc.go:126 +0x36
runtime.systemstack(0x53ae00)
/usr/local/go/src/runtime/asm_amd64.s:298 +0x79
runtime.mstart()
/usr/local/go/src/runtime/proc.go:1079
goroutine 1 [running]:
runtime.systemstack_switch()
/usr/local/go/src/runtime/asm_amd64.s:252 fp=0xc420022768 sp=0xc420022760
runtime.main()
/usr/local/go/src/runtime/proc.go:127 +0x6c fp=0xc4200227c0 sp=0xc420022768
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2086 +0x1 fp=0xc4200227c8 sp=0xc4200227c0
Configuration file '/etc/clickhouse-server/config.xml' isn't readable by user with id '101'
Run the Alpine Image
I have far better things to do with my day than figure out why this works -- but simply running the alpine image (for god only knows why) seems to work a treat.
docker run yandex/clickhouse-server:21.11.10.1-alpine
Create A Multi-Arch Manifest
Before discovering that the Alpine image just magically works, I settled on using an alternative clickhouse image that works. My problem being (and this is just a local dev setup) that we have multiple team members working on this project, and I want all the members of the team (mostly who are using X86-64 boxes) to be able to develop against a docker compose file that would also bring up some supporting services for our app.
This is a local development box -- so security is not a consideration, but ease of use is.
Directory Structure
/Dockerfile.arm64-base /Dockerfile.x86_64 /users.xml /config.xml
The contents of both dockerfiles are listed below, and as you can see, they're both extremely simple. We're just getting the appropriate base image, and then including 2 configuration files for clickhouse.
The actual contents of users.xml and config.xml aren't important here -- they're just configured for very easy local development, with some of the appropriate settings that we need locally.
FROM --platform=linux/arm64/v8 lunalabsltd/clickhouse-server:21.7.2.7-arm
COPY ./users.xml /etc/clickhouse-server/users.xml
COPY ./config.xml /etc/clickhouse-server/config.xml
FROM --platform=linux/amd64 yandex/clickhouse-server:latest
COPY ./users.xml /etc/clickhouse-server/users.xml
COPY ./config.xml /etc/clickhouse-server/config.xml
Finally, we need to create the manifest and publish it to dockerhub, so that we can share a single image that will be available to our team members, regardless of which platform they're developing on.
djcurtis@djc-mpb ~ % docker build \
-t myworkplace/clickhouse-dev:arm8-latest \
-f clickhouse/Dockerfile.arm64-base ./clickhouse
djcurtis@djc-mpb ~ % docker build \
-t myworkplace/clickhouse-dev:x86-latest \
-f clickhouse/Dockerfile.x86_64 ./clickhouse
djcurtis@djc-mpb ~ % docker push myworkplace/clickhouse-dev --all-tags
djcurtis@djc-mpb ~ % docker manifest create \
myworkplace/clickhouse-dev:latest \
--amend myworkplace/clickhouse-dev:arm8-latest \
--amend myworkplace/clickhouse-dev:x86-latest
djcurtis@djc-mpb ~ % docker manifest push myworkplace/clickhouse-dev:latest
The final step is just to include our multiarch image into some
Local Installation
Installing Clickhouse on MacOS ARM64.
This approach is simpler, as Yandex has a prebuild binary available specifically for the M1 Mac, but there is one tiny gotcha in regards to Mac OS's system integrity protection.
djcurtis@djc-mbp ~ % sudo su -
djc-mbp:~ root# cd /opt
djc-mbp:opt root# curl -O 'https://builds.clickhouse.com/master/macos-aarch64/clickhouse' \
&& chmod a+x ./clickhouse
djc-mbp:opt root# ./clickhouse install
Copying ClickHouse binary to /usr/bin/clickhouse.new
Code: 76. DB::ErrnoException: Cannot open file /usr/bin/clickhouse.new, errno: 1, strerror: Operation not permitted. (CANNOT_OPEN_FILE) (version 22.1.1.2545)
Now, the error is because Apple doesn't allow anything to be written to /usr/bin by the user, even if you've authorized with root privileges (because.. reasons?). A bit frustrating, if you're expecing that root privileges actually means you have root privileges.
Basically, you have 2 options at this point -- you can temporarily disable system integrity protection, complete the install, and then reenable it. This is probably the more advisable root, however it does require doing at least one reboot, which I wasn't quite willing do to in the middle of my working day. Fortunately, both the installation script and the server start script for clickhouse allow you to specify an alternative location.
djc-mbp:opt root# ./clickhouse install --binary-path=/usr/local/bin
ClickHouse has been successfully installed.
Start clickhouse-server with:
sudo clickhouse start
Start clickhouse-client with:
clickhouse-client --password
And finally, we can start the server -- however be warned, you ALSO must specify the binary-path flag when starting clickhouse's server (a bit of an oversight on clickhouse's part.. but not that inconvenient).
djc-mbp:opt root# clickhouse start --binary-path=/usr/local/bin
And with that, you should have functioning clickhouse on your m1 machine. Hopefully, this will be improved at some point in the future.