PG16 Issues with Timescale Toolkit

[jon-sully]

Howdy! I've had a hell of a couple of days trying to figure out what all is going on here, and I don't know that I'd even say I have successfully but I at least wanted to post back some notes.

So 3727aa7 bumped the Postgres version for TimescaleDB itself up to 16. 5f635cd quickly followed since the extension gets compiled as .dylib now, not a .so, and I think those layers work fine — installing Timescale DB via Homebrew on a fresh Homebrew Postgres (16) works fine.

Things do not work when you try to install the Toolkit after that. For starters, the Toolkit formula references PG15 instead of 16 in multiple places:

• homebrew-tap/timescaledb-toolkit.rb

  Line 10 in 5f635cd

  depends_on "postgresql@15"

• homebrew-tap/timescaledb-toolkit.rb

  Line 13 in 5f635cd

  Formula["postgresql@15"]

• homebrew-tap/timescaledb-toolkit.rb

  Line 27 in 5f635cd

  system "cargo", "pgrx", "init", "--pg15", "pg_config"

• homebrew-tap/timescaledb-toolkit.rb

  Line 33 in 5f635cd

  system "cargo", "run", "--bin", "post-install", "--", "--dir", "target/release/timescaledb_toolkit-pg15"

And hard-codes for the .so file extension here.

Though perhaps more problematically, the post-install script for the toolkit itself (not this repo) still looks for .so here and here

All of essentially prevent a new install of the Toolkit from running / working at all. 🙁

[jon-sully]

Oh, and I think the formula needs to target toolkit v1.18.0 now instead of 1.17.0 too

homebrew-tap/timescaledb-toolkit.rb

Line 4 in 5f635cd

url "https://github.com/timescale/timescaledb-toolkit/archive/refs/tags/1.17.0.tar.gz"

[jon-sully]

In case anybody needs to get timescale toolkit installed in the meantime, here's the absolutely insane way I went about it.

First, open up the Homebrew formula (should be /opt/homebrew/Library/Taps/timescale/homebrew-tap/timescaledb-toolkit.rb) and edit the url on line 4 and sha256 on line 5 to:

url "https://github.com/timescale/timescaledb-toolkit/archive/refs/tags/1.18.0.tar.gz"
sha256 "f876fd8bdd5516237378e9c7556fec13740f5c8021911b1a8ceb8b1bde4c9cac"

Next, run brew install --interactive timescaledb-toolkit. It will yield a new shell session.

Now we start manually running the commands that the formula would (essentially) do anyway, just with correct version numbers and such.

curl -L https://github.com/pgcentralfoundation/pgrx/archive/refs/tags/v0.10.2.tar.gz -o cargo-pgrx.tar.gz
# then
mkdir pgrx
# then 
tar -xzf cargo-pgrx.tar.gz -C pgrx
# then 
cargo install --locked --path pgrx/pgrx-0.10.2/cargo-pgrx
# then 
cargo pgrx init --pg16 pg_config
# then
cd extension
# then 
cargo pgrx package
# then 
cd ..

Then we have to open up the post-install script from the toolkit code itself and edit it. However you prefer, but for VS code it's open -a Visual\ Studio\ Code tools/post-install/src/main.rs

We'll edit line #119 from "so" to "dylib" and lines 192 and 193 both from ".so" to ".dylib". Then save and close the file. We'll now run that post-install script:

cargo run --bin post-install -- --dir target/release/timescaledb_toolkit-pg16

Now, since the brew shell wouldn't let me move the actual extension files into the PG directory (🙄), the next step is to get your current tmp file path:

pwd
#=> /private/tmp/timescaledb-toolkit-20231207-79814-3k8arq

Then open a new shell of your own (with your normal user access, not one spun up by Homebrew for you) separate from the brew shell. DO NOT close or exit the brew shell yet. If you do, Brew will automatically delete all files (clean up) the /tmp directory it gave you. We still need those files.

So, open up a new shell / console in a new tab / pane / etc. and let's manually copy the extension files from the /tmp subdirectory they were just built in, into our Homebrew Postgres installation. Your version numbers may vary (so double check) but there are three commands:

# cp <from> <to>
cp /private/tmp/timescaledb-toolkit-20231207-79814-3k8arq/timescaledb-toolkit-1.18.0/target/release/timescaledb_toolkit-pg16/opt/homebrew/opt/postgresql@16/lib/postgresql/timescaledb_toolkit-1.18.0.dylib /opt/homebrew/Cellar/postgresql@16/16.1_1/lib/postgresql/

cp /private/tmp/timescaledb-toolkit-20231207-79814-3k8arq/timescaledb-toolkit-1.18.0/target/release/timescaledb_toolkit-pg16/opt/homebrew/opt/postgresql@16/share/postgresql@16/extension/timescaledb_toolkit--1.18.0.sql /opt/homebrew/Cellar/postgresql@16/16.1_1/share/postgresql@16/extension

cp /private/tmp/timescaledb-toolkit-20231207-79814-3k8arq/timescaledb-toolkit-1.18.0/target/release/timescaledb_toolkit-pg16/opt/homebrew/opt/postgresql@16/share/postgresql@16/extension/timescaledb_toolkit.control /opt/homebrew/Cellar/postgresql@16/16.1_1/share/postgresql@16/extension

Then we can close the Homebrew console session (which will automatically clean up the /tmp directory) by firing exit (Homebrew will say nothing was installed; that is fine)

[jonatas]

Hey @jon-sully, thanks for documenting all your efforts here. It can be very useful for upgrading the actual instructions.

Also, I can say I face the same as a Mac user and as an alternative I'm using the timescaledb-ha docker image that contains the toolkit on pg16.