Skip to content

update build to latest duckdb #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

update build to latest duckdb #27

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
36749df
update build to latest duckdb
Kayrnt Jul 29, 2023
6e25d88
Add missing changes for unsupported archs
Kayrnt Jul 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ BUILD_FLAGS=-DEXTENSION_STATIC_BUILD=1 -DBUILD_TPCH_EXTENSION=1 -DBUILD_PARQUET_
CLIENT_FLAGS :=

# These flags will make DuckDB build the extension
EXTENSION_FLAGS=-DDUCKDB_OOT_EXTENSION_NAMES="odbc_scanner" -DDUCKDB_OOT_EXTENSION_ODBC_SCANNER_PATH="$(PROJ_DIR)" -DDUCKDB_OOT_EXTENSION_ODBC_SCANNER_SHOULD_LINK="TRUE" -DDUCKDB_OOT_EXTENSION_ODBC_SCANNER_INCLUDE_PATH="$(PROJ_DIR)src/include"
EXT_NAME="odbc_scanner"
EXT_NAME_UPPERCASE=$(shell echo $(EXT_NAME) | tr '[:lower:]' '[:upper:]')
EXTENSION_FLAGS=-DDUCKDB_EXTENSION_NAMES=${EXT_NAME} -DDUCKDB_EXTENSION_${EXT_NAME_UPPERCASE}_PATH="$(PROJ_DIR)" -DDUCKDB_EXTENSION_${EXT_NAME_UPPERCASE}_SHOULD_LINK="TRUE" -DDUCKDB_EXTENSION_${EXT_NAME_UPPERCASE}_LOAD_TESTS="FALSE" -DDUCKDB_EXTENSION_${EXT_NAME_UPPERCASE}_INCLUDE_PATH="$(PROJ_DIR)src/include"
Comment on lines +26 to +28
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I extracted the odbc_scanner string to reuse it the way DuckDB parse it in its CMake
The actual changes are:

  • DDUCKDB_OOT_EXTENSION -> DDUCKDB_EXTENSION
  • new required flag -DDUCKDB_EXTENSION_${EXT_NAME_UPPERCASE}_LOAD_TESTS="FALSE" (I don't know we want it false or not, but at least it works)


pull:
git submodule init
Expand All @@ -36,13 +38,13 @@ clean:

# Main build
debug:
mkdir -p build/debug && \
cmake $(GENERATOR) $(FORCE_COLOR) $(EXTENSION_FLAGS) ${CLIENT_FLAGS} -DEXTENSION_STATIC_BUILD=1 -DCMAKE_BUILD_TYPE=Debug ${BUILD_FLAGS} -S ./duckdb/ -B build/debug && \
mkdir -p build/debug && \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-DEXTENSION_STATIC_BUILD=1 is already in the ${BUILD_FLAGS} so it's just a clean up

cmake $(GENERATOR) $(FORCE_COLOR) $(EXTENSION_FLAGS) ${CLIENT_FLAGS} -DCMAKE_BUILD_TYPE=Debug ${BUILD_FLAGS} -S ./duckdb/ -B build/debug && \
cmake --build build/debug --config Debug

release:
mkdir -p build/release && \
cmake $(GENERATOR) $(FORCE_COLOR) $(EXTENSION_FLAGS) ${CLIENT_FLAGS} -DEXTENSION_STATIC_BUILD=1 -DCMAKE_BUILD_TYPE=Release ${BUILD_FLAGS} -S ./duckdb/ -B build/release && \
cmake $(GENERATOR) $(FORCE_COLOR) $(EXTENSION_FLAGS) ${CLIENT_FLAGS} -DCMAKE_BUILD_TYPE=Release ${BUILD_FLAGS} -S ./duckdb/ -B build/release && \
cmake --build build/release --config Release

# Client build
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ to the correct version of `unixodbc`.

```shell
nix run .#build
./build/release/duckdb
./build/release/duckdb -unsigned
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not required but I thought it was relevant as we are going to load the unsigned extension later on

```

To use ODBC DSN's with driver paths managed by the `odbc-drivers-nix` flake run the generate nix apps.
Expand Down
2 changes: 1 addition & 1 deletion duckdb
Open in desktop
Submodule duckdb updated 1771 files
19 changes: 11 additions & 8 deletions flake.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,17 @@
stdenv = pkgs.llvmPackages_15.stdenv;

# Define a helper function to conditionally include the db2-odbc-driver package as not all arch are supported
getDb2OdbcDriver =
if system == "aarch64-linux" || system == "aarch64-darwin" then null else pkgs.db2-odbc-driver {};
isAarch64 = system == "aarch64-linux" || system == "aarch64-darwin";

getDb2OdbcDriver = if isAarch64 then null else pkgs.db2-odbc-driver;
getDb2OdbcDriverLibFile = if stdenv.isDarwin then "libdb2.dylib" else "libdb2.so";
getDb2OdbcDriverPath = if isAarch64 then "" else "DB2_DRIVER_PATH=${getDb2OdbcDriver}/lib/${getDb2OdbcDriverLibFile}";


in rec {
# packages exported by the flake
packages = {
db2-odbc-driver = getDb2OdbcDriver;
db2-odbc-driver = if getDb2OdbcDriver != null then getDb2OdbcDriver {} else null;
postgres-odbc-driver = pkgs.postgres-odbc-driver {};
};

Expand Down Expand Up @@ -65,15 +68,15 @@
generate-odbcinst-ini = {
type = "app";
program = toString (pkgs.writeScript "generate-odbcinst-ini" ''
DB2_DRIVER_PATH=${packages.db2-odbc-driver}/lib/${if stdenv.isDarwin then "libdb2.dylib" else "libdb2.so"} \
${getDb2OdbcDriverPath} \
POSTGRES_DRIVER_PATH=${packages.postgres-odbc-driver}/lib/psqlodbca.so \
envsubst < ./templates/.odbcinst.ini.template > .odbcinst.ini
'');
};
ls-odbc-driver-paths = {
type = "app";
program = toString (pkgs.writeScript "ls-odbc-driver-paths" ''
echo "db2 ${packages.db2-odbc-driver}/lib/${if stdenv.isDarwin then "libdb2.dylib" else "libdb2.so"}"
${if isAarch64 then "" else "echo \"db2 ${getDb2OdbcDriverPath}"}
echo "postgres ${packages.postgres-odbc-driver}/lib/psqlodbca.so"
'');
};
Expand All @@ -93,7 +96,7 @@
cmake
ninja
openssl
packages.db2-odbc-driver
getDb2OdbcDriver
packages.postgres-odbc-driver
]
)}:$PATH"
Expand All @@ -118,7 +121,7 @@
cmake
ninja
openssl
packages.db2-odbc-driver
getDb2OdbcDriver
packages.postgres-odbc-driver
]
)}:$PATH"
Expand Down Expand Up @@ -152,7 +155,7 @@
pkgs.unixODBC
# psql cli
pkgs.postgresql_15
packages.db2-odbc-driver
getDb2OdbcDriver
packages.postgres-odbc-driver
];
};
Expand Down
2 changes: 1 addition & 1 deletion src/include/odbc_scanner_extension.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#include "duckdb.hpp"

namespace duckdb {
class Odbc_scannerExtension : public Extension {
class OdbcScannerExtension : public Extension {
public:
void Load(DuckDB &db) override;
std::string Name() override;
Expand Down
4 changes: 2 additions & 2 deletions src/odbc_scanner_extension.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ static void LoadInternal(DatabaseInstance &instance) {
con.Commit();
}

void Odbc_scannerExtension::Load(DuckDB &db) { LoadInternal(*db.instance); }
std::string Odbc_scannerExtension::Name() { return "odbc_scanner"; }
void OdbcScannerExtension::Load(DuckDB &db) { LoadInternal(*db.instance); }
std::string OdbcScannerExtension::Name() { return "odbc_scanner"; }
} // namespace duckdb

extern "C" {
Expand Down