Maturin build hangs on a file lock.

[omrischwarz]

I'm not sure where to bring this, as it's a matter with the build tooling, but this is a problem that is only occurring on Maturin bulls for me.

I have an Ubuntu 22.04 server, running with rustup set to the default toolchain, (1.94) and Maturin installed as a pip package 1.12.6.

I am trying to build a wheel file for my project, and when I run maturin build --release --features python -v -v it stalls on the following step:

Fresh docopt v1.1.1 RunningCARGO=/home/ubuntu/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_CFG_FEATURE=default,pyo3,python CARGO_CFG_PANIC=unwind CARGO_CFG_TARGET_ABI='' CARGO_CFG_TARGET_ARCH=x86_64 CARGO_CFG_TARGET_ENDIAN=little CARGO_CFG_TARGET_ENV=gnu CARGO_CFG_TARGET_FAMILY=unix CARGO_CFG_TARGET_FEATURE=fxsr,sse,sse2 CARGO_CFG_TARGET_HAS_ATOMIC=16,32,64,8,ptr CARGO_CFG_TARGET_OS=linux CARGO_CFG_TARGET_POINTER_WIDTH=64 CARGO_CFG_TARGET_VENDOR=unknown CARGO_CFG_UNIX='' CARGO_ENCODED_RUSTFLAGS='' CARGO_FEATURE_DEFAULT=1 CARGO_FEATURE_PYO3=1 CARGO_FEATURE_PYTHON=1 CARGO_MANIFEST_DIR=/home/ubuntu/fos_scalability CARGO_MANIFEST_PATH=/home/ubuntu/fos_scalability/Cargo.toml CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='internal and confidential' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=spb_utils CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' DEBUG=false HOST=x86_64-unknown-linux-gnu LD_LIBRARY_PATH='/home/ubuntu/fos_scalability/target/release:/home/ubuntu/fos_scalability/target/release/deps:/home/ubuntu/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/ubuntu/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib' NUM_JOBS=32 OPT_LEVEL=3 OUT_DIR=/home/ubuntu/fos_scalability/target/release/build/spb_utils-b9e6337f915f1f9e/out PROFILE=release RUSTC=/home/ubuntu/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc RUSTDOC=/home/ubuntu/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustdoc TARGET=x86_64-unknown-linux-gnu /home/ubuntu/fos_scalability/target/release/build/spb_utils-e1042e0ddf5a6f6d/build-script-buildwarning:avro-rs(lib) generated 7 warnings warning:itertools` (lib) generated 6 warnings
Fresh sparkplug-rs v0.5.1
Building [=======================> ] 363/365: spb_utils(build)

`

Diggin in with ps:

ubuntu@enst005-array01:~/fos_scalability$ ps -faeH|grep -A4 maturin
ubuntu 1369969 1344272 0 23:22 pts/0 00:00:00 maturin build --release --features python -v -v
ubuntu 1370499 1369969 0 23:22 pts/0 00:00:00 /home/ubuntu/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo rustc --profile release --features python --message-format json-render-diagnostics -vv --manifest-path /home/ubuntu/fos_scalability/Cargo.toml --lib --crate-type cdylib
ubuntu 1371088 1370499 0 23:22 pts/0 00:00:00 /home/ubuntu/fos_scalability/target/release/build/spb_utils-e1042e0ddf5a6f6d/build-script-build
ubuntu 1371089 1371088 0 23:22 pts/0 00:00:00 maturin build --release --features python
ubuntu 1371220 1371089 0 23:22 pts/0 00:00:00 /home/ubuntu/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo rustc --profile release --features python --message-format json-render-diagnostics --manifest-path /home/ubuntu/fos_scalability/Cargo.toml --lib --crate-type cdylib
ubuntu 1380799 1344272 0 23:24 pts/0 00:00:00 ps -faeH
ubuntu 1380800 1344272 0 23:24 pts/0 00:00:00 grep --color=auto -A4 maturin

checking the bottom process with trace:

ubuntu@enst005-array01:~/fos_scalability$ sudo strace -fp 1371220
strace: Process 1371220 attached
flock(13, LOCK_EX^Cstrace: Process 1371220 detached
<detached ...>

And looking at its file handles shows that the flock call is on this file:

ubuntu@enst005-array01:~/fos_scalability$ ls -l /proc/1371220/fd
total 0
lr-x------ 1 ubuntu ubuntu 64 Mar 11 23:25 0 -> /dev/null
l-wx------ 1 ubuntu ubuntu 64 Mar 11 23:25 1 -> 'pipe:[11490144]'
lrwx------ 1 ubuntu ubuntu 64 Mar 11 23:25 10 -> /home/ubuntu/.cargo/.global-cache
lrwx------ 1 ubuntu ubuntu 64 Mar 11 23:25 11 -> 'socket:[11512439]'
lrwx------ 1 ubuntu ubuntu 64 Mar 11 23:25 12 -> /home/ubuntu/.cargo/.package-cache-mutate
lrwx------ 1 ubuntu ubuntu 64 Mar 11 23:25 13 -> /home/ubuntu/fos_scalability/target/release/.cargo-lock

I've made sure there were no other cargo, rustc , or rls processes running.
I've removed these lock files, and even rebooted, and replicated the build setup on another host, and all
came back to stopping at this particular point.

Running that Cargo invocation by hand also stops at that point. What should I proceed with to get past this problem?

[omrischwarz]

Out of curiosity I switched to nightly, and am hanging in the same spot:

ubuntu 1534833 1344272 0 23:43 pts/0 00:00:00 maturin build --release --features python -v -v -v
ubuntu 1535079 1534833 8 23:43 pts/0 00:00:03 /home/ubuntu/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo rustc --profile release --features python --message-format json-render-diagnostics -vvv --manifest-path /home/ubuntu/fos_scalability/Cargo.toml --lib --crate-type cdylib
ubuntu 1539222 1535079 0 23:43 pts/0 00:00:00 /home/ubuntu/fos_scalability/target/release/build/spb_utils-8326934a7e8ecd2e/build-script-build
ubuntu 1539228 1539222 0 23:43 pts/0 00:00:00 maturin build --release --features python
ubuntu 1539638 1539228 0 23:43 pts/0 00:00:00 /home/ubuntu/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo rustc --profile release --features python --message-format json-render-diagnostics --manifest-path /home/ubuntu/fos_scalability/Cargo.toml --lib --crate-type cdylib

ubuntu@enst005-array01:~/fos_scalability$ sudo strace -fp 1539638
strace: Process 1539638 attached
flock(13, LOCK_EX^Cstrace: Process 1539638 detached
<detached ...>

[messense]

.cargo-lock is managed by Cargo, we don't lock/unlock it. It's usually because some other processes are also try to build the project like rust-analyzer in your IDE.