DISCLAIMER: these notes reflect my knowledge and understanding of subjects, not the ultimate truth. Most are written in process of learning things. Corrections are welcome
Runtime code sharing in Rust
📅 2025-12-22 📄 source
Are we dynamic linking shared libraries yet? Or should we?
Split from 2025-03-07_sdl3-image-viewer.html
Software is modular and modules are reused all the time, making it desirable to share them in runtime between programs depending on them for efficiency. However this brings problems. Every module evolves and can be built in many different ways, including version and build-time config choices; program developed against one concrete version of module can fail to work with other concrete version. Rust ecosystem seems to have almost entirely ditched runtime code sharing.
Note: in Rust world word "module" has different special meaning, but I use it here synonymically with "library"
Dependency substitution mechanisms can be classified by software lifecycle point at which concrete dep ver is picked, known in CS as dep binding time:
- runtime binding: concrete dep ver picked at runtime. It's considered not important which concrete versions of dep the program was developed and built against. Information about dep is loosely provided at some package management level, usually as name and major version. It is considered that any concrete version which satisfies this constraint should be ok and program should work well with it. This is mostly true for software which is developed within "C/Unix culture" where library devs care about API and ABI backwards compatibility, program devs expect their software to work with range of versions of dep, testing is performed accordingly by devs or maintainers, and issues caused by running with different concrete dep ver are treated as bugs by dev, software development being done cooperatively by library devs, program devs and distros maintainers who perform additional testing of various combinations and send bug reports upstream, individual libs and programs are seen not as final products to be delivered to user but as building blocks of larger system which have to be integrated by maintainers and may have to be polished in process. This is how e. g. (Debian-based part of) Ubuntu works (for most of software, incl. virtually all C/C++ and Python software). It works well almost always for "core" Linux software which belongs to that "culture", but something breaking because of dep replacement after sys upgrade is something which most long term users saw more then once (distros do lots of testing to avoid it, but it's impossible to test all possible combinations which might materialize on users machines), and more complex software with bigger "deps surface" breaks more often, which is one of primary reasons for Ubuntu pushing Snap and other distros similar container-like/immutable solutions for complex software, as well as immutable distros such as NixOS.
- build time binding: concrete dep ver picked at build time. It's considered not important which concrete version of dep the program was developed against, but it's considered important against which one it was built; installed instance of program always continues using same concrete versions of dependencies which it was built against, until it's upgraded/replaced with new instance built against new concrete versions of deps. It is expected that if program build succeeded against these concrete deps (and possibly passed tests), it should work well. This is how NixOS works (without flakes or within flakes set using same nixpkgs revision) (for most of software); in any nixpkgs revision, almost all software depending on some lib is built against same recent concrete ver of this lib which is "default" in this nixpkgs revision, sharing it in runtime, and only some software which is discovered to be broken with this concrete version of the lib gets "custom" version(s). Unimaginable number of combinations to be tested is reduced to some number of closures occurring in sequence of nixpkgs revisions picked for publication on stable channel. It still somewhat "blurs lines" in development, making it potentially possible that behavior of program on users machines differs from devs (in a way which satisfies maintainers), which might or might not be seen as problem, depending on mindset. I personally see it as practical sweet spot between efficiency and reliability for desktop OS; I'm using NixOS for many years and I never saw something breaking because concrete dependency ver chosen at build time (in stable nixpkgs channel) happened to be not good for program.
- development time binding: concrete dep ver picked at development time. It's considered important which concrete version of dep the program was developed against. When program is built from source, same concrete versions of deps specified by dev in build config are (fetched, built and) used. This is how Rust works (well, not Rust itself, but almost all Rust projects with their Cargo manifests with lock files). This maximizes reliability (reproducibility) and minimizes (eliminates) devs work for maintaining compatibility but effectively kills runtime code sharing. Probability of many programs in some env depending on same concrete dep ver is so low that in Rust ecosystem devs don't even bother to support building shared libs, unless project provides C FFI (
dylib exists, but good luck trying to use it to build some set of Rust software with notable modules built as shared libs).
Common tech solutions for runtime code sharing seem to grow from that "C/Unix culture" from 70s. This includes "dynamic linking", "dynamic linker" and "dynamic library" concepts. It's strongly associated with "runtime binding" and opposed by many Rust devs because of associated reliability problems which are growing pain with growing complexity of software. And actually historical implementation is really not perfect for "build time binding" ( https://github.com/NixOS/nixpkgs/issues/24844 abs paths in ELF DT_NEEDED? https://lwn.net/Articles/961117/ ).
There are more issues with Rust runtime code sharing:
- "no stable Rust ABI". Not a problem in "build time binding" case, also in C world there are problems with using together binaries and libs built with different versions of same toolchain, too; runtime code sharing for programs built with same version of Rust toolchain would be enough
- generics monomorphization. Lots of library code is generic code which can't be compiled to anything until specialized by caller. In practice, however, lots of generics are mostly specialized to small set of types; it should be possible to create tool which finds all specializations used by given set of programs and builds concrete ver of lib providing them all
- weak culture of development of modules with backwards compatibility even at API level and zero culture of developing programs which are tested against some range of dep versions
- significant part of community seems to have "extremely progressive mindset" with rejecting "old" solutions being part of their identity
But how much efficiency benefit? Maybe it's not really worth? My 1st Rust project is image viewer which I moved from SDL_Image to image-rs; binary grew from 30kb to 18Mb. I believe this overhead is obviously mostly code of Rust image decoders (some of which it could share with most desktop apps, some with almost none) and std lib parts. And it doesn't even use Rust libs for UI at this point. Better analysis is needed for numeric estimates for desktop system with typical set of apps, but for now I tend to think that difference would be significant...
https://www.reddit.com/r/rust/comments/13vkut6/shared_libraries/
https://github.com/pop-os/cosmic-epoch/issues/649 I wondered what if Cosmic desktop at least splits common code into some kind of "runtime". It doesn't
https://drewdevault.com/dynlib.html looks like typical "top few % get most (few % of dynamic libs are shared by most progs, libc obviously being the king), others get rest (most dynamic libs are actually not shared at all)", doesn't convince me that sharing is not worth. Yes, average benefit is very small, but average program is rarely executed, and those which run most of the time? Every web browser worker process getting dozens or hundreds Mb overhead?
Perhaps currently growing RAM shortage crisis will force community to reconsider...
Comments are not implemented, but you can create issue on GitHub or check existing ones
Return to index