Support prune in Bun workspaces

[joshunrau]

Which project is this feature idea for?

Turborepo

Describe the feature you'd like to request

Support turbo prune when using bun.

Describe the solution you'd like

Running turbo prune should work with bun the same as with npm.

Describe alternatives you've considered

Not using bun or not using turbo

[nathanhammond]

We're aware of this limitation. Serializing to the binary Bun lockfile format is a non-trivial task and not imminently forthcoming.

For now, if you want to use turbo prune for a Bun project, you can losslessly use it with either yarn or pnpm. We recommend choosing yarn because the node_modules layout is most-similar to Bun and it is more-likely to be compatible with your project.

cd ./path/to/bun/project
bunx @turbo/workspaces@latest convert ./ yarn --skip-install
turbo prune

If you don't want any of the automated changes to be permanent you can then proceed to revert all changes in the repository.

Note that the output pruned repository must use the package manager you select (yarn or pnpm)—it cannot use bun.

[jordanebelanger]

Note that the output pruned repository must use the package manager you select (yarn or pnpm)—it cannot use bun.

Doesn't bun support the yarn lockfile format? Why the need to switch package manager even after converting a bun.lockb to a yarn lockfile?

P.S would love for this issue to get attention, this is the last blocker for us to move over to bun as our package manager.

[chris-olszewski]

Bun supports serializing the bun.lockb to a yarn.lock to make it human readable, but it doesn't support using parsing a yarn.lock or creating a bun.lockb from a yarn.lock.

There's some chatter around bun getting support for using a package-lock.json and that could open a path forward to turbo prune supporting bun.

[wladpaiva]

Bun supports a human readable lock file now
https://bun.sh/guides/install/yarnlock

Maybe an easy fix would be to have an option for turbo to read specific lock files the same way bun does
turbo prune --yarn

[jukkamarttinen]

This would be the best option, if possible.

[thecrabcode]

Is there any update on this guys? We would also like to use Turborepo alongside bun, but this issue is quite important for us. I'd be happy to help in any way I can

[AirP0WeR]

+1 to this problem

[baptisteArno]

YES please 🙏

[XenoWad01]

I am also very interested in this being supported. My team would love to use bun alongside turbo but this is the main bottleneck preventing us from doing so.

[jukkamarttinen]

This would be lot appreciated for us too.

[mtsewrs]

This is the only thing blocking my company from switching to bun

[XenoWad01]

I am unsure if this actually fixed the problem as I didnt have time to test that the lock file works. But what seemed to have worked for me was doing turbo prune --scope=@bookd/bookd-api --docker; exit 0 which basically ignores the error alltogether. Im not sure if it works properly or not but it does generate an out/ folder properly. It does generate an out folder but I suspect it just contains the non pruned version of the lock file. Here is my docker file for reference:

FROM node:20.11.0 AS init


# RUN npm install -g unzip
RUN npm install -g bun

RUN bun upgrade --canary

RUN npm install -g synp
RUN npm install -g [email protected]


WORKDIR /usr/src/app
COPY . .

RUN turbo prune --scope=@bookd/bookd-api --docker; exit 0

FROM init AS installer

WORKDIR /usr/src/app


COPY --chown=node .gitignore .gitignore
COPY --from=init --chown=node /usr/src/app/out/json/ ./



COPY --from=init --chown=node /usr/src/app/out/full/ ./

RUN rm -rf /usr/src/app/out/
RUN bun install --frozen-lockfile --production
RUN bun run build

ENV PORT=4000

EXPOSE $PORT

WORKDIR /usr/src/app/packages/bookd-api

RUN chown node /usr/src/app/* && chmod 755 /usr/src/app/*

USER node
CMD bun run /usr/src/app/packages/bookd-api/app.ts

(Dont mind the npm stuf it seems bun install -g does not really work inside docker for me because the it doesent seem to add the commands to path inside docker only)

From what I understand the reason for this not being supported is that prune actually generates a new lockfile from the old one with only a certain package's dependencies in it and it cannot currently parse the .lockb format. Bun currently supports installing from lockb (and from package-lock.json if you are on canary as of this moment). Here is the open issue related to it supporting yarn.lock file: oven-sh/bun#6409. As soon as they support installing from a yarn.lock file this should become a non problem because bun install can generate a yarn.lock file with bun install -y or adding [install.lockfile] print = "yarn" to your bunfig.toml file at the root of your project. In theory you could also use https://github.com/imsnif/synp for converting the yarn.lock file to a package-lock.json file. Turbo seems to auto-detect the type of lockfile you have and will whine about you having multiple ones. What could fix this is a argument for turbo or turbo prune to specify the exact lockfile to use (as mentioned here #7456 (comment)). The current workaround proposed by me is that you just delete the lock files you dont need inside docker.

TLDR:

(be on bun canary) => install with bun install -y always or add the [install.lockfile] print = "yarn" to your bunfig.toml file => (inside docker) => install synp => delete .lockb file => convert yarn.lock to package-lock.json with snyp => remove old yarn.lock => you currently have a package-lock.json and if you are on canary you can install from it with bun.

I didnt yet have time to fully test this theory but if someone has time and manages to do so please let me know I would appreciate it alot. Otherwise I will try to come back to you when I properly pull this off ( which will take some time as I am new to docker and devops in general ).

[luksch42]

@XenoWad01 any updates?

[XenoWad01]

@terion-name Yea I am also confused. Here is my output I just ran docker build --pull -t trybun . &> build.log. I used the exact same dockerfile as above. And we got the same turbo version too ... I will look around again tomorrow if I get time.

#0 building with "default" instance using docker driver

#1 [internal] load .dockerignore
#1 transferring context:
#1 transferring context: 152B done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 751B done
#2 DONE 0.1s

#3 [internal] load metadata for docker.io/oven/bun:1.1.17
#3 DONE 0.8s

#4 [base 1/1] FROM docker.io/oven/bun:1.1.17@sha256:7b5c05b562efbbec704df426a9eef18f3b935bdc868678895a935bf120bda062
#4 DONE 0.0s

#5 [internal] load build context
#5 transferring context: 901.47kB 0.5s done
#5 DONE 0.5s

#6 [pruned 1/5] RUN bun add -g [email protected]
#6 CACHED

#7 [pruned 2/5] WORKDIR /usr/src/app
#7 CACHED

#8 [pruned 3/5] COPY . .
#8 DONE 1.1s

#9 [pruned 4/5] RUN rm ./bun.lockb
#9 DONE 0.6s

#10 [pruned 5/5] RUN bun run turbo prune --scope=@bookd/bookd-api --docker
#10 0.791 
#10 0.791 Attention:
#10 0.791 Turborepo now collects completely anonymous telemetry regarding usage.
#10 0.791 This information is used to shape the Turborepo roadmap and prioritize features.
#10 0.791 You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
#10 0.791 https://turbo.build/repo/docs/telemetry
#10 0.791 
#10 0.948 Generating pruned monorepo for @bookd/bookd-api in /usr/src/app/out
#10 0.978  - Added @bookd/bookd-api
#10 DONE 1.5s

#11 [installer 1/7] WORKDIR /usr/src/app
#11 DONE 0.2s

#12 [installer 2/7] COPY --chown=bun .gitignore .gitignore
#12 DONE 0.2s

#13 [installer 3/7] COPY --from=pruned --chown=bun /usr/src/app/out/json/ ./
#13 DONE 0.2s

#14 [installer 4/7] COPY --from=pruned --chown=bun /usr/src/app/out/full/ ./
#14 DONE 0.3s

#15 [installer 5/7] COPY --from=pruned --chown=bun /usr/src/app/out/yarn.lock ./yarn.lock
#15 DONE 0.2s

#16 [installer 6/7] RUN rm -rf /usr/src/app/out/
#16 DONE 0.6s

#17 [installer 7/7] RUN bun install --frozen-lockfile --production
#17 0.676 bun install v1.1.17 (bb66bba1)
#17 0.688 Resolving dependencies
#17 21.48 Resolved, downloaded and extracted [6547]
#17 21.48 warn: incorrect peer dependency "[email protected]"
#17 21.48 
#17 21.48 warn: incorrect peer dependency "[email protected]"
#17 27.14 Saved yarn.lock
#17 27.16 
#17 27.16 + [email protected]
#17 27.16 
#17 27.16 897 packages installed [26.49s]
#17 27.16 
#17 27.16 Blocked 1 postinstall. Run `bun pm untrusted` for details.
#17 DONE 28.1s

#18 exporting to image
#18 exporting layers

[XenoWad01]

@nzapponi I will also try to look for evidance of bun supporting --frozen-lockfile installing from yarn.lock.

[XenoWad01]

@nzapponi So according to this closed issue it should be able to install from a yarn.lock file: oven-sh/bun#1751. So if I am not missing something this should work.

[XenoWad01]

@terion-name I just checked and It seems I forgot to include that this is in my package.json with the current working setup. "packageManager": "[email protected]", And i just checked if I get the error you got if i instead use "packageManager": "[email protected]",. So that is what the difference is. So basically just add in that line you mentioned above before prunning: RUN json -I -f package.json -e "this.packageManager=\"[email protected]\"".

[XenoWad01]

Turbo basically throws that error based on what you have in package.json under "packageManager". Maybe they should just warn instead of erroring... seeing as you can actually prune bun workspaces if you do these workarounds. This would at least get us rid of the RUN json [...] command and make this solution less hacky until they will actually support bun lockfiles.

[nicholasoxford]

When deploying docker files to Fly.io, using a Turborepo, having the prune command makes life a lot easier. Especially as your monorepo grows in size

[terion-name]

Bun supports yarn.lock for very long time. It is a shame that it is not supported in turbo =(

[apex-woot]

+1
It would be great to have bun support prune in any form to be able to use turbo prune!

[maxktz]

Bun v1.2 officially has json lock files, will we get an update on that please?

[ruaanetwork]

any updates..

[jarvis394]

https://bun.sh/blog/bun-v1.1.39#bun-lock-is-bun-s-new-text-based-lockfile

In this release, we've added a new human-readable lockfile format: bun.lock. This makes git diffs and merge conflicts work better, and unblocks tooling like Dependabot, Renovate, Turbo prune, and more.

Really hoping to see an update on this 🚢

[baptisteArno]

No one subscribed to this thread have the skills to implement this?? 🤓

[jordanebelanger]

Wish I could help but personally, I have never been unemployed long enough to learn Rust 🥴

[baptisteArno]

Maybe Devin can do this? 🧌

[eduardofischer]

We're also looking forward to this 🙂

[cjkihl]

Any updates?

[camero2734]

I've taken a stab at implementation here: #10175

[skeie]

This will be awesome!

[anthonyshew]

@camero2734 Thank you for the contiribution! This is now released in v2.4.5-canary.5 and above!

[baptisteArno]

Amazing 🔥

[camero2734]

An update -- there was a small issue remaining mostly with peerDependencies (but also possibly with devDependencies) in v2.4.5-canary.5 -- I've made another fix in #10219

[lbxa]

Getting an issue installing a pruned workspace with turborepo @camero2734 Let me know if you've seen this before otherwise I can go ahead and create an issue for it.

Update: Even solving the above issue just leads to more...

FYI using node-linker=hoisted in .npmrc

[lbxa]

@camero2734 any ideas? ^ This is the last blocker before the mighty combo of bun + turbo can be production ready for me.

[JonasRothmann]

im having these issues again suddenly

[ArvidAnderson]

I also experience this issue.

[afonsocrg]

@martiinii I spent so much time trying to fix this, until I read your comment! It solved it!! Thank you

[anthonyshew]

The current issue to track with on these issues can be found here: #10658

[Andy-d-g]

[anthonyshew]

The tracking issue is right above this comment. 😄

The most recent canaries of Turborepo have a fix for the new Bun lockfile version. Please open an issue with a reproduction if you run into issues.

[lmyslinski]

I can confirm that using turbo@canary has fixed my issues.

❯ docker build . --progress=plain -t "paperjet"
#0 building with "orbstack" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.75kB done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/oven/bun:1.3.0
#2 DONE 0.2s

#3 [internal] load .dockerignore
#3 transferring context: 733B done
#3 DONE 0.0s

#4 [base 1/3] FROM docker.io/oven/bun:1.3.0@sha256:00cccad6e9c66bbacc250851f689168606aaea551ac473e908bbcf00a5645025
#4 DONE 0.0s

#5 [base 2/3] WORKDIR /usr/src/app
#5 CACHED

#6 [installer 1/4] WORKDIR /usr/src/app
#6 CACHED

#7 [base 3/3] RUN --mount=type=cache,target=/root/.bun/install/cache     bun install -g [email protected]
#7 CACHED

#8 [internal] load build context
#8 transferring context: 2.92MB 5.1s
#8 transferring context: 3.85MB 5.5s done
#8 DONE 5.6s

#9 [pruner 1/2] COPY . .
#9 DONE 9.8s

#10 [pruner 2/2] RUN turbo prune @paperjet/api @paperjet/dashboard --docker
#10 0.321  WARNING  No locally installed `turbo` found in your repository. Using globally installed version (2.5.9-canary.8), which can cause unexpected behavior.
#10 0.321
#10 0.321 Installing the version in your repository (2.5.9-canary.8) before calling `turbo` will result in more predictable behavior across environments.
#10 0.326
#10 0.326 Attention:
#10 0.326 Turborepo now collects completely anonymous telemetry regarding usage.
#10 0.326 This information is used to shape the Turborepo roadmap and prioritize features.
#10 0.326 You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
#10 0.326 https://turborepo.com/docs/telemetry
#10 0.326
#10 0.327 turbo 2.5.9-canary.8
#10 0.327
#10 0.390 Generating pruned monorepo for @paperjet/api, @paperjet/dashboard in /usr/src/app/out
#10 0.418  - Added @paperjet/api
#10 0.428  - Added @paperjet/auth
#10 0.770  - Added @paperjet/dashboard
#10 0.801  - Added @paperjet/db
#10 0.809  - Added @paperjet/email
#10 0.822  - Added @paperjet/engine
#10 0.844  - Added @paperjet/queue
#10 0.850  - Added @paperjet/shared
#10 0.850  - Added @paperjet/tsconfig
#10 DONE 1.6s

#11 [installer 2/4] COPY --from=pruner /usr/src/app/out/json/ .
#11 DONE 0.1s

#12 [builder 2/4] COPY --from=pruner /usr/src/app/out/full/ .
#12 ...

#13 [installer 3/4] COPY --from=pruner /usr/src/app/out/bun.lock ./bun.lock
#13 DONE 0.1s

#14 [installer 4/4] RUN --mount=type=cache,target=/root/.bun/install/cache     bun install
#14 0.094 bun install v1.3.0 (b0a6feca)
#14 ...

#12 [builder 2/4] COPY --from=pruner /usr/src/app/out/full/ .
#12 DONE 4.0s

#14 [installer 4/4] RUN --mount=type=cache,target=/root/.bun/install/cache     bun install

See https://github.com/MLNativeAI/paperjet/ for reference.

[frixou89]

This worked for me.

bun install --linker hoisted without --frozen-lockfile flag

FROM oven/bun:1.3.1-alpine AS base
 
FROM base AS builder
WORKDIR /app
RUN bun install --global turbo@^2
COPY . .
 
RUN turbo prune api --docker
 
FROM base AS installer
WORKDIR /app
 
COPY --from=builder /app/out/json/ .
RUN bun install --linker hoisted
 
COPY --from=builder /app/out/full/ .
RUN bun run turbo build
 
FROM base AS runner
WORKDIR /app
 
COPY --from=installer /app/apps/api/dist .

EXPOSE 3000

ENTRYPOINT ["./app"]

[lbxa]

@frixou89 weird it still doesn't work for me. I followed these steps on turbo@canary.

bun turbo prune <package> --docker
cd out/json
bun install --linker hoisted

Got this output:

bun install --linker hoisted                   
bun install v1.3.3 (274e01c7)
1889 |     "express": ["[email protected]", "", { "dependencies": { "accepts": "1.3.8", "array
           ^
error: Failed to resolve prod dependency 'cookie' for package 'express'
    at bun.lock:1889:5
InvalidPackageInfo: failed to parse lockfile: 'bun.lock'

warn: Ignoring lockfile